-- Logs begin at Mon 2022-08-22 14:57:40 UTC, end at Mon 2022-08-22 16:21:01 UTC. -- Aug 22 15:45:48.267998 dv-cinder-842541-5-iscsi systemd[1]: Starting Devstack devstack@keystone.service... Aug 22 15:45:48.277862 dv-cinder-842541-5-iscsi devstack@keystone.service[81755]: [uWSGI] getting INI configuration from /etc/keystone/keystone-uwsgi-public.ini Aug 22 15:45:48.280119 dv-cinder-842541-5-iscsi devstack@keystone.service[81755]: *** Starting uWSGI 2.0.18-debian (64bit) on [Mon Aug 22 15:45:48 2022] *** Aug 22 15:45:48.280119 dv-cinder-842541-5-iscsi devstack@keystone.service[81755]: compiled with version: 10.0.1 20200405 (experimental) [master revision 0be9efad938:fcb98e4978a:705510a708d3642c9c962beb663c476167e4e8a4] on 11 April 2020 11:15:55 Aug 22 15:45:48.280119 dv-cinder-842541-5-iscsi devstack@keystone.service[81755]: os: Linux-5.4.0-72-generic #80-Ubuntu SMP Mon Apr 12 17:35:00 UTC 2021 Aug 22 15:45:48.280119 dv-cinder-842541-5-iscsi devstack@keystone.service[81755]: nodename: dv-cinder-842541-5-iscsi Aug 22 15:45:48.280119 dv-cinder-842541-5-iscsi devstack@keystone.service[81755]: machine: x86_64 Aug 22 15:45:48.280119 dv-cinder-842541-5-iscsi devstack@keystone.service[81755]: clock source: unix Aug 22 15:45:48.280119 dv-cinder-842541-5-iscsi devstack@keystone.service[81755]: pcre jit disabled Aug 22 15:45:48.280119 dv-cinder-842541-5-iscsi devstack@keystone.service[81755]: detected number of CPU cores: 4 Aug 22 15:45:48.280119 dv-cinder-842541-5-iscsi devstack@keystone.service[81755]: current working directory: / Aug 22 15:45:48.280119 dv-cinder-842541-5-iscsi devstack@keystone.service[81755]: detected binary path: /usr/bin/uwsgi-core Aug 22 15:45:48.280119 dv-cinder-842541-5-iscsi devstack@keystone.service[81755]: your processes number limit is 47680 Aug 22 15:45:48.280119 dv-cinder-842541-5-iscsi devstack@keystone.service[81755]: your memory page size is 4096 bytes Aug 22 15:45:48.280119 dv-cinder-842541-5-iscsi devstack@keystone.service[81755]: detected max file descriptor number: 2048 Aug 22 15:45:48.280119 dv-cinder-842541-5-iscsi devstack@keystone.service[81755]: lock engine: pthread robust mutexes Aug 22 15:45:48.280928 dv-cinder-842541-5-iscsi devstack@keystone.service[81755]: thunder lock: enabled Aug 22 15:45:48.280928 dv-cinder-842541-5-iscsi devstack@keystone.service[81755]: uwsgi socket 0 bound to UNIX address /var/run/uwsgi/keystone-wsgi-public.socket fd 4 Aug 22 15:45:48.280928 dv-cinder-842541-5-iscsi devstack@keystone.service[81755]: Python version: 3.8.10 (default, Jun 22 2022, 20:18:18) [GCC 9.4.0] Aug 22 15:45:48.320477 dv-cinder-842541-5-iscsi devstack@keystone.service[81755]: Python main interpreter initialized at 0x55e589d729c0 Aug 22 15:45:48.320477 dv-cinder-842541-5-iscsi devstack@keystone.service[81755]: python threads support enabled Aug 22 15:45:48.320477 dv-cinder-842541-5-iscsi devstack@keystone.service[81755]: your server socket listen backlog is limited to 100 connections Aug 22 15:45:48.320477 dv-cinder-842541-5-iscsi devstack@keystone.service[81755]: your mercy for graceful operations on workers is 90 seconds Aug 22 15:45:48.320871 dv-cinder-842541-5-iscsi devstack@keystone.service[81755]: mapped 403077 bytes (393 KB) for 2 cores Aug 22 15:45:48.320950 dv-cinder-842541-5-iscsi devstack@keystone.service[81755]: *** Operational MODE: preforking *** Aug 22 15:45:48.321193 dv-cinder-842541-5-iscsi devstack@keystone.service[81755]: *** uWSGI is running in multiple interpreter mode *** Aug 22 15:45:48.321193 dv-cinder-842541-5-iscsi devstack@keystone.service[81755]: spawned uWSGI master process (pid: 81755) Aug 22 15:45:48.321409 dv-cinder-842541-5-iscsi systemd[1]: Started Devstack devstack@keystone.service. Aug 22 15:45:48.322549 dv-cinder-842541-5-iscsi devstack@keystone.service[81755]: spawned uWSGI worker 1 (pid: 81767, cores: 1) Aug 22 15:45:48.322549 dv-cinder-842541-5-iscsi devstack@keystone.service[81755]: spawned uWSGI worker 2 (pid: 81768, cores: 1) Aug 22 15:45:48.322549 dv-cinder-842541-5-iscsi devstack@keystone.service[81755]: running "unix_signal:15 gracefully_kill_them_all" (master-start)... Aug 22 15:45:52.765855 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Aug 22 15:45:52.765855 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: from cryptography.utils import int_from_bytes Aug 22 15:45:52.766671 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Aug 22 15:45:52.766671 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: from cryptography.utils import int_from_bytes Aug 22 15:45:52.766992 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Aug 22 15:45:52.766992 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: from cryptography.utils import int_from_bytes Aug 22 15:45:52.767490 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Aug 22 15:45:52.767490 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: from cryptography.utils import int_from_bytes Aug 22 15:45:52.816879 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Aug 22 15:45:52.817062 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=81768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Aug 22 15:45:52.817242 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG uwsgi [-] command line args: [] {{(pid=81768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Aug 22 15:45:52.817510 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=81768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Aug 22 15:45:52.817685 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG uwsgi [-] ================================================================================ {{(pid=81768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Aug 22 15:45:52.817937 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG uwsgi [-] admin_token = **** {{(pid=81768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 22 15:45:52.818071 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Aug 22 15:45:52.818182 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG uwsgi [-] config_dir = [] {{(pid=81768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 22 15:45:52.818267 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=81767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Aug 22 15:45:52.818369 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=81768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 22 15:45:52.818447 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG uwsgi [-] command line args: [] {{(pid=81767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Aug 22 15:45:52.818548 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG uwsgi [-] config_source = [] {{(pid=81768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 22 15:45:52.818704 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=81767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Aug 22 15:45:52.818789 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG uwsgi [-] debug = True {{(pid=81768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 22 15:45:52.818870 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG uwsgi [-] ================================================================================ {{(pid=81767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Aug 22 15:45:52.818947 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=81768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 22 15:45:52.819161 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG uwsgi [-] admin_token = **** {{(pid=81767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 22 15:45:52.819325 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=81768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 22 15:45:52.819474 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG uwsgi [-] config_dir = [] {{(pid=81767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 22 15:45:52.819590 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG uwsgi [-] insecure_debug = False {{(pid=81768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 22 15:45:52.819698 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=81767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 22 15:45:52.819800 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=81768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 22 15:45:52.819896 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG uwsgi [-] config_source = [] {{(pid=81767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 22 15:45:52.819896 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG uwsgi [-] debug = True {{(pid=81767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 22 15:45:52.820085 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=81768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 22 15:45:52.820218 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=81767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 22 15:45:52.820356 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG uwsgi [-] list_limit = None {{(pid=81768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 22 15:45:52.820453 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=81767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 22 15:45:52.820562 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG uwsgi [-] log_config_append = None {{(pid=81768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 22 15:45:52.820664 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG uwsgi [-] insecure_debug = False {{(pid=81767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 22 15:45:52.820765 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=81768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 22 15:45:52.820765 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG uwsgi [-] log_dir = None {{(pid=81768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 22 15:45:52.821019 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=81767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 22 15:45:52.821151 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG uwsgi [-] log_file = None {{(pid=81768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 22 15:45:52.821415 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=81767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 22 15:45:52.821520 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=81768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 22 15:45:52.821642 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG uwsgi [-] list_limit = None {{(pid=81767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 22 15:45:52.821642 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG uwsgi [-] log_config_append = None {{(pid=81767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 22 15:45:52.821813 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=81768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 22 15:45:52.821982 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=81767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 22 15:45:52.822065 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=81768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 22 15:45:52.822142 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG uwsgi [-] log_dir = None {{(pid=81767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 22 15:45:52.822266 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=81768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 22 15:45:52.822351 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG uwsgi [-] log_file = None {{(pid=81767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 22 15:45:52.822428 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=81768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 22 15:45:52.822547 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=81767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 22 15:45:52.822629 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=81768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 22 15:45:52.822707 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=81767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 22 15:45:52.822866 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=81768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 22 15:45:52.822950 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=81767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 22 15:45:52.823043 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=81768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 22 15:45:52.823113 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=81767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 22 15:45:52.823239 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=81768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 22 15:45:52.823349 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=81767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 22 15:45:52.823450 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=81768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 22 15:45:52.823530 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=81767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 22 15:45:52.823609 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=81768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 22 15:45:52.823726 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=81767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 22 15:45:52.823802 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=81768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 22 15:45:52.823923 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=81767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 22 15:45:52.824007 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=81768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 22 15:45:52.824085 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=81767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 22 15:45:52.824200 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG uwsgi [-] notification_format = cadf {{(pid=81768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 22 15:45:52.824281 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=81767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 22 15:45:52.824353 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=81768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 22 15:45:52.824433 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=81767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 22 15:45:52.824575 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=81767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 22 15:45:52.825310 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.15/identity {{(pid=81768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 22 15:45:52.825440 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=81767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 22 15:45:52.825581 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG uwsgi [-] publish_errors = False {{(pid=81768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 22 15:45:52.825657 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG uwsgi [-] notification_format = cadf {{(pid=81767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 22 15:45:52.825727 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=81768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 22 15:45:52.825889 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=81767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 22 15:45:52.825974 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=81768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 22 15:45:52.826095 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=81768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 22 15:45:52.826330 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=81768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 22 15:45:52.826408 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.15/identity {{(pid=81767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 22 15:45:52.826557 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG uwsgi [-] publish_errors = False {{(pid=81767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 22 15:45:52.826676 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=81768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 22 15:45:52.826791 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG uwsgi [-] standard_threads = False {{(pid=81768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 22 15:45:52.826874 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=81767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 22 15:45:52.827023 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG uwsgi [-] strict_password_check = False {{(pid=81768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 22 15:45:52.827102 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=81767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 22 15:45:52.827202 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=81767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 22 15:45:52.827320 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=81768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 22 15:45:52.827454 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG uwsgi [-] use_eventlog = False {{(pid=81768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 22 15:45:52.827530 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=81767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 22 15:45:52.827640 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG uwsgi [-] use_journal = False {{(pid=81768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 22 15:45:52.827790 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG uwsgi [-] use_json = False {{(pid=81768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 22 15:45:52.827876 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=81767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 22 15:45:52.827954 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG uwsgi [-] use_stderr = False {{(pid=81768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 22 15:45:52.828031 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG uwsgi [-] standard_threads = False {{(pid=81767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 22 15:45:52.828124 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG uwsgi [-] use_syslog = False {{(pid=81768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 22 15:45:52.828259 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG uwsgi [-] strict_password_check = False {{(pid=81767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 22 15:45:52.828359 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG uwsgi [-] watch_log_file = False {{(pid=81768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 22 15:45:52.828514 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=81767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 22 15:45:52.828619 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=81768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 15:45:52.828752 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG uwsgi [-] use_eventlog = False {{(pid=81767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 22 15:45:52.828832 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG uwsgi [-] use_journal = False {{(pid=81767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 22 15:45:52.828902 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=81768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 15:45:52.829003 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG uwsgi [-] use_json = False {{(pid=81767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 22 15:45:52.829108 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=81768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 15:45:52.829187 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG uwsgi [-] use_stderr = False {{(pid=81767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 22 15:45:52.829346 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG uwsgi [-] use_syslog = False {{(pid=81767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 22 15:45:52.829429 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=81768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 15:45:52.829582 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG uwsgi [-] watch_log_file = False {{(pid=81767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 22 15:45:52.829664 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=81768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 15:45:52.829849 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=81768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 15:45:52.829931 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=81767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 15:45:52.830078 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=81767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 15:45:52.830193 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=81768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 15:45:52.830316 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=81768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 15:45:52.830396 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=81767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 15:45:52.830567 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=81768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 15:45:52.830670 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=81767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 15:45:52.830816 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=81768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 15:45:52.830900 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=81767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 15:45:52.831057 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=81767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 15:45:52.831134 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=81768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 15:45:52.831310 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=81768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 15:45:52.831402 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=81767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 15:45:52.831562 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=81768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 15:45:52.831640 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=81767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 15:45:52.831799 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=81768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 15:45:52.831880 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=81767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 15:45:52.832048 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=81768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 15:45:52.832126 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=81767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 15:45:52.832298 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=81767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 15:45:52.832375 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=81768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 15:45:52.832534 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=81768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 15:45:52.832617 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=81767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 15:45:52.832783 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=81767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 15:45:52.832861 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=81768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 15:45:52.833026 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=81767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 15:45:52.833102 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=81768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 15:45:52.833270 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=81768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 15:45:52.833352 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=81767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 15:45:52.833507 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=81768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 15:45:52.833584 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=81767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 15:45:52.833752 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=81767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 15:45:52.833829 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=81768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 15:45:52.834011 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=81768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 15:45:52.834093 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=81767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 15:45:52.834265 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=81768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 15:45:52.834348 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=81767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 15:45:52.834514 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=81767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 15:45:52.834595 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=81768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 15:45:52.834745 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=81768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 15:45:52.834830 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=81767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 15:45:52.834977 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=81768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 15:45:52.835054 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=81767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 15:45:52.835208 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=81768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 15:45:52.835305 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=81767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 15:45:52.835468 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=81768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 15:45:52.835558 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=81767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 15:45:52.835719 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=81768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 15:45:52.835808 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=81767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 15:45:52.835990 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=81767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 15:45:52.836099 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=81768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 15:45:52.836228 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=81767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 15:45:52.836460 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=81768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 15:45:52.836539 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=81767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 15:45:52.836796 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=81767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 15:45:52.836944 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=81768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 15:45:52.837027 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=81767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 15:45:52.837241 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=81768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 15:45:52.837336 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=81767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 15:45:52.837553 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=81767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 15:45:52.837693 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=81768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 15:45:52.837793 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=81767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 15:45:52.838994 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=81768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 15:45:52.838994 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=81768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 15:45:52.839173 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=81767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 15:45:52.839241 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=81768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 15:45:52.839331 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=81767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 15:45:52.839490 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=81768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 15:45:52.839574 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=81767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 15:45:52.839727 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=81768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 15:45:52.839824 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=81767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 15:45:52.840084 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=81768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 15:45:52.840498 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=81768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 15:45:52.840575 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=81767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 15:45:52.840837 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=81767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 15:45:52.840949 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=81768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 15:45:52.841086 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=81767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 15:45:52.841263 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=81768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 15:45:52.841340 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=81767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 15:45:52.841555 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=81767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 15:45:52.841649 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=81768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 15:45:52.841795 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=81767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 15:45:52.841999 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=81768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 15:45:52.842102 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=81767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 15:45:52.842267 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=81767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 15:45:52.842345 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=81768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 15:45:52.842525 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=81767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 15:45:52.842704 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=81768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 15:45:52.842781 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=81767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 15:45:52.842995 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=81767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 15:45:52.843074 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=81768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 15:45:52.843234 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=81767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 15:45:52.843477 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=81768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 15:45:52.843591 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=81767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 15:45:52.843733 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=81767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 15:45:52.843858 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=81768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 15:45:52.843990 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=81767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 15:45:52.844234 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=81767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 15:45:52.844312 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=81768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 15:45:52.844481 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=81767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 15:45:52.844653 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=81768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 15:45:52.844775 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=81767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 15:45:52.844954 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=81767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 15:45:52.845098 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=81768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 15:45:52.845239 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=81767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 15:45:52.845424 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=81767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 15:45:52.845529 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=81768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 15:45:52.845661 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=81767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 15:45:52.845918 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=81767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 15:45:52.846000 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=81768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 15:45:52.846168 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=81767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 15:45:52.846403 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=81767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 15:45:52.846481 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=81768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 15:45:52.846640 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=81767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 15:45:52.846862 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=81768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 15:45:52.846949 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=81767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 15:45:52.847136 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=81767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 15:45:52.847333 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=81768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 15:45:52.847414 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=81767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 15:45:52.847642 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=81767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 15:45:52.847733 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=81768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 15:45:52.847870 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=81767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 15:45:52.848112 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=81768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 15:45:52.848192 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=81767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 15:45:52.848377 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=81767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 15:45:52.848491 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=81768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 15:45:52.848632 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=81767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 15:45:52.848869 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=81768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 15:45:52.848959 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=81767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 15:45:52.849108 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=81767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 15:45:52.849274 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=81768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 15:45:52.849432 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=81767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 15:45:52.849576 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=81767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 15:45:52.849654 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=81768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 15:45:52.849828 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG uwsgi [-] cache.enabled = True {{(pid=81767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 15:45:52.850014 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=81768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 15:45:52.850091 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=81767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 15:45:52.850298 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=81767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 15:45:52.850377 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=81768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 15:45:52.850546 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=81767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 15:45:52.850728 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=81768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 15:45:52.850807 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=81767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 15:45:52.851010 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=81767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 15:45:52.851159 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=81768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 15:45:52.851239 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=81767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 15:45:52.851502 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=81767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 15:45:52.851583 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=81768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 15:45:52.851742 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=81767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 15:45:52.851929 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=81768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 15:45:52.852007 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=81767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 15:45:52.852236 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=81767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 15:45:52.852347 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=81768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 15:45:52.852479 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=81767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 15:45:52.852721 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=81767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 15:45:52.852800 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=81768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 15:45:52.852966 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=81767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 15:45:52.853089 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=81768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 15:45:52.853208 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=81767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 15:45:52.853446 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=81768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 15:45:52.853528 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=81767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 15:45:52.853683 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=81767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 15:45:52.853819 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG uwsgi [-] cache.enabled = True {{(pid=81768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 15:45:52.853957 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=81767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 15:45:52.854161 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=81767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 15:45:52.854241 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=81768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 15:45:52.854396 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=81767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 15:45:52.854571 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=81768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 15:45:52.854750 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=81767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 15:45:52.854991 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=81767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 15:45:52.855077 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=81768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 15:45:52.855239 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=81767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 15:45:52.855486 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=81768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 15:45:52.855567 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=81767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 15:45:52.855747 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=81767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 15:45:52.855950 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=81768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 15:45:52.856050 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=81767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 15:45:52.856190 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=81768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 15:45:52.856271 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=81767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 15:45:52.856438 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=81768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 15:45:52.856517 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=81767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 15:45:52.856685 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=81768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 15:45:52.856766 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=81767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 15:45:52.856956 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=81768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 15:45:52.857025 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG uwsgi [-] auth.external = None {{(pid=81767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 15:45:52.857200 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG uwsgi [-] auth.mapped = None {{(pid=81767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 15:45:52.857278 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=81768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 15:45:52.857444 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=81768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 15:45:52.857523 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=81767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 15:45:52.857693 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=81768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 15:45:52.857821 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=81767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 15:45:52.857937 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=81768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 15:45:52.858051 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG uwsgi [-] auth.password = None {{(pid=81767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 15:45:52.858171 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG uwsgi [-] auth.token = None {{(pid=81767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 15:45:52.858257 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=81768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 15:45:52.858418 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=81768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 15:45:52.858549 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=81767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 15:45:52.858635 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG uwsgi [-] catalog.caching = True {{(pid=81767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 15:45:52.858708 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=81768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 15:45:52.858879 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=81768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 15:45:52.858969 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=81767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 15:45:52.859111 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=81768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 15:45:52.859190 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=81767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 15:45:52.859372 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=81768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 15:45:52.859463 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=81767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 15:45:52.859609 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=81768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 15:45:52.859704 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=81767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 15:45:52.859849 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=81768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 15:45:52.859970 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=81767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 15:45:52.860088 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG uwsgi [-] credential.caching = True {{(pid=81767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 15:45:52.860159 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=81768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 15:45:52.860323 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=81768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 15:45:52.860400 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG uwsgi [-] credential.driver = sql {{(pid=81767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 15:45:52.860575 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=81768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 15:45:52.860653 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=81767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 15:45:52.860815 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=81768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 15:45:52.860916 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=81767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 15:45:52.861166 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=81768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 15:45:52.861309 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=81767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 15:45:52.861406 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=81768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 15:45:52.861481 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=81767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 15:45:52.861575 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=81768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 15:45:52.861715 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=81767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 15:45:52.861795 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG uwsgi [-] auth.external = None {{(pid=81768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 15:45:52.861976 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=81767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 15:45:52.862062 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG uwsgi [-] auth.mapped = None {{(pid=81768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 15:45:52.862290 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=81767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 15:45:52.862448 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=81768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 15:45:52.862617 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=81767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 15:45:52.862730 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=81768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 15:45:52.862809 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=81767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 15:45:52.862890 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG uwsgi [-] auth.password = None {{(pid=81768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 15:45:52.863007 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=81767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 15:45:52.863086 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG uwsgi [-] auth.token = None {{(pid=81768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 15:45:52.863260 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=81767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 15:45:52.863353 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=81768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 15:45:52.863545 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG uwsgi [-] catalog.caching = True {{(pid=81768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 15:45:52.863636 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=81767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 15:45:52.863794 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=81767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 15:45:52.863873 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=81768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 15:45:52.864026 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=81768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 15:45:52.864107 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=81767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 15:45:52.864330 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=81768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 15:45:52.864431 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG uwsgi [-] federation.caching = True {{(pid=81767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 15:45:52.864506 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=81767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 15:45:52.864576 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=81768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 15:45:52.864749 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=81768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 15:45:52.864826 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG uwsgi [-] federation.driver = sql {{(pid=81767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 15:45:52.864988 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG uwsgi [-] credential.caching = True {{(pid=81768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 15:45:52.865145 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=81767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 15:45:52.865220 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=81767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 15:45:52.865289 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG uwsgi [-] credential.driver = sql {{(pid=81768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 15:45:52.865459 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=81767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 15:45:52.865538 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=81768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 15:45:52.865701 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=81767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 15:45:52.865787 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=81768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 15:45:52.866039 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=81768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 15:45:52.866132 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=81767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 15:45:52.866194 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=81768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 15:45:52.866252 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=81767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 15:45:52.866419 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=81768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 15:45:52.866508 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=81767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 15:45:52.866676 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=81768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 15:45:52.866760 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=81767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 15:45:52.866935 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=81768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 15:45:52.867022 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=81767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 15:45:52.867175 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=81768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 15:45:52.867300 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG uwsgi [-] identity.caching = True {{(pid=81767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 15:45:52.867461 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=81768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 15:45:52.867549 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=81767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 15:45:52.867749 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=81768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 15:45:52.867838 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=81767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 15:45:52.867998 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=81768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 15:45:52.868097 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=81767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 15:45:52.868255 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=81767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 15:45:52.868340 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=81768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 15:45:52.868497 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=81768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 15:45:52.868582 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG uwsgi [-] identity.driver = sql {{(pid=81767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 15:45:52.868737 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=81767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 15:45:52.868882 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=81768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 15:45:52.868997 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=81767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 15:45:52.869104 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG uwsgi [-] federation.caching = True {{(pid=81768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 15:45:52.869209 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=81768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 15:45:52.869312 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=81767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 15:45:52.869431 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG uwsgi [-] federation.driver = sql {{(pid=81768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 15:45:52.869540 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=81767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 15:45:52.869679 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=81768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 15:45:52.869800 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=81767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 15:45:52.869907 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=81767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 15:45:52.870009 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=81768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 15:45:52.870124 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=81767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 15:45:52.870231 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=81768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 15:45:52.870370 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=81767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 15:45:52.870483 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=81768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 15:45:52.870623 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=81767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 15:45:52.870734 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=81768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 15:45:52.870852 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=81768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 15:45:52.870968 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=81767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 15:45:52.871083 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=81768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 15:45:52.871190 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=81767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 15:45:52.871383 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=81768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 15:45:52.871532 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=81767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 15:45:52.872195 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=81768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 15:45:52.872312 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=81767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 15:45:52.872411 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG uwsgi [-] identity.caching = True {{(pid=81768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 15:45:52.872508 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=81767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 15:45:52.872604 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=81768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 15:45:52.872701 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=81767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 15:45:52.872888 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=81768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 15:45:52.872990 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=81767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 15:45:52.873131 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=81768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 15:45:52.873234 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=81767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 15:45:52.873333 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=81768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 15:45:52.873443 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=81767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 15:45:52.873555 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=81767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 15:45:52.873650 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG uwsgi [-] identity.driver = sql {{(pid=81768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 15:45:52.873758 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=81768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 15:45:52.873866 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=81767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 15:45:52.873979 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=81768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 15:45:52.874092 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=81767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 15:45:52.874210 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=81768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 15:45:52.874315 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=81767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 15:45:52.874435 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=81768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 15:45:52.874540 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=81767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 15:45:52.874684 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=81768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 15:45:52.874809 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=81767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 15:45:52.874939 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=81768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 15:45:52.875056 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=81767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 15:45:52.875172 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=81768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 15:45:52.875351 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=81767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 15:45:52.875535 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=81768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 15:45:52.875668 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=81767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 15:45:52.875799 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=81768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 15:45:52.875914 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=81767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 15:45:52.876022 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=81768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 15:45:52.876133 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=81767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 15:45:52.876133 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=81767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 15:45:52.876340 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=81768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 15:45:52.876454 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG uwsgi [-] ldap.password = **** {{(pid=81767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 15:45:52.876549 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=81768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 15:45:52.876638 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=81767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 15:45:52.876727 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=81768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 15:45:52.876815 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=81767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 15:45:52.876903 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=81768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 15:45:52.877002 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=81767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 15:45:52.877093 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=81768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 15:45:52.877366 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=81767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 15:45:52.877553 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=81768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 15:45:52.877729 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=81767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 15:45:52.877895 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=81768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 15:45:52.878072 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=81767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 15:45:52.878240 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=81768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 15:45:52.878240 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=81768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 15:45:52.878551 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=81767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 15:45:52.878861 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=81768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 15:45:52.879204 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=81767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 15:45:52.879204 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=81767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 15:45:52.879577 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=81768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 15:45:52.879745 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=81767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 15:45:52.879927 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=81768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 15:45:52.879927 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=81768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 15:45:52.880258 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=81767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 15:45:52.880437 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=81768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 15:45:52.880604 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=81767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 15:45:52.880781 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=81768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 15:45:52.880954 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=81767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 15:45:52.881129 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=81768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 15:45:52.881306 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=81767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 15:45:52.881472 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=81768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 15:45:52.881646 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG uwsgi [-] ldap.user = None {{(pid=81767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 15:45:52.881821 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=81768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 15:45:52.881988 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=81767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 15:45:52.882170 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=81768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 15:45:52.882350 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=81767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 15:45:52.882531 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=81768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 15:45:52.882698 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=81767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 15:45:52.882864 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG uwsgi [-] ldap.password = **** {{(pid=81768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 15:45:52.883028 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=81767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 15:45:52.883195 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=81768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 15:45:52.883477 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=81767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 15:45:52.883648 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=81768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 15:45:52.884005 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=81767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 15:45:52.884174 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=81768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 15:45:52.884476 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=81767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 15:45:52.884678 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=81768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 15:45:52.884889 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=81767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 15:45:52.885061 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=81768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 15:45:52.885242 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=81767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 15:45:52.885411 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=81768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 15:45:52.885594 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=81767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 15:45:52.885762 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=81768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 15:45:52.885944 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=81767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 15:45:52.886111 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=81768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 15:45:52.886341 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=81767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 15:45:52.886510 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=81768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 15:45:52.886693 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=81767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 15:45:52.886869 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=81768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 15:45:52.887052 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=81767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 15:45:52.887236 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=81768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 15:45:52.887236 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=81768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 15:45:52.887580 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=81767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 15:45:52.887762 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=81768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 15:45:52.887942 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=81767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 15:45:52.888124 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=81768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 15:45:52.888291 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=81767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 15:45:52.888465 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG uwsgi [-] ldap.user = None {{(pid=81768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 15:45:52.888630 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=81767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 15:45:52.889570 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=81768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 15:45:52.889742 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=81767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 15:45:52.889922 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=81768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 15:45:52.890093 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=81767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 15:45:52.890258 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=81768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 15:45:52.890476 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=81767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 15:45:52.890476 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=81767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 15:45:52.890825 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=81768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 15:45:52.891004 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=81767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 15:45:52.891576 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=81768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 15:45:52.891750 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=81767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 15:45:52.891924 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=81768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 15:45:52.892091 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=81767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 15:45:52.892258 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=81768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 15:45:52.892258 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=81768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 15:45:52.892585 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=81767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 15:45:52.892751 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=81768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 15:45:52.892929 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG uwsgi [-] policy.driver = sql {{(pid=81767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 15:45:52.893098 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=81768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 15:45:52.893318 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=81767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 15:45:52.893652 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=81768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 15:45:52.893847 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=81767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 15:45:52.894020 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=81768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 15:45:52.894191 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=81767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 15:45:52.894370 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=81768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 15:45:52.894588 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG uwsgi [-] receipt.caching = True {{(pid=81767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 15:45:52.894808 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=81768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 15:45:52.894986 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=81767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 15:45:52.895151 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=81768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 15:45:52.895336 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=81767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 15:45:52.895502 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=81768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 15:45:52.895675 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=81767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 15:45:52.895939 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=81768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 15:45:52.896109 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=81767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 15:45:52.896273 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=81768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 15:45:52.896484 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=81767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 15:45:52.896680 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=81768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 15:45:52.896850 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG uwsgi [-] resource.caching = True {{(pid=81767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 15:45:52.897025 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=81768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 15:45:52.897195 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=81767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 15:45:52.897390 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=81768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 15:45:52.897556 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG uwsgi [-] resource.driver = sql {{(pid=81767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 15:45:52.897721 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=81768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 15:45:52.897884 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=81767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 15:45:52.898085 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=81768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 15:45:52.898085 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=81768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 15:45:52.898395 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=81767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 15:45:52.898586 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=81768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 15:45:52.898755 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=81767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 15:45:52.898931 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=81768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 15:45:52.899136 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG uwsgi [-] revoke.caching = True {{(pid=81767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 15:45:52.899389 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG uwsgi [-] policy.driver = sql {{(pid=81768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 15:45:52.899559 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=81767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 15:45:52.899724 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=81768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 15:45:52.899724 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=81768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 15:45:52.900033 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=81767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 15:45:52.900210 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=81768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 15:45:52.900386 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG uwsgi [-] role.cache_time = None {{(pid=81767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 15:45:52.900567 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG uwsgi [-] receipt.caching = True {{(pid=81768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 15:45:52.900733 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG uwsgi [-] role.caching = True {{(pid=81767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 15:45:52.900926 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=81768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 15:45:52.901090 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG uwsgi [-] role.driver = sql {{(pid=81767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 15:45:52.901278 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=81768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 15:45:52.901443 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG uwsgi [-] role.list_limit = None {{(pid=81767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 15:45:52.901608 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=81768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 15:45:52.901772 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=81767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 15:45:52.901947 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=81768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 15:45:52.902142 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=81767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 15:45:52.902352 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=81768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 15:45:52.902541 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=81767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 15:45:52.902911 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG uwsgi [-] resource.caching = True {{(pid=81768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 15:45:52.903123 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=81767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 15:45:52.903363 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=81768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 15:45:52.903571 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=81767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 15:45:52.903758 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG uwsgi [-] resource.driver = sql {{(pid=81768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 15:45:52.903891 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=81767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 15:45:52.904033 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=81768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 15:45:52.904160 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=81767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 15:45:52.904285 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=81768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 15:45:52.904419 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=81767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 15:45:52.904545 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=81768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 15:45:52.904545 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG uwsgi [-] revoke.caching = True {{(pid=81768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 15:45:52.904779 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=81767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 15:45:52.904908 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=81768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 15:45:52.905036 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=81767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 15:45:52.905171 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=81768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 15:45:52.905299 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=81767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 15:45:52.905427 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG uwsgi [-] role.cache_time = None {{(pid=81768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 15:45:52.905554 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=81767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 15:45:52.905692 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG uwsgi [-] role.caching = True {{(pid=81768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 15:45:52.905820 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=81767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 15:45:52.905956 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG uwsgi [-] role.driver = sql {{(pid=81768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 15:45:52.905956 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG uwsgi [-] role.list_limit = None {{(pid=81768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 15:45:52.906192 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=81767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 15:45:52.906316 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=81768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 15:45:52.906439 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=81767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 15:45:52.906588 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=81768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 15:45:52.906724 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=81767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 15:45:52.906853 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=81768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 15:45:52.906978 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=81767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 15:45:52.907103 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=81768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 15:45:52.907227 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=81767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 15:45:52.907390 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=81768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 15:45:52.907637 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=81767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 15:45:52.907766 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=81768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 15:45:52.907893 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=81767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 15:45:52.908023 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=81768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 15:45:52.908151 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=81767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 15:45:52.908279 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=81768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 15:45:52.908414 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=81767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 15:45:52.908542 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=81768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 15:45:52.908667 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=81767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 15:45:52.908794 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=81768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 15:45:52.908922 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=81767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 15:45:52.909365 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=81768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 15:45:52.909510 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=81767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 15:45:52.909648 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=81768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 15:45:52.909775 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=81767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 15:45:52.909901 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=81768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 15:45:52.910030 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=81767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 15:45:52.910030 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=81767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 15:45:52.910279 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=81768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 15:45:52.910413 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=81767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 15:45:52.910541 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=81768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 15:45:52.910667 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=81767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 15:45:52.910794 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=81768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 15:45:52.910927 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=81767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 15:45:52.911055 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=81768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 15:45:52.911181 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG uwsgi [-] token.cache_time = None {{(pid=81767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 15:45:52.911332 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=81768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 15:45:52.911458 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG uwsgi [-] token.caching = True {{(pid=81767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 15:45:52.911584 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=81768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 15:45:52.911718 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=81767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 15:45:52.911869 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=81768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 15:45:52.911996 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG uwsgi [-] token.provider = fernet {{(pid=81767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 15:45:52.912156 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=81768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 15:45:52.912292 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=81767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 15:45:52.912423 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=81768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 15:45:52.912556 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=81767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 15:45:52.912681 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=81768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 15:45:52.912811 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=81767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 15:45:52.912934 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=81768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 15:45:52.913058 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=81767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 15:45:52.913193 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=81768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 15:45:52.913325 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=81767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 15:45:52.913449 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=81768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 15:45:52.913572 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=81767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 15:45:52.913696 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=81768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 15:45:52.913820 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG uwsgi [-] trust.driver = sql {{(pid=81767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 15:45:52.913944 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=81768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 15:45:52.914075 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=81767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 15:45:52.914199 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=81768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 15:45:52.914324 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=81767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 15:45:52.914447 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=81768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 15:45:52.914570 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=81767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 15:45:52.914653 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=81768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 15:45:52.914832 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=81767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 15:45:52.914958 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG uwsgi [-] token.cache_time = None {{(pid=81768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 15:45:52.915083 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=81767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 15:45:52.915209 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG uwsgi [-] token.caching = True {{(pid=81768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 15:45:52.915355 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=81767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 15:45:52.915485 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=81768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 15:45:52.915622 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=81767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 15:45:52.915749 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG uwsgi [-] token.provider = fernet {{(pid=81768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 15:45:52.915749 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=81768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 15:45:52.915987 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=81767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 15:45:52.916111 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=81768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 15:45:52.916238 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG uwsgi [-] database.connection = **** {{(pid=81767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 15:45:52.916385 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=81768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 15:45:52.916525 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=81767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 15:45:52.916662 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=81768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 15:45:52.916789 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=81767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 15:45:52.916914 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=81768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 15:45:52.917039 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=81767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 15:45:52.917164 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=81768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 15:45:52.917298 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=81767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 15:45:52.917426 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG uwsgi [-] trust.driver = sql {{(pid=81768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 15:45:52.917552 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=81767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 15:45:52.917677 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=81768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 15:45:52.917802 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=81767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 15:45:52.917929 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=81768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 15:45:52.918087 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=81767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 15:45:52.918217 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=81768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 15:45:52.918350 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=81767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 15:45:52.919144 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=81768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 15:45:52.919233 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=81767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 15:45:52.919325 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=81768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 15:45:52.919464 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=81767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 15:45:52.919549 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=81768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 15:45:52.919611 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=81767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 15:45:52.919671 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=81768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 15:45:52.919731 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=81767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 15:45:52.919791 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=81768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 15:45:52.919858 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=81767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 15:45:52.919858 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=81767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 15:45:52.919964 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG uwsgi [-] database.connection = **** {{(pid=81768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 15:45:52.920031 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=81767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 15:45:52.920099 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=81768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 15:45:52.920160 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=81767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 15:45:52.920218 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=81768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 15:45:52.920277 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=81767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 15:45:52.920344 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=81768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 15:45:52.920404 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=81767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 15:45:52.920463 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=81768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 15:45:52.920521 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Aug 22 15:45:52.920580 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=81768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 15:45:52.920580 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=81768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 15:45:52.920580 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=81768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 15:45:52.920580 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=81768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 15:45:52.920580 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=81768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 15:45:52.920580 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=81768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 15:45:52.920580 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=81768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 15:45:52.920580 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=81768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 15:45:52.920580 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=81768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 15:45:52.921012 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=81768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 15:45:52.921012 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=81768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 15:45:52.921012 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=81768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 15:45:52.921012 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=81768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 15:45:52.921012 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=81768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 15:45:52.921012 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Aug 22 15:45:52.941509 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=81767) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 22 15:45:52.941794 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=81767) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 22 15:45:52.942000 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=81767) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 22 15:45:52.942535 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=81768) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 22 15:45:52.942767 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=81768) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 22 15:45:52.942989 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=81768) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 22 15:45:52.950758 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=81767) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 22 15:45:52.951602 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=81768) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 22 15:45:52.979689 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=81768) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 22 15:45:52.979988 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=81768) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 22 15:45:52.980129 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=81767) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 22 15:45:52.980268 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=81767) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 22 15:45:53.034754 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=81767) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 22 15:45:53.035004 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=81768) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 22 15:45:53.084972 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=81767) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 22 15:45:53.085206 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=81768) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 22 15:45:53.085347 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=81767) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 22 15:45:53.085487 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=81768) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 22 15:45:53.243397 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81767) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 22 15:45:53.243674 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81768) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 22 15:45:53.243836 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=81767) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 22 15:45:53.243974 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=81768) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 22 15:45:53.244108 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=81767) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 22 15:45:53.244234 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=81768) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 22 15:45:53.244377 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=81767) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 22 15:45:53.244510 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=81768) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 22 15:45:53.244633 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=81767) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 22 15:45:53.244755 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=81768) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 22 15:45:53.244920 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=81767) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 22 15:45:53.245117 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=81768) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 22 15:45:53.268398 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81768) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 22 15:45:53.268655 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81767) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 22 15:45:53.268802 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=81768) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 22 15:45:53.268946 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=81767) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 22 15:45:53.269085 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=81768) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 22 15:45:53.269251 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=81767) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 22 15:45:53.269386 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=81768) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 22 15:45:53.269516 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=81767) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 22 15:45:53.269647 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=81768) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 22 15:45:53.269777 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=81767) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 22 15:45:53.269777 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=81767) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 22 15:45:53.270028 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=81768) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 22 15:45:53.270156 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81767) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 22 15:45:53.270283 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81768) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 22 15:45:53.281630 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=81768) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 22 15:45:53.281859 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=81767) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 22 15:45:53.295240 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81768) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 22 15:45:53.295436 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=81768) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 22 15:45:53.295555 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81767) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 22 15:45:53.295644 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=81768) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 22 15:45:53.295786 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=81767) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 22 15:45:53.295884 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=81768) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 22 15:45:53.296006 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=81767) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 22 15:45:53.296094 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=81768) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 22 15:45:53.296402 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=81767) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 22 15:45:53.296567 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=81768) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 22 15:45:53.296670 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=81767) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 22 15:45:53.296761 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81768) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 22 15:45:53.296840 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=81767) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 22 15:45:53.296840 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81767) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 22 15:45:53.313701 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=81768) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 22 15:45:53.313866 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=81767) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 22 15:45:53.313968 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=81768) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Aug 22 15:45:53.314047 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=81767) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Aug 22 15:45:53.314231 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=81768) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 22 15:45:53.314314 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=81767) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 22 15:45:53.314430 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=81768) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Aug 22 15:45:53.314511 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=81767) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Aug 22 15:45:53.314704 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=81768) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 22 15:45:53.314789 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=81767) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 22 15:45:53.314966 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=81768) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 22 15:45:53.315046 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=81767) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 22 15:45:53.315518 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=81768) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 22 15:45:53.315616 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=81767) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 22 15:45:53.315786 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=81767) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 22 15:45:53.315867 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=81768) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 22 15:45:53.325190 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=81768) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 22 15:45:53.325291 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=81767) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 22 15:45:53.325466 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=81768) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 22 15:45:53.325564 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=81767) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 22 15:45:53.325738 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=81768) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 22 15:45:53.325821 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=81767) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 22 15:45:53.326024 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=81768) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 22 15:45:53.326106 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=81767) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 22 15:45:53.333053 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `type` (API Prefix: ) [/credentials, /credentials/] {{(pid=81768) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 22 15:45:53.333244 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `type` (API Prefix: ) [/credentials, /credentials/] {{(pid=81767) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 22 15:45:53.335977 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `type` (API Prefix: ) [/domains, /domains/] {{(pid=81767) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 22 15:45:53.336144 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `type` (API Prefix: ) [/domains, /domains/] {{(pid=81768) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 22 15:45:53.336284 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=81768) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 22 15:45:53.336443 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=81767) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 22 15:45:53.336577 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=81768) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 22 15:45:53.336707 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=81767) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 22 15:45:53.336837 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=81768) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 22 15:45:53.336965 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=81767) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 22 15:45:53.337103 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=81768) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 22 15:45:53.337252 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=81767) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 22 15:45:53.337383 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=81768) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 22 15:45:53.337644 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=81767) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 22 15:45:53.337770 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=81768) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 22 15:45:53.337901 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=81767) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 22 15:45:53.338036 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=81768) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 22 15:45:53.338163 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=81767) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 22 15:45:53.338288 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=81768) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 22 15:45:53.338415 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=81767) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 22 15:45:53.338548 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=81768) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 22 15:45:53.338674 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=81767) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 22 15:45:53.338806 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=81768) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 22 15:45:53.338934 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=81767) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 22 15:45:53.362697 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=81768) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 22 15:45:53.362932 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=81767) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 22 15:45:53.364218 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `type` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=81768) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 22 15:45:53.364365 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `type` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=81767) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 22 15:45:53.364496 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=81767) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 22 15:45:53.364622 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=81768) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 22 15:45:53.369169 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `type` (API Prefix: ) [/groups, /groups/] {{(pid=81768) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 22 15:45:53.369341 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `type` (API Prefix: ) [/groups, /groups/] {{(pid=81767) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 22 15:45:53.369471 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=81768) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 22 15:45:53.369597 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=81767) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 22 15:45:53.369727 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=81768) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 22 15:45:53.369858 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=81767) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 22 15:45:53.376210 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `type` (API Prefix: ) [/limits, /limits/] {{(pid=81767) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 22 15:45:53.376502 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=81767) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 22 15:45:53.377246 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `type` (API Prefix: ) [/limits, /limits/] {{(pid=81768) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 22 15:45:53.377536 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=81768) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 22 15:45:53.380623 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `type` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=81767) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 22 15:45:53.380922 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=81767) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 22 15:45:53.381211 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=81767) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 22 15:45:53.381501 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=81767) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 22 15:45:53.381639 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `type` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=81768) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 22 15:45:53.381785 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=81767) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 22 15:45:53.381932 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=81768) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 22 15:45:53.382070 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=81767) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 22 15:45:53.382219 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=81768) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 22 15:45:53.382346 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=81767) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 22 15:45:53.382504 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=81768) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 22 15:45:53.382623 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=81767) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 22 15:45:53.382780 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=81768) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 22 15:45:53.383054 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=81768) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 22 15:45:53.383346 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=81768) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 22 15:45:53.383619 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=81768) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 22 15:45:53.400103 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=81767) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 22 15:45:53.400391 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=81767) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 22 15:45:53.400902 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=81768) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 22 15:45:53.401193 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=81768) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 22 15:45:53.404811 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `type` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=81767) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 22 15:45:53.405610 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `type` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=81768) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 22 15:45:53.408221 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=81767) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 22 15:45:53.408525 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=81767) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 22 15:45:53.408850 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=81768) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 22 15:45:53.409148 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=81768) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 22 15:45:53.412770 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `type` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=81767) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 22 15:45:53.413342 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `type` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=81768) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 22 15:45:53.415933 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `type` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=81767) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 22 15:45:53.416504 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `type` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=81768) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 22 15:45:53.419746 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=81767) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 22 15:45:53.420048 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=81767) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 22 15:45:53.420336 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=81767) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 22 15:45:53.420535 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=81768) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 22 15:45:53.420631 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=81767) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 22 15:45:53.420709 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=81768) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 22 15:45:53.420901 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=81767) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 22 15:45:53.420999 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=81768) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 22 15:45:53.421193 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=81767) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 22 15:45:53.421276 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=81768) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 22 15:45:53.421477 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=81768) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 22 15:45:53.421776 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=81768) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 22 15:45:53.437518 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `type` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=81767) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 22 15:45:53.437794 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `type` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=81768) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 22 15:45:53.437886 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=81767) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 22 15:45:53.438106 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=81768) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 22 15:45:53.438200 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=81767) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 22 15:45:53.438346 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=81767) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 22 15:45:53.438441 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=81768) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 22 15:45:53.438612 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=81768) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 22 15:45:53.445373 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=81767) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 22 15:45:53.445521 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=81768) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 22 15:45:53.446899 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=81767) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 22 15:45:53.447054 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=81768) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 22 15:45:53.447176 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=81767) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 22 15:45:53.447355 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=81768) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 22 15:45:53.449891 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `type` (API Prefix: ) [/policies, /policies/] {{(pid=81767) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 22 15:45:53.450016 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `type` (API Prefix: ) [/policies, /policies/] {{(pid=81768) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 22 15:45:53.450175 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=81767) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 22 15:45:53.450315 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=81768) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 22 15:45:53.450509 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=81767) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 22 15:45:53.450594 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=81768) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 22 15:45:53.450804 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=81767) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 22 15:45:53.450891 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=81768) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 22 15:45:53.451090 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=81767) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 22 15:45:53.451174 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=81768) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 22 15:45:53.462344 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `type` (API Prefix: ) [/projects, /projects/] {{(pid=81768) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 22 15:45:53.462625 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=81768) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 22 15:45:53.462902 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=81768) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 22 15:45:53.463090 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `type` (API Prefix: ) [/projects, /projects/] {{(pid=81767) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 22 15:45:53.463178 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=81768) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 22 15:45:53.463406 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=81767) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 22 15:45:53.463521 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=81768) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 22 15:45:53.463689 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=81767) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 22 15:45:53.463781 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=81768) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 22 15:45:53.463972 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=81767) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 22 15:45:53.464081 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=81768) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 22 15:45:53.464258 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=81767) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 22 15:45:53.464534 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=81767) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 22 15:45:53.464812 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=81767) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 22 15:45:53.479839 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `type` (API Prefix: ) [/regions, /regions/] {{(pid=81768) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 22 15:45:53.481543 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `type` (API Prefix: ) [/regions, /regions/] {{(pid=81767) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 22 15:45:53.483530 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `type` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=81768) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 22 15:45:53.484635 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `type` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=81767) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 22 15:45:53.486416 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=81768) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 22 15:45:53.487565 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=81767) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 22 15:45:53.487878 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=81768) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 22 15:45:53.489011 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=81767) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 22 15:45:53.489341 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `type` (API Prefix: ) [/roles, /roles/] {{(pid=81768) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 22 15:45:53.489629 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=81768) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 22 15:45:53.489925 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=81768) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 22 15:45:53.490496 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `type` (API Prefix: ) [/roles, /roles/] {{(pid=81767) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 22 15:45:53.490814 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=81767) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 22 15:45:53.491117 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=81767) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 22 15:45:53.496430 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=81768) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 22 15:45:53.497760 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=81767) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 22 15:45:53.497914 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `type` (API Prefix: ) [/services, /services/] {{(pid=81768) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 22 15:45:53.499226 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `type` (API Prefix: ) [/services, /services/] {{(pid=81767) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 22 15:45:53.500817 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=81768) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 22 15:45:53.501105 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=81768) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 22 15:45:53.508494 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=81768) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 22 15:45:53.508494 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=81768) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 22 15:45:53.508651 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=81767) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 22 15:45:53.508651 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=81767) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 22 15:45:53.508651 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=81767) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 22 15:45:53.508651 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=81767) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 22 15:45:53.509569 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `type` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=81768) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 22 15:45:53.509859 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=81768) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 22 15:45:53.510139 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=81768) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 22 15:45:53.511071 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `type` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=81767) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 22 15:45:53.511402 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=81767) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 22 15:45:53.511689 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=81767) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 22 15:45:53.517103 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `type` (API Prefix: ) [/users, /users/] {{(pid=81768) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 22 15:45:53.517386 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=81768) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 22 15:45:53.517667 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=81768) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 22 15:45:53.517947 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=81768) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 22 15:45:53.518239 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=81768) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 22 15:45:53.518516 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=81768) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 22 15:45:53.518645 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `type` (API Prefix: ) [/users, /users/] {{(pid=81767) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 22 15:45:53.518799 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=81768) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 22 15:45:53.518961 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=81767) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 22 15:45:53.519107 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=81768) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 22 15:45:53.519238 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=81767) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 22 15:45:53.519394 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=81768) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 22 15:45:53.519519 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=81767) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 22 15:45:53.519661 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=81768) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 22 15:45:53.519793 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=81767) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 22 15:45:53.519934 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=81768) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 22 15:45:53.520065 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=81767) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 22 15:45:53.520196 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=81768) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 22 15:45:53.520327 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=81767) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 22 15:45:53.520457 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=81768) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 22 15:45:53.520596 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=81767) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 22 15:45:53.520734 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=81768) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 22 15:45:53.520861 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=81767) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 22 15:45:53.521076 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=81767) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 22 15:45:53.521352 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=81767) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 22 15:45:53.521635 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=81767) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 22 15:45:53.521902 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=81767) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 22 15:45:53.522174 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=81767) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 22 15:45:53.556161 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: WSGI app 0 (mountpoint='') ready in 5 seconds on interpreter 0x55e589d729c0 pid: 81768 (default app) Aug 22 15:45:53.557646 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: WSGI app 0 (mountpoint='') ready in 5 seconds on interpreter 0x55e589d729c0 pid: 81767 (default app) Aug 22 15:45:53.559637 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3056521c-2742-4634-b408-27dbf2b36866 None None] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:45:53.571580 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3056521c-2742-4634-b408-27dbf2b36866 None None] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:45:53.571580 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3056521c-2742-4634-b408-27dbf2b36866 None None] PATH_INFO: `/v3` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:45:53.571580 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 1/1] 10.222.0.15 () {50 vars in 791 bytes} [Mon Aug 22 15:45:53 2022] GET /identity/v3/ => generated 255 bytes in 8 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 15:46:01.295047 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21dd1207-b2d3-4333-a1ec-dca5d38ec00c None None] REQUEST_METHOD: `GET` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:46:01.295699 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21dd1207-b2d3-4333-a1ec-dca5d38ec00c None None] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:46:01.295699 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21dd1207-b2d3-4333-a1ec-dca5d38ec00c None None] PATH_INFO: `/` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:46:01.299797 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 1/2] 10.222.0.15 () {54 vars in 927 bytes} [Mon Aug 22 15:46:01 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 22 15:46:01.306781 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbd993b5-8600-42ea-92ff-c0a9dfe2662d None None] REQUEST_METHOD: `POST` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:46:01.307225 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbd993b5-8600-42ea-92ff-c0a9dfe2662d None None] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:46:01.307338 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbd993b5-8600-42ea-92ff-c0a9dfe2662d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:46:01.383685 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: INFO dbcounter [None req-bbd993b5-8600-42ea-92ff-c0a9dfe2662d None None] Registered counter for database keystone Aug 22 15:46:01.406072 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG oslo_db.sqlalchemy.engines [None req-bbd993b5-8600-42ea-92ff-c0a9dfe2662d None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=81768) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Aug 22 15:46:01.412430 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writer thread running {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Aug 22 15:46:01.597678 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG passlib.handlers.bcrypt [None req-bbd993b5-8600-42ea-92ff-c0a9dfe2662d None None] detected 'bcrypt' backend, version '3.2.2' {{(pid=81768) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Aug 22 15:46:01.598011 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG passlib.handlers.bcrypt [None req-bbd993b5-8600-42ea-92ff-c0a9dfe2662d None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=81768) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Aug 22 15:46:01.654215 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.auth.core [None req-bbd993b5-8600-42ea-92ff-c0a9dfe2662d None None] MFA Rules not processed for user `6b2199ae0fb0480aa917ada391f5ec54`. Rule list: `[]` (Enabled: `True`). {{(pid=81768) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 15:46:01.655911 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-bbd993b5-8600-42ea-92ff-c0a9dfe2662d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 15:46:01.789310 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 2/3] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Aug 22 15:46:01 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 484 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Aug 22 15:46:02.294381 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21dd1207-b2d3-4333-a1ec-dca5d38ec00c None None] Authenticating user token {{(pid=81767) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 15:46:02.364336 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: INFO dbcounter [None req-21dd1207-b2d3-4333-a1ec-dca5d38ec00c None None] Registered counter for database keystone Aug 22 15:46:02.383660 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG oslo_db.sqlalchemy.engines [None req-21dd1207-b2d3-4333-a1ec-dca5d38ec00c None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=81767) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Aug 22 15:46:02.390406 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG dbcounter [-] [81767] Writer thread running {{(pid=81767) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Aug 22 15:46:02.509416 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21dd1207-b2d3-4333-a1ec-dca5d38ec00c None None] Validating token access rules against request {{(pid=81767) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 15:46:02.514367 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af852dc4-77bc-41eb-87a3-2b6bd5c41737 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a98b1e4431940ab8d1fc95df2858a6c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81767) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 15:46:02.514929 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af852dc4-77bc-41eb-87a3-2b6bd5c41737 admin admin] REQUEST_METHOD: `GET` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:46:02.515200 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af852dc4-77bc-41eb-87a3-2b6bd5c41737 admin admin] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:46:02.515496 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af852dc4-77bc-41eb-87a3-2b6bd5c41737 admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:46:02.523612 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af852dc4-77bc-41eb-87a3-2b6bd5c41737 admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 15:46:02.574807 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: WARNING keystone.common.rbac_enforcer.enforcer [None req-af852dc4-77bc-41eb-87a3-2b6bd5c41737 admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Aug 22 15:46:02.575146 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af852dc4-77bc-41eb-87a3-2b6bd5c41737 admin admin] RBAC: Authorization granted {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 15:46:02.585451 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 2/4] 10.222.0.15 () {58 vars in 1324 bytes} [Mon Aug 22 15:46:02 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 292 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Aug 22 15:46:02.592285 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbd993b5-8600-42ea-92ff-c0a9dfe2662d None None] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 15:46:02.606720 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbd993b5-8600-42ea-92ff-c0a9dfe2662d None None] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 15:46:02.611338 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a527413-0056-4bc0-a41c-2b8016638f90 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a98b1e4431940ab8d1fc95df2858a6c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 15:46:02.612459 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a527413-0056-4bc0-a41c-2b8016638f90 admin admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:46:02.612743 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a527413-0056-4bc0-a41c-2b8016638f90 admin admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:46:02.613018 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a527413-0056-4bc0-a41c-2b8016638f90 admin admin] PATH_INFO: `/v3/projects/9a98b1e4431940ab8d1fc95df2858a6c` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:46:02.614603 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a527413-0056-4bc0-a41c-2b8016638f90 admin admin] RBAC: Authorizing `identity:get_project(project_id=9a98b1e4431940ab8d1fc95df2858a6c)` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 15:46:02.668509 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: WARNING keystone.common.rbac_enforcer.enforcer [None req-4a527413-0056-4bc0-a41c-2b8016638f90 admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Aug 22 15:46:02.668834 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a527413-0056-4bc0-a41c-2b8016638f90 admin admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 15:46:02.673638 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 3/5] 10.222.0.15 () {56 vars in 1207 bytes} [Mon Aug 22 15:46:02 2022] GET /identity/v3/projects/9a98b1e4431940ab8d1fc95df2858a6c => generated 344 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 15:46:04.597706 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf83b210-4f4f-411c-be76-58f9dc656e7b None None] REQUEST_METHOD: `GET` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:46:04.598203 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf83b210-4f4f-411c-be76-58f9dc656e7b None None] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:46:04.598203 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf83b210-4f4f-411c-be76-58f9dc656e7b None None] PATH_INFO: `/` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:46:04.602405 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 3/6] 10.222.0.15 () {54 vars in 927 bytes} [Mon Aug 22 15:46:04 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 22 15:46:04.608200 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95e196bb-9ad0-4adf-83b3-3b7ac27f4cb3 None None] REQUEST_METHOD: `POST` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:46:04.608639 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95e196bb-9ad0-4adf-83b3-3b7ac27f4cb3 None None] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:46:04.608723 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95e196bb-9ad0-4adf-83b3-3b7ac27f4cb3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:46:04.628352 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.auth.core [None req-95e196bb-9ad0-4adf-83b3-3b7ac27f4cb3 None None] MFA Rules not processed for user `6b2199ae0fb0480aa917ada391f5ec54`. Rule list: `[]` (Enabled: `True`). {{(pid=81768) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 15:46:04.629896 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-95e196bb-9ad0-4adf-83b3-3b7ac27f4cb3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 15:46:04.637784 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 4/7] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Aug 22 15:46:04 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Aug 22 15:46:04.905113 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf83b210-4f4f-411c-be76-58f9dc656e7b None None] Authenticating user token {{(pid=81767) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 15:46:04.918051 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf83b210-4f4f-411c-be76-58f9dc656e7b None None] Validating token access rules against request {{(pid=81767) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 15:46:04.922745 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f29a89cd-4dc1-4544-99a9-266af0ad6e45 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a98b1e4431940ab8d1fc95df2858a6c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81767) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 15:46:04.923426 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f29a89cd-4dc1-4544-99a9-266af0ad6e45 admin admin] REQUEST_METHOD: `GET` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:46:04.923706 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f29a89cd-4dc1-4544-99a9-266af0ad6e45 admin admin] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:46:04.923980 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f29a89cd-4dc1-4544-99a9-266af0ad6e45 admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:46:04.933229 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f29a89cd-4dc1-4544-99a9-266af0ad6e45 admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 15:46:04.935039 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f29a89cd-4dc1-4544-99a9-266af0ad6e45 admin admin] RBAC: Authorization granted {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 15:46:04.945862 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 4/8] 10.222.0.15 () {58 vars in 1324 bytes} [Mon Aug 22 15:46:04 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 22 15:46:04.952459 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95e196bb-9ad0-4adf-83b3-3b7ac27f4cb3 None None] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 15:46:04.962400 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95e196bb-9ad0-4adf-83b3-3b7ac27f4cb3 None None] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 15:46:04.967085 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e97a7cb-4811-4010-bed2-beb9b7c0e749 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a98b1e4431940ab8d1fc95df2858a6c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 15:46:04.967785 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e97a7cb-4811-4010-bed2-beb9b7c0e749 admin admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:46:04.968068 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e97a7cb-4811-4010-bed2-beb9b7c0e749 admin admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:46:04.968341 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e97a7cb-4811-4010-bed2-beb9b7c0e749 admin admin] PATH_INFO: `/v3/users/6b2199ae0fb0480aa917ada391f5ec54` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:46:04.969748 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e97a7cb-4811-4010-bed2-beb9b7c0e749 admin admin] RBAC: Authorizing `identity:get_user(user_id=6b2199ae0fb0480aa917ada391f5ec54)` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 15:46:04.971759 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e97a7cb-4811-4010-bed2-beb9b7c0e749 admin admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 15:46:04.973423 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 5/9] 10.222.0.15 () {56 vars in 1198 bytes} [Mon Aug 22 15:46:04 2022] GET /identity/v3/users/6b2199ae0fb0480aa917ada391f5ec54 => generated 247 bytes in 22 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 15:46:06.968611 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23262510-92ef-4bd9-bd3e-d789b32a9d68 None None] REQUEST_METHOD: `GET` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:46:06.969156 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23262510-92ef-4bd9-bd3e-d789b32a9d68 None None] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:46:06.969156 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23262510-92ef-4bd9-bd3e-d789b32a9d68 None None] PATH_INFO: `/` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:46:06.970468 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 5/10] 10.222.0.15 () {54 vars in 927 bytes} [Mon Aug 22 15:46:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 15:46:06.976080 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ba8029f-a02e-4234-bb20-cf7a16abbb71 None None] REQUEST_METHOD: `POST` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:46:06.976486 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ba8029f-a02e-4234-bb20-cf7a16abbb71 None None] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:46:06.976626 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ba8029f-a02e-4234-bb20-cf7a16abbb71 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:46:06.994964 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.auth.core [None req-4ba8029f-a02e-4234-bb20-cf7a16abbb71 None None] MFA Rules not processed for user `6b2199ae0fb0480aa917ada391f5ec54`. Rule list: `[]` (Enabled: `True`). {{(pid=81768) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 15:46:06.996212 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-4ba8029f-a02e-4234-bb20-cf7a16abbb71 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 15:46:07.066518 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42cf082f-4491-455b-8cdf-366dbfae468f None None] REQUEST_METHOD: `GET` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:46:07.066799 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42cf082f-4491-455b-8cdf-366dbfae468f None None] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:46:07.067256 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42cf082f-4491-455b-8cdf-366dbfae468f None None] PATH_INFO: `/` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:46:07.068606 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 6/11] 10.222.0.15 () {54 vars in 927 bytes} [Mon Aug 22 15:46:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 15:46:07.074217 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aa239a9-5bb4-4fae-9fc3-e8d426b8aa0a None None] REQUEST_METHOD: `POST` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:46:07.074499 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aa239a9-5bb4-4fae-9fc3-e8d426b8aa0a None None] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:46:07.074777 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aa239a9-5bb4-4fae-9fc3-e8d426b8aa0a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:46:07.088685 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 6/12] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Aug 22 15:46:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 22 15:46:07.119231 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG passlib.handlers.bcrypt [None req-9aa239a9-5bb4-4fae-9fc3-e8d426b8aa0a None None] detected 'bcrypt' backend, version '3.2.2' {{(pid=81767) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Aug 22 15:46:07.119671 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG passlib.handlers.bcrypt [None req-9aa239a9-5bb4-4fae-9fc3-e8d426b8aa0a None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=81767) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Aug 22 15:46:07.165325 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.auth.core [None req-9aa239a9-5bb4-4fae-9fc3-e8d426b8aa0a None None] MFA Rules not processed for user `6b2199ae0fb0480aa917ada391f5ec54`. Rule list: `[]` (Enabled: `True`). {{(pid=81767) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 15:46:07.166927 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.fernet_utils [None req-9aa239a9-5bb4-4fae-9fc3-e8d426b8aa0a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81767) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 15:46:07.267759 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 7/13] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Aug 22 15:46:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 22 15:46:07.364712 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ba8029f-a02e-4234-bb20-cf7a16abbb71 None None] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 15:46:07.389739 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=100 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 15:46:07.474093 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ba8029f-a02e-4234-bb20-cf7a16abbb71 None None] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 15:46:07.511084 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ba2bb5a-b4ff-4774-8b56-fcdec1a56e67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 15:46:07.511757 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba2bb5a-b4ff-4774-8b56-fcdec1a56e67 None admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:46:07.512039 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba2bb5a-b4ff-4774-8b56-fcdec1a56e67 None admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:46:07.512305 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba2bb5a-b4ff-4774-8b56-fcdec1a56e67 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:46:07.524943 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ba2bb5a-b4ff-4774-8b56-fcdec1a56e67 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 22 15:46:07.533936 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application [None req-1ba2bb5a-b4ff-4774-8b56-fcdec1a56e67 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 22 15:46:07.533936 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 15:46:07.533936 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 15:46:07.533936 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 15:46:07.533936 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 15:46:07.533936 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 15:46:07.533936 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 15:46:07.533936 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 15:46:07.533936 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 15:46:07.533936 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 15:46:07.533936 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 15:46:07.533936 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 15:46:07.533936 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 22 15:46:07.533936 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 22 15:46:07.533936 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 22 15:46:07.533936 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 22 15:46:07.535098 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 22 15:46:07.535098 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 22 15:46:07.535098 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 22 15:46:07.535098 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 22 15:46:07.535098 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 15:46:07.535098 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 15:46:07.535098 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 15:46:07.535098 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 15:46:07.535098 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 15:46:07.535098 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 15:46:07.535098 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 15:46:07.535098 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application with Lock( Aug 22 15:46:07.535098 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 15:46:07.535098 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self._enter() Aug 22 15:46:07.535098 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 15:46:07.535098 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 15:46:07.536147 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 15:46:07.536147 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self.creator() Aug 22 15:46:07.536147 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 15:46:07.536147 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application created_value = creator( Aug 22 15:46:07.536147 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 22 15:46:07.536147 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 22 15:46:07.536147 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 22 15:46:07.536147 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 22 15:46:07.536147 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 22 15:46:07.536147 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 22 15:46:07.536147 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 22 15:46:07.536147 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application Aug 22 15:46:07.536147 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 7/14] 10.222.0.15 () {56 vars in 1096 bytes} [Mon Aug 22 15:46:07 2022] GET /identity/v3/roles/admin => generated 83 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 15:46:07.537085 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aa239a9-5bb4-4fae-9fc3-e8d426b8aa0a None None] Authenticating user token {{(pid=81767) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 15:46:07.541289 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ba2bb5a-b4ff-4774-8b56-fcdec1a56e67 None admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 15:46:07.627240 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=100 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 15:46:07.640906 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aa239a9-5bb4-4fae-9fc3-e8d426b8aa0a None None] Validating token access rules against request {{(pid=81767) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 15:46:07.643185 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG dbcounter [-] [81767] Writing DB stats keystone:SELECT=100 {{(pid=81767) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 15:46:07.652922 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ba2bb5a-b4ff-4774-8b56-fcdec1a56e67 None admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 15:46:07.680362 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d21cb074-9da8-4be0-8d36-b770f9c9cf8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81767) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 15:46:07.681472 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d21cb074-9da8-4be0-8d36-b770f9c9cf8e None admin] REQUEST_METHOD: `GET` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:46:07.681747 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d21cb074-9da8-4be0-8d36-b770f9c9cf8e None admin] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:46:07.682057 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d21cb074-9da8-4be0-8d36-b770f9c9cf8e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:46:07.683426 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dca8d39f-3653-4dd2-9374-54d454021cd8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 15:46:07.684040 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dca8d39f-3653-4dd2-9374-54d454021cd8 None admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:46:07.684313 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dca8d39f-3653-4dd2-9374-54d454021cd8 None admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:46:07.684646 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dca8d39f-3653-4dd2-9374-54d454021cd8 None admin] PATH_INFO: `/v3/roles` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:46:07.685282 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dca8d39f-3653-4dd2-9374-54d454021cd8 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81768) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 15:46:07.685578 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dca8d39f-3653-4dd2-9374-54d454021cd8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 15:46:07.686981 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dca8d39f-3653-4dd2-9374-54d454021cd8 None admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 15:46:07.699095 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 8/15] 10.222.0.15 () {56 vars in 1099 bytes} [Mon Aug 22 15:46:07 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 22 15:46:07.705190 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dca8d39f-3653-4dd2-9374-54d454021cd8 None admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 15:46:07.756560 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d21cb074-9da8-4be0-8d36-b770f9c9cf8e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 15:46:07.758158 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d21cb074-9da8-4be0-8d36-b770f9c9cf8e None admin] RBAC: Authorization granted {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 15:46:07.821670 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dca8d39f-3653-4dd2-9374-54d454021cd8 None admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 15:46:07.856969 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-841f4c77-d5ea-4d81-a108-efe05bf8041a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 15:46:07.857705 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-841f4c77-d5ea-4d81-a108-efe05bf8041a None admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:46:07.858056 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-841f4c77-d5ea-4d81-a108-efe05bf8041a None admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:46:07.858407 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-841f4c77-d5ea-4d81-a108-efe05bf8041a None admin] PATH_INFO: `/v3/users/6b2199ae0fb0480aa917ada391f5ec54` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:46:07.859601 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-841f4c77-d5ea-4d81-a108-efe05bf8041a None admin] RBAC: Authorizing `identity:get_user(user_id=6b2199ae0fb0480aa917ada391f5ec54)` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 15:46:07.861076 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-841f4c77-d5ea-4d81-a108-efe05bf8041a None admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 15:46:07.862662 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 9/16] 10.222.0.15 () {56 vars in 1177 bytes} [Mon Aug 22 15:46:07 2022] GET /identity/v3/users/6b2199ae0fb0480aa917ada391f5ec54 => generated 247 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 15:46:07.868400 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 8/17] 10.222.0.15 () {58 vars in 1282 bytes} [Mon Aug 22 15:46:07 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 333 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 22 15:46:07.868575 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-841f4c77-d5ea-4d81-a108-efe05bf8041a None admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 15:46:07.873714 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d21cb074-9da8-4be0-8d36-b770f9c9cf8e None admin] Authenticating user token {{(pid=81767) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 15:46:07.877530 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG dbcounter [-] [81767] Writing DB stats keystone:SELECT=100 {{(pid=81767) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 15:46:07.883416 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=100 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 15:46:07.974549 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-841f4c77-d5ea-4d81-a108-efe05bf8041a None admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 15:46:07.981817 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d21cb074-9da8-4be0-8d36-b770f9c9cf8e None admin] Validating token access rules against request {{(pid=81767) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 15:46:08.010299 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2c2ed8f-1e73-4679-9e42-27de601f4cf2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 15:46:08.010930 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2c2ed8f-1e73-4679-9e42-27de601f4cf2 None admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:46:08.011204 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2c2ed8f-1e73-4679-9e42-27de601f4cf2 None admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:46:08.011519 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2c2ed8f-1e73-4679-9e42-27de601f4cf2 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:46:08.012629 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2c2ed8f-1e73-4679-9e42-27de601f4cf2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 15:46:08.014767 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2c2ed8f-1e73-4679-9e42-27de601f4cf2 None admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 15:46:08.016362 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 10/18] 10.222.0.15 () {56 vars in 1108 bytes} [Mon Aug 22 15:46:07 2022] GET /identity/v3/domains/default => generated 199 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 22 15:46:08.018607 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42fd3be4-73d4-4a5c-9253-46f3068b95fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81767) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 15:46:08.019504 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42fd3be4-73d4-4a5c-9253-46f3068b95fb None admin] REQUEST_METHOD: `GET` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:46:08.019787 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42fd3be4-73d4-4a5c-9253-46f3068b95fb None admin] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:46:08.020065 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42fd3be4-73d4-4a5c-9253-46f3068b95fb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:46:08.022254 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2c2ed8f-1e73-4679-9e42-27de601f4cf2 None admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 15:46:08.039745 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42fd3be4-73d4-4a5c-9253-46f3068b95fb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 15:46:08.041222 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42fd3be4-73d4-4a5c-9253-46f3068b95fb None admin] RBAC: Authorization granted {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 15:46:08.061170 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application [None req-42fd3be4-73d4-4a5c-9253-46f3068b95fb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 22 15:46:08.061170 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 15:46:08.061170 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 15:46:08.061170 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 15:46:08.061170 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 15:46:08.061170 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 15:46:08.061170 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 15:46:08.061170 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 15:46:08.061170 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 15:46:08.061170 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 15:46:08.061170 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 15:46:08.061170 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 15:46:08.061170 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 22 15:46:08.061170 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 22 15:46:08.061170 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 22 15:46:08.061170 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 22 15:46:08.062124 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 15:46:08.062124 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 15:46:08.062124 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 15:46:08.062124 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 15:46:08.062124 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 15:46:08.062124 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 15:46:08.062124 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 15:46:08.062124 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application with Lock( Aug 22 15:46:08.062124 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 15:46:08.062124 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return self._enter() Aug 22 15:46:08.062124 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 15:46:08.062124 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 15:46:08.062124 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 15:46:08.062124 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return self.creator() Aug 22 15:46:08.062124 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 15:46:08.062124 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application created_value = creator( Aug 22 15:46:08.062124 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 22 15:46:08.062904 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 22 15:46:08.062904 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 22 15:46:08.062904 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application Aug 22 15:46:08.062904 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 9/19] 10.222.0.15 () {56 vars in 1108 bytes} [Mon Aug 22 15:46:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 15:46:08.068186 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42fd3be4-73d4-4a5c-9253-46f3068b95fb None admin] Authenticating user token {{(pid=81767) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 15:46:08.108996 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=100 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 15:46:08.128096 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG dbcounter [-] [81767] Writing DB stats keystone:SELECT=100 {{(pid=81767) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 15:46:08.135094 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2c2ed8f-1e73-4679-9e42-27de601f4cf2 None admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 15:46:08.173105 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eb0b2f4-bee6-4dde-b77c-a9ba7ae0b1c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 15:46:08.173716 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eb0b2f4-bee6-4dde-b77c-a9ba7ae0b1c9 None admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:46:08.173880 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42fd3be4-73d4-4a5c-9253-46f3068b95fb None admin] Validating token access rules against request {{(pid=81767) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 15:46:08.173996 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eb0b2f4-bee6-4dde-b77c-a9ba7ae0b1c9 None admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:46:08.174297 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eb0b2f4-bee6-4dde-b77c-a9ba7ae0b1c9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:46:08.175020 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eb0b2f4-bee6-4dde-b77c-a9ba7ae0b1c9 None admin] RBAC: Adding query filter params (role.id=9d9848cfd22e4b089db443c350e56670, scope.domain.id=default, user.id=6b2199ae0fb0480aa917ada391f5ec54) {{(pid=81768) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 15:46:08.175357 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eb0b2f4-bee6-4dde-b77c-a9ba7ae0b1c9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 15:46:08.176839 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eb0b2f4-bee6-4dde-b77c-a9ba7ae0b1c9 None admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 15:46:08.184400 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 11/20] 10.222.0.15 () {56 vars in 1322 bytes} [Mon Aug 22 15:46:08 2022] GET /identity/v3/role_assignments?user.id=6b2199ae0fb0480aa917ada391f5ec54&scope.domain.id=default&role.id=9d9848cfd22e4b089db443c350e56670 => generated 233 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 22 15:46:08.215776 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-507d20d7-a58d-45e8-b2c7-a216665f5d9a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81767) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 15:46:08.216447 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-507d20d7-a58d-45e8-b2c7-a216665f5d9a None admin] REQUEST_METHOD: `GET` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:46:08.216721 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-507d20d7-a58d-45e8-b2c7-a216665f5d9a None admin] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:46:08.216993 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-507d20d7-a58d-45e8-b2c7-a216665f5d9a None admin] PATH_INFO: `/v3/domains` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:46:08.217673 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-507d20d7-a58d-45e8-b2c7-a216665f5d9a None admin] RBAC: Adding query filter params (name=Default) {{(pid=81767) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 15:46:08.217973 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-507d20d7-a58d-45e8-b2c7-a216665f5d9a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 15:46:08.219425 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-507d20d7-a58d-45e8-b2c7-a216665f5d9a None admin] RBAC: Authorization granted {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 15:46:08.241194 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 10/21] 10.222.0.15 () {56 vars in 1109 bytes} [Mon Aug 22 15:46:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 22 15:46:10.725152 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5468a65f-4ef3-41d1-a486-45994303e404 None None] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:46:10.725644 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5468a65f-4ef3-41d1-a486-45994303e404 None None] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:46:10.726206 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5468a65f-4ef3-41d1-a486-45994303e404 None None] PATH_INFO: `/` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:46:10.728962 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 12/22] 10.222.0.15 () {54 vars in 927 bytes} [Mon Aug 22 15:46:10 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 15:46:10.738085 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-652ea8d9-52ad-422e-9402-4659820e61a4 None None] REQUEST_METHOD: `POST` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:46:10.739427 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-652ea8d9-52ad-422e-9402-4659820e61a4 None None] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:46:10.740165 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-652ea8d9-52ad-422e-9402-4659820e61a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:46:10.771002 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.auth.core [None req-652ea8d9-52ad-422e-9402-4659820e61a4 None None] MFA Rules not processed for user `6b2199ae0fb0480aa917ada391f5ec54`. Rule list: `[]` (Enabled: `True`). {{(pid=81767) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 15:46:10.773390 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.fernet_utils [None req-652ea8d9-52ad-422e-9402-4659820e61a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81767) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 15:46:10.843834 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bb39b9d-261d-420b-953d-401df8a1f28d None None] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:46:10.849387 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bb39b9d-261d-420b-953d-401df8a1f28d None None] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:46:10.849387 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bb39b9d-261d-420b-953d-401df8a1f28d None None] PATH_INFO: `/` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:46:10.849387 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 13/23] 10.222.0.15 () {54 vars in 927 bytes} [Mon Aug 22 15:46:10 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 15:46:10.853651 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2858b229-c53c-4b2e-b984-6ba2789c23f5 None None] REQUEST_METHOD: `POST` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:46:10.854257 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2858b229-c53c-4b2e-b984-6ba2789c23f5 None None] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:46:10.854840 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2858b229-c53c-4b2e-b984-6ba2789c23f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:46:10.907603 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.auth.core [None req-2858b229-c53c-4b2e-b984-6ba2789c23f5 None None] MFA Rules not processed for user `6b2199ae0fb0480aa917ada391f5ec54`. Rule list: `[]` (Enabled: `True`). {{(pid=81768) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 15:46:10.920245 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-2858b229-c53c-4b2e-b984-6ba2789c23f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 15:46:10.936408 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 11/24] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Aug 22 15:46:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 200 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 22 15:46:11.028032 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 14/25] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Aug 22 15:46:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 22 15:46:11.121606 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ad4a6bf-1a11-40a4-8778-8fad2e440c54 None None] REQUEST_METHOD: `GET` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:46:11.121888 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ad4a6bf-1a11-40a4-8778-8fad2e440c54 None None] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:46:11.122319 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ad4a6bf-1a11-40a4-8778-8fad2e440c54 None None] PATH_INFO: `/` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:46:11.124163 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 12/26] 10.222.0.15 () {54 vars in 927 bytes} [Mon Aug 22 15:46:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 15:46:11.129300 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67a3b045-680a-410c-8019-1cf527497aa1 None None] REQUEST_METHOD: `POST` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:46:11.129821 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67a3b045-680a-410c-8019-1cf527497aa1 None None] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:46:11.130310 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67a3b045-680a-410c-8019-1cf527497aa1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:46:11.156567 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.auth.core [None req-67a3b045-680a-410c-8019-1cf527497aa1 None None] MFA Rules not processed for user `6b2199ae0fb0480aa917ada391f5ec54`. Rule list: `[]` (Enabled: `True`). {{(pid=81768) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 15:46:11.158156 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-67a3b045-680a-410c-8019-1cf527497aa1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 15:46:11.241608 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 15/27] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Aug 22 15:46:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 22 15:46:11.291678 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f41a357-8619-49a5-85f9-d776ddf2bb27 None None] REQUEST_METHOD: `GET` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:46:11.292288 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f41a357-8619-49a5-85f9-d776ddf2bb27 None None] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:46:11.292772 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f41a357-8619-49a5-85f9-d776ddf2bb27 None None] PATH_INFO: `/` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:46:11.294315 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 13/28] 10.222.0.15 () {54 vars in 927 bytes} [Mon Aug 22 15:46:11 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 15:46:11.300219 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74357fe8-f108-40e1-9bfa-35751879abdf None None] REQUEST_METHOD: `POST` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:46:11.300732 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74357fe8-f108-40e1-9bfa-35751879abdf None None] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:46:11.301199 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74357fe8-f108-40e1-9bfa-35751879abdf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:46:11.310419 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=100 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 15:46:11.324226 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.auth.core [None req-74357fe8-f108-40e1-9bfa-35751879abdf None None] MFA Rules not processed for user `6b2199ae0fb0480aa917ada391f5ec54`. Rule list: `[]` (Enabled: `True`). {{(pid=81768) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 15:46:11.335397 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-74357fe8-f108-40e1-9bfa-35751879abdf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 15:46:11.417325 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 16/29] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Aug 22 15:46:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 22 15:46:11.419425 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f41a357-8619-49a5-85f9-d776ddf2bb27 None None] Authenticating user token {{(pid=81767) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 15:46:11.474126 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74357fe8-f108-40e1-9bfa-35751879abdf None None] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 15:46:11.483048 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG dbcounter [-] [81767] Writing DB stats keystone:SELECT=100 {{(pid=81767) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 15:46:11.593231 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f41a357-8619-49a5-85f9-d776ddf2bb27 None None] Validating token access rules against request {{(pid=81767) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 15:46:11.595856 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74357fe8-f108-40e1-9bfa-35751879abdf None None] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 15:46:11.637598 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89f697df-6884-4db9-9123-cdc7cd4aa841 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 15:46:11.639125 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89f697df-6884-4db9-9123-cdc7cd4aa841 None admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:46:11.639650 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89f697df-6884-4db9-9123-cdc7cd4aa841 None admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:46:11.640615 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89f697df-6884-4db9-9123-cdc7cd4aa841 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:46:11.643265 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=100 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 15:46:11.654462 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89f697df-6884-4db9-9123-cdc7cd4aa841 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 15:46:11.657817 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89f697df-6884-4db9-9123-cdc7cd4aa841 None admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 15:46:11.668933 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e421cd6-5906-46b3-8fd5-bcbe1fa9f3d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81767) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 15:46:11.669601 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e421cd6-5906-46b3-8fd5-bcbe1fa9f3d9 None admin] REQUEST_METHOD: `GET` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:46:11.670063 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e421cd6-5906-46b3-8fd5-bcbe1fa9f3d9 None admin] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:46:11.670478 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e421cd6-5906-46b3-8fd5-bcbe1fa9f3d9 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:46:11.674274 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application [None req-89f697df-6884-4db9-9123-cdc7cd4aa841 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 22 15:46:11.674274 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 15:46:11.674274 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 15:46:11.674274 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 15:46:11.674274 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 15:46:11.674274 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 15:46:11.674274 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 15:46:11.674274 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 15:46:11.674274 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 15:46:11.674274 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 15:46:11.674274 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 15:46:11.674274 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 15:46:11.674274 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 22 15:46:11.674274 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 22 15:46:11.674274 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 22 15:46:11.674274 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 22 15:46:11.675099 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 15:46:11.675099 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 15:46:11.675099 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 15:46:11.675099 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 15:46:11.675099 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 15:46:11.675099 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 15:46:11.675099 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 15:46:11.675099 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application with Lock( Aug 22 15:46:11.675099 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 15:46:11.675099 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self._enter() Aug 22 15:46:11.675099 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 15:46:11.675099 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 15:46:11.675099 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 15:46:11.675099 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self.creator() Aug 22 15:46:11.675099 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 15:46:11.675099 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application created_value = creator( Aug 22 15:46:11.675099 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 22 15:46:11.675907 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 22 15:46:11.675907 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 22 15:46:11.675907 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application Aug 22 15:46:11.677934 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 17/30] 10.222.0.15 () {56 vars in 1108 bytes} [Mon Aug 22 15:46:11 2022] GET /identity/v3/domains/Default => generated 87 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 22 15:46:11.687664 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89f697df-6884-4db9-9123-cdc7cd4aa841 None admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 15:46:11.701271 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e421cd6-5906-46b3-8fd5-bcbe1fa9f3d9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 22 15:46:11.707914 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application [None req-0e421cd6-5906-46b3-8fd5-bcbe1fa9f3d9 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 22 15:46:11.707914 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 15:46:11.707914 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 15:46:11.707914 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 15:46:11.707914 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 15:46:11.707914 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 15:46:11.707914 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 15:46:11.707914 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 15:46:11.707914 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 15:46:11.707914 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 15:46:11.707914 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 15:46:11.707914 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 15:46:11.707914 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 22 15:46:11.707914 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 22 15:46:11.707914 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 22 15:46:11.707914 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 22 15:46:11.708743 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 22 15:46:11.708743 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 22 15:46:11.708743 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 22 15:46:11.708743 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 22 15:46:11.708743 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 15:46:11.708743 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 15:46:11.708743 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 15:46:11.708743 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 15:46:11.708743 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 15:46:11.708743 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 15:46:11.708743 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 15:46:11.708743 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application with Lock( Aug 22 15:46:11.708743 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 15:46:11.708743 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return self._enter() Aug 22 15:46:11.708743 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 15:46:11.708743 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 15:46:11.709688 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 15:46:11.709688 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return self.creator() Aug 22 15:46:11.709688 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 15:46:11.709688 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application created_value = creator( Aug 22 15:46:11.709688 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 22 15:46:11.709688 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 22 15:46:11.709688 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 22 15:46:11.709688 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 22 15:46:11.709688 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 22 15:46:11.709688 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 22 15:46:11.709688 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 22 15:46:11.709688 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application Aug 22 15:46:11.709688 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 14/31] 10.222.0.15 () {56 vars in 1096 bytes} [Mon Aug 22 15:46:11 2022] GET /identity/v3/roles/admin => generated 83 bytes in 290 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 15:46:11.718772 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e421cd6-5906-46b3-8fd5-bcbe1fa9f3d9 None admin] Authenticating user token {{(pid=81767) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 15:46:11.881590 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e421cd6-5906-46b3-8fd5-bcbe1fa9f3d9 None admin] Validating token access rules against request {{(pid=81767) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 15:46:11.886075 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89f697df-6884-4db9-9123-cdc7cd4aa841 None admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 15:46:11.891662 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG dbcounter [-] [81767] Writing DB stats keystone:SELECT=100 {{(pid=81767) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 15:46:11.924321 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d0f9790-6b8c-4e58-9dc8-3e0b40e1ffb1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81767) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 15:46:11.924937 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d0f9790-6b8c-4e58-9dc8-3e0b40e1ffb1 None admin] REQUEST_METHOD: `GET` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:46:11.925218 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d0f9790-6b8c-4e58-9dc8-3e0b40e1ffb1 None admin] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:46:11.925492 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d0f9790-6b8c-4e58-9dc8-3e0b40e1ffb1 None admin] PATH_INFO: `/v3/roles` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:46:11.926285 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d0f9790-6b8c-4e58-9dc8-3e0b40e1ffb1 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81767) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 15:46:11.926593 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d0f9790-6b8c-4e58-9dc8-3e0b40e1ffb1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 15:46:11.928165 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d0f9790-6b8c-4e58-9dc8-3e0b40e1ffb1 None admin] RBAC: Authorization granted {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 15:46:11.933384 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8c00182-4684-4ed3-94af-82b7b744702d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 15:46:11.934387 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8c00182-4684-4ed3-94af-82b7b744702d None admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:46:11.934877 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8c00182-4684-4ed3-94af-82b7b744702d None admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:46:11.935986 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8c00182-4684-4ed3-94af-82b7b744702d None admin] PATH_INFO: `/v3/domains` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:46:11.936895 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8c00182-4684-4ed3-94af-82b7b744702d None admin] RBAC: Adding query filter params (name=Default) {{(pid=81768) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 15:46:11.937396 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8c00182-4684-4ed3-94af-82b7b744702d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 15:46:11.939560 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8c00182-4684-4ed3-94af-82b7b744702d None admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 15:46:11.942948 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 15/32] 10.222.0.15 () {56 vars in 1099 bytes} [Mon Aug 22 15:46:11 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 15:46:11.948442 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d0f9790-6b8c-4e58-9dc8-3e0b40e1ffb1 None admin] Authenticating user token {{(pid=81767) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 15:46:11.968396 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 18/33] 10.222.0.15 () {56 vars in 1109 bytes} [Mon Aug 22 15:46:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 281 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 22 15:46:11.969644 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8c00182-4684-4ed3-94af-82b7b744702d None admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 15:46:12.030160 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=100 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 15:46:12.085324 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d0f9790-6b8c-4e58-9dc8-3e0b40e1ffb1 None admin] Validating token access rules against request {{(pid=81767) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 15:46:12.095303 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8c00182-4684-4ed3-94af-82b7b744702d None admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 15:46:12.132527 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f091b208-ad8f-4c54-8ffb-2680996ae16e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81767) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 15:46:12.133324 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f091b208-ad8f-4c54-8ffb-2680996ae16e None admin] REQUEST_METHOD: `POST` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:46:12.133702 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f091b208-ad8f-4c54-8ffb-2680996ae16e None admin] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:46:12.134096 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f091b208-ad8f-4c54-8ffb-2680996ae16e None admin] PATH_INFO: `/v3/roles` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:46:12.134752 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e52fc17d-04d7-4b78-a19b-f641453b61ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 15:46:12.135001 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f091b208-ad8f-4c54-8ffb-2680996ae16e None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 15:46:12.135762 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e52fc17d-04d7-4b78-a19b-f641453b61ca None admin] REQUEST_METHOD: `POST` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:46:12.136195 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e52fc17d-04d7-4b78-a19b-f641453b61ca None admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:46:12.136631 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e52fc17d-04d7-4b78-a19b-f641453b61ca None admin] PATH_INFO: `/v3/roles` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:46:12.136735 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f091b208-ad8f-4c54-8ffb-2680996ae16e None admin] RBAC: Authorization granted {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 15:46:12.137526 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e52fc17d-04d7-4b78-a19b-f641453b61ca None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 15:46:12.139078 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e52fc17d-04d7-4b78-a19b-f641453b61ca None admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 15:46:12.391391 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 16/34] 10.222.0.15 () {60 vars in 1131 bytes} [Mon Aug 22 15:46:11 2022] POST /identity/v3/roles => generated 219 bytes in 448 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 22 15:46:12.395434 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f091b208-ad8f-4c54-8ffb-2680996ae16e None admin] Authenticating user token {{(pid=81767) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 15:46:12.431018 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG dbcounter [-] [81767] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81767) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 15:46:12.457002 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 19/35] 10.222.0.15 () {60 vars in 1131 bytes} [Mon Aug 22 15:46:11 2022] POST /identity/v3/roles => generated 225 bytes in 488 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 22 15:46:12.458298 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e52fc17d-04d7-4b78-a19b-f641453b61ca None admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 15:46:12.524508 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f091b208-ad8f-4c54-8ffb-2680996ae16e None admin] Validating token access rules against request {{(pid=81767) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 15:46:12.577191 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5890ec22-c7ee-4633-9e99-dea9eb6a2af0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81767) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 15:46:12.577191 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5890ec22-c7ee-4633-9e99-dea9eb6a2af0 None admin] REQUEST_METHOD: `GET` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:46:12.577191 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5890ec22-c7ee-4633-9e99-dea9eb6a2af0 None admin] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:46:12.577191 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5890ec22-c7ee-4633-9e99-dea9eb6a2af0 None admin] PATH_INFO: `/v3/users/6b2199ae0fb0480aa917ada391f5ec54` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:46:12.581674 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5890ec22-c7ee-4633-9e99-dea9eb6a2af0 None admin] RBAC: Authorizing `identity:get_user(user_id=6b2199ae0fb0480aa917ada391f5ec54)` {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 15:46:12.582481 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5890ec22-c7ee-4633-9e99-dea9eb6a2af0 None admin] RBAC: Authorization granted {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 15:46:12.584613 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 17/36] 10.222.0.15 () {56 vars in 1177 bytes} [Mon Aug 22 15:46:12 2022] GET /identity/v3/users/6b2199ae0fb0480aa917ada391f5ec54 => generated 247 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 15:46:12.586131 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7038da39-2e37-447b-a701-1ad111160ba0 None None] REQUEST_METHOD: `GET` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:46:12.586417 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7038da39-2e37-447b-a701-1ad111160ba0 None None] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:46:12.586689 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7038da39-2e37-447b-a701-1ad111160ba0 None None] PATH_INFO: `/` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:46:12.588280 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 18/37] 10.222.0.15 () {54 vars in 927 bytes} [Mon Aug 22 15:46:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 15:46:12.589154 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7def60e2-3216-41c0-86b3-aee9d450f10c None None] REQUEST_METHOD: `GET` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:46:12.589428 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7def60e2-3216-41c0-86b3-aee9d450f10c None None] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:46:12.590053 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7def60e2-3216-41c0-86b3-aee9d450f10c None None] PATH_INFO: `/` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:46:12.591001 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 19/38] 10.222.0.15 () {54 vars in 927 bytes} [Mon Aug 22 15:46:12 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 15:46:12.593317 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7def60e2-3216-41c0-86b3-aee9d450f10c None None] Authenticating user token {{(pid=81767) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 15:46:12.614413 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e52fc17d-04d7-4b78-a19b-f641453b61ca None admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 15:46:12.623551 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 15:46:12.659813 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a1cd534-bd3f-486d-9a12-1e4cf548fc71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 15:46:12.660368 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a1cd534-bd3f-486d-9a12-1e4cf548fc71 None admin] REQUEST_METHOD: `POST` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:46:12.660648 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a1cd534-bd3f-486d-9a12-1e4cf548fc71 None admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:46:12.660941 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a1cd534-bd3f-486d-9a12-1e4cf548fc71 None admin] PATH_INFO: `/v3/projects` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:46:12.661569 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a1cd534-bd3f-486d-9a12-1e4cf548fc71 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 15:46:12.663405 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a1cd534-bd3f-486d-9a12-1e4cf548fc71 None admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 15:46:12.716375 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG dbcounter [-] [81767] Writing DB stats keystone:SELECT=100 {{(pid=81767) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 15:46:12.723334 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7def60e2-3216-41c0-86b3-aee9d450f10c None None] Validating token access rules against request {{(pid=81767) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 15:46:12.758069 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63e4431f-153a-4883-971e-e09b5809c092 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81767) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 15:46:12.758738 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63e4431f-153a-4883-971e-e09b5809c092 None admin] REQUEST_METHOD: `GET` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:46:12.759018 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63e4431f-153a-4883-971e-e09b5809c092 None admin] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:46:12.759305 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63e4431f-153a-4883-971e-e09b5809c092 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:46:12.760505 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63e4431f-153a-4883-971e-e09b5809c092 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 15:46:12.762040 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63e4431f-153a-4883-971e-e09b5809c092 None admin] RBAC: Authorization granted {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 15:46:12.763643 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 20/39] 10.222.0.15 () {56 vars in 1108 bytes} [Mon Aug 22 15:46:12 2022] GET /identity/v3/domains/default => generated 199 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 15:46:12.765440 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09702274-d6e2-4b9a-8962-dda834a2bce9 None None] REQUEST_METHOD: `POST` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:46:12.765440 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09702274-d6e2-4b9a-8962-dda834a2bce9 None None] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:46:12.765440 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09702274-d6e2-4b9a-8962-dda834a2bce9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:46:12.783869 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.auth.core [None req-09702274-d6e2-4b9a-8962-dda834a2bce9 None None] MFA Rules not processed for user `6b2199ae0fb0480aa917ada391f5ec54`. Rule list: `[]` (Enabled: `True`). {{(pid=81767) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 15:46:12.785384 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.fernet_utils [None req-09702274-d6e2-4b9a-8962-dda834a2bce9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81767) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 15:46:12.797091 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 20/40] 10.222.0.15 () {60 vars in 1141 bytes} [Mon Aug 22 15:46:12 2022] POST /identity/v3/projects => generated 301 bytes in 340 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 22 15:46:12.798418 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e51d5d69-595c-4f6f-973d-8f5d9e66a111 None None] REQUEST_METHOD: `POST` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:46:12.798737 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e51d5d69-595c-4f6f-973d-8f5d9e66a111 None None] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:46:12.799019 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e51d5d69-595c-4f6f-973d-8f5d9e66a111 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:46:12.819550 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.auth.core [None req-e51d5d69-595c-4f6f-973d-8f5d9e66a111 None None] MFA Rules not processed for user `6b2199ae0fb0480aa917ada391f5ec54`. Rule list: `[]` (Enabled: `True`). {{(pid=81768) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 15:46:12.821401 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-e51d5d69-595c-4f6f-973d-8f5d9e66a111 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 15:46:12.865637 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 21/41] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Aug 22 15:46:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 22 15:46:12.866308 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09702274-d6e2-4b9a-8962-dda834a2bce9 None None] Authenticating user token {{(pid=81767) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 15:46:12.919906 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 21/42] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Aug 22 15:46:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 22 15:46:12.982936 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG dbcounter [-] [81767] Writing DB stats keystone:SELECT=100 {{(pid=81767) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 15:46:12.989967 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09702274-d6e2-4b9a-8962-dda834a2bce9 None None] Validating token access rules against request {{(pid=81767) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 15:46:13.028085 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-804aac13-83a0-4793-809b-8b4c6a76feeb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81767) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 15:46:13.029617 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-804aac13-83a0-4793-809b-8b4c6a76feeb None admin] REQUEST_METHOD: `PUT` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:46:13.029905 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-804aac13-83a0-4793-809b-8b4c6a76feeb None admin] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:46:13.030182 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-804aac13-83a0-4793-809b-8b4c6a76feeb None admin] PATH_INFO: `/v3/domains/default/users/6b2199ae0fb0480aa917ada391f5ec54/roles/9d9848cfd22e4b089db443c350e56670` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:46:13.032266 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-804aac13-83a0-4793-809b-8b4c6a76feeb None admin] RBAC: Authorizing `identity:create_grant(domain_id=default, user_id=6b2199ae0fb0480aa917ada391f5ec54, role_id=9d9848cfd22e4b089db443c350e56670)` {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 15:46:13.033833 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-804aac13-83a0-4793-809b-8b4c6a76feeb None admin] RBAC: Authorization granted {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 15:46:13.134402 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e51d5d69-595c-4f6f-973d-8f5d9e66a111 None None] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 15:46:13.139430 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 22/43] 10.222.0.15 () {58 vars in 1361 bytes} [Mon Aug 22 15:46:12 2022] PUT /identity/v3/domains/default/users/6b2199ae0fb0480aa917ada391f5ec54/roles/9d9848cfd22e4b089db443c350e56670 => generated 0 bytes in 273 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 15:46:13.204974 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-804aac13-83a0-4793-809b-8b4c6a76feeb None admin] Authenticating user token {{(pid=81767) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 15:46:13.243456 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 15:46:13.256426 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e51d5d69-595c-4f6f-973d-8f5d9e66a111 None None] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 15:46:13.332650 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-804aac13-83a0-4793-809b-8b4c6a76feeb None admin] Validating token access rules against request {{(pid=81767) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 15:46:13.379010 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcf97fb4-f64d-4a26-8472-c1b2f1a0ccac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81767) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 15:46:13.379651 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcf97fb4-f64d-4a26-8472-c1b2f1a0ccac None admin] REQUEST_METHOD: `POST` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:46:13.379926 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcf97fb4-f64d-4a26-8472-c1b2f1a0ccac None admin] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:46:13.380193 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcf97fb4-f64d-4a26-8472-c1b2f1a0ccac None admin] PATH_INFO: `/v3/roles` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:46:13.380857 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcf97fb4-f64d-4a26-8472-c1b2f1a0ccac None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 15:46:13.382243 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcf97fb4-f64d-4a26-8472-c1b2f1a0ccac None admin] RBAC: Authorization granted {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 15:46:13.405694 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c403fa25-5c04-40d4-ab67-8b3724b1b35a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 15:46:13.406534 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c403fa25-5c04-40d4-ab67-8b3724b1b35a None admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:46:13.406994 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c403fa25-5c04-40d4-ab67-8b3724b1b35a None admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:46:13.407683 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c403fa25-5c04-40d4-ab67-8b3724b1b35a None admin] PATH_INFO: `/v3/domains/default` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:46:13.408813 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c403fa25-5c04-40d4-ab67-8b3724b1b35a None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 15:46:13.410290 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c403fa25-5c04-40d4-ab67-8b3724b1b35a None admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 15:46:13.411998 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 22/44] 10.222.0.15 () {56 vars in 1108 bytes} [Mon Aug 22 15:46:13 2022] GET /identity/v3/domains/default => generated 199 bytes in 278 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 22 15:46:13.417355 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c403fa25-5c04-40d4-ab67-8b3724b1b35a None admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 15:46:13.470278 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 23/45] 10.222.0.15 () {60 vars in 1131 bytes} [Mon Aug 22 15:46:13 2022] POST /identity/v3/roles => generated 223 bytes in 266 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 22 15:46:13.530026 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c403fa25-5c04-40d4-ab67-8b3724b1b35a None admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 15:46:13.568815 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e0b5664-4080-4850-b939-aa889b7e62a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 15:46:13.569442 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e0b5664-4080-4850-b939-aa889b7e62a3 None admin] REQUEST_METHOD: `POST` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:46:13.569724 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e0b5664-4080-4850-b939-aa889b7e62a3 None admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:46:13.569997 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e0b5664-4080-4850-b939-aa889b7e62a3 None admin] PATH_INFO: `/v3/projects` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:46:13.570661 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e0b5664-4080-4850-b939-aa889b7e62a3 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 15:46:13.572162 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e0b5664-4080-4850-b939-aa889b7e62a3 None admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 15:46:13.676012 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 23/46] 10.222.0.15 () {60 vars in 1141 bytes} [Mon Aug 22 15:46:13 2022] POST /identity/v3/projects => generated 312 bytes in 259 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 22 15:46:15.107887 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64fde42d-2518-4254-9752-e0871ff58086 None None] REQUEST_METHOD: `GET` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:46:15.108411 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64fde42d-2518-4254-9752-e0871ff58086 None None] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:46:15.108519 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64fde42d-2518-4254-9752-e0871ff58086 None None] PATH_INFO: `/` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:46:15.109775 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 24/47] 10.222.0.15 () {54 vars in 927 bytes} [Mon Aug 22 15:46:15 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 15:46:15.115326 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4456c26-8874-4abb-9d7a-ef25519dac75 None None] REQUEST_METHOD: `POST` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:46:15.115736 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4456c26-8874-4abb-9d7a-ef25519dac75 None None] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:46:15.115892 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4456c26-8874-4abb-9d7a-ef25519dac75 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:46:15.133964 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.auth.core [None req-f4456c26-8874-4abb-9d7a-ef25519dac75 None None] MFA Rules not processed for user `6b2199ae0fb0480aa917ada391f5ec54`. Rule list: `[]` (Enabled: `True`). {{(pid=81768) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 15:46:15.135144 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-f4456c26-8874-4abb-9d7a-ef25519dac75 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 15:46:15.137189 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 15:46:15.204442 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 24/48] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Aug 22 15:46:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 22 15:46:15.476650 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64fde42d-2518-4254-9752-e0871ff58086 None None] Authenticating user token {{(pid=81767) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 15:46:15.493036 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG dbcounter [-] [81767] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81767) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 15:46:15.582665 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64fde42d-2518-4254-9752-e0871ff58086 None None] Validating token access rules against request {{(pid=81767) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 15:46:15.597832 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5b3d5ea-7a8e-478b-9007-9064dfba61b0 None None] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:46:15.598113 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5b3d5ea-7a8e-478b-9007-9064dfba61b0 None None] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:46:15.598384 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5b3d5ea-7a8e-478b-9007-9064dfba61b0 None None] PATH_INFO: `/` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:46:15.599766 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 25/49] 10.222.0.15 () {54 vars in 927 bytes} [Mon Aug 22 15:46:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 22 15:46:15.604958 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f739957-e9b9-4b16-a2b7-9ed47b00710a None None] REQUEST_METHOD: `POST` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:46:15.605328 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f739957-e9b9-4b16-a2b7-9ed47b00710a None None] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:46:15.605683 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f739957-e9b9-4b16-a2b7-9ed47b00710a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:46:15.619036 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33e6719c-211a-4eb5-a2de-249b3d0f0d1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81767) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 15:46:15.619735 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33e6719c-211a-4eb5-a2de-249b3d0f0d1f None admin] REQUEST_METHOD: `GET` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:46:15.620015 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33e6719c-211a-4eb5-a2de-249b3d0f0d1f None admin] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:46:15.620295 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33e6719c-211a-4eb5-a2de-249b3d0f0d1f None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:46:15.628760 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.auth.core [None req-6f739957-e9b9-4b16-a2b7-9ed47b00710a None None] MFA Rules not processed for user `6b2199ae0fb0480aa917ada391f5ec54`. Rule list: `[]` (Enabled: `True`). {{(pid=81768) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 15:46:15.630038 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-6f739957-e9b9-4b16-a2b7-9ed47b00710a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 15:46:15.630792 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33e6719c-211a-4eb5-a2de-249b3d0f0d1f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 22 15:46:15.635953 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application [None req-33e6719c-211a-4eb5-a2de-249b3d0f0d1f None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 22 15:46:15.635953 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 15:46:15.635953 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 15:46:15.635953 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 15:46:15.635953 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 15:46:15.635953 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 15:46:15.635953 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 15:46:15.635953 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 15:46:15.635953 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 15:46:15.635953 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 15:46:15.635953 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 15:46:15.635953 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 15:46:15.635953 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 22 15:46:15.635953 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 22 15:46:15.635953 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 22 15:46:15.635953 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 22 15:46:15.636775 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 22 15:46:15.636775 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 22 15:46:15.636775 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 22 15:46:15.636775 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 22 15:46:15.636775 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 15:46:15.636775 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 15:46:15.636775 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 15:46:15.636775 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 15:46:15.636775 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 15:46:15.636775 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 15:46:15.636775 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 15:46:15.636775 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application with Lock( Aug 22 15:46:15.636775 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 15:46:15.636775 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return self._enter() Aug 22 15:46:15.636775 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 15:46:15.636775 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 15:46:15.638715 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 15:46:15.638715 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return self.creator() Aug 22 15:46:15.638715 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 15:46:15.638715 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application created_value = creator( Aug 22 15:46:15.638715 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 22 15:46:15.638715 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 22 15:46:15.638715 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 22 15:46:15.638715 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 22 15:46:15.638715 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 22 15:46:15.638715 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 22 15:46:15.638715 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 22 15:46:15.638715 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application Aug 22 15:46:15.638715 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 25/50] 10.222.0.15 () {56 vars in 1096 bytes} [Mon Aug 22 15:46:15 2022] GET /identity/v3/roles/admin => generated 83 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 15:46:15.643306 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33e6719c-211a-4eb5-a2de-249b3d0f0d1f None admin] Authenticating user token {{(pid=81767) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 15:46:15.698163 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 26/51] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Aug 22 15:46:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 22 15:46:15.728982 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG dbcounter [-] [81767] Writing DB stats keystone:SELECT=100 {{(pid=81767) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 15:46:15.754271 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33e6719c-211a-4eb5-a2de-249b3d0f0d1f None admin] Validating token access rules against request {{(pid=81767) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 15:46:15.788864 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a22af14-b08b-42b3-91c0-76f3e73744db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81767) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 15:46:15.789481 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a22af14-b08b-42b3-91c0-76f3e73744db None admin] REQUEST_METHOD: `GET` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:46:15.789750 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a22af14-b08b-42b3-91c0-76f3e73744db None admin] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:46:15.790019 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a22af14-b08b-42b3-91c0-76f3e73744db None admin] PATH_INFO: `/v3/roles` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:46:15.790687 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a22af14-b08b-42b3-91c0-76f3e73744db None admin] RBAC: Adding query filter params (name=admin) {{(pid=81767) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 15:46:15.790984 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a22af14-b08b-42b3-91c0-76f3e73744db None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 15:46:15.792393 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a22af14-b08b-42b3-91c0-76f3e73744db None admin] RBAC: Authorization granted {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 15:46:15.800811 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 26/52] 10.222.0.15 () {56 vars in 1099 bytes} [Mon Aug 22 15:46:15 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 22 15:46:15.806600 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f739957-e9b9-4b16-a2b7-9ed47b00710a None None] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 15:46:15.876631 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=100 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 15:46:15.910332 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f739957-e9b9-4b16-a2b7-9ed47b00710a None None] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 15:46:15.944658 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76a4c342-a6e3-456b-af46-a92f73bbacaf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 15:46:15.945288 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76a4c342-a6e3-456b-af46-a92f73bbacaf None admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:46:15.945571 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76a4c342-a6e3-456b-af46-a92f73bbacaf None admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:46:15.945844 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76a4c342-a6e3-456b-af46-a92f73bbacaf None admin] PATH_INFO: `/v3/users/6b2199ae0fb0480aa917ada391f5ec54` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:46:15.946962 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76a4c342-a6e3-456b-af46-a92f73bbacaf None admin] RBAC: Authorizing `identity:get_user(user_id=6b2199ae0fb0480aa917ada391f5ec54)` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 15:46:15.948474 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76a4c342-a6e3-456b-af46-a92f73bbacaf None admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 15:46:15.950058 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 27/53] 10.222.0.15 () {56 vars in 1177 bytes} [Mon Aug 22 15:46:15 2022] GET /identity/v3/users/6b2199ae0fb0480aa917ada391f5ec54 => generated 247 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 15:46:15.956314 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a22af14-b08b-42b3-91c0-76f3e73744db None admin] Authenticating user token {{(pid=81767) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 15:46:15.964186 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76a4c342-a6e3-456b-af46-a92f73bbacaf None admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 15:46:16.061746 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a22af14-b08b-42b3-91c0-76f3e73744db None admin] Validating token access rules against request {{(pid=81767) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 15:46:16.071016 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76a4c342-a6e3-456b-af46-a92f73bbacaf None admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 15:46:16.095563 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d5a785c-69b1-403b-9ca2-d296113179dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81767) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 15:46:16.096240 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d5a785c-69b1-403b-9ca2-d296113179dc None admin] REQUEST_METHOD: `GET` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:46:16.096526 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d5a785c-69b1-403b-9ca2-d296113179dc None admin] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:46:16.096803 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d5a785c-69b1-403b-9ca2-d296113179dc None admin] PATH_INFO: `/v3/domains/default` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:46:16.097935 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d5a785c-69b1-403b-9ca2-d296113179dc None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 15:46:16.099412 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d5a785c-69b1-403b-9ca2-d296113179dc None admin] RBAC: Authorization granted {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 15:46:16.101129 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 27/54] 10.222.0.15 () {56 vars in 1108 bytes} [Mon Aug 22 15:46:15 2022] GET /identity/v3/domains/default => generated 199 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 15:46:16.104702 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-514c34dd-c899-4e05-b800-ef7cd2903fde None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 15:46:16.105333 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-514c34dd-c899-4e05-b800-ef7cd2903fde None admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:46:16.105610 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-514c34dd-c899-4e05-b800-ef7cd2903fde None admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:46:16.105884 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-514c34dd-c899-4e05-b800-ef7cd2903fde None admin] PATH_INFO: `/v3/domains/default` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:46:16.106934 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d5a785c-69b1-403b-9ca2-d296113179dc None admin] Authenticating user token {{(pid=81767) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 15:46:16.107049 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-514c34dd-c899-4e05-b800-ef7cd2903fde None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 15:46:16.108538 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-514c34dd-c899-4e05-b800-ef7cd2903fde None admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 15:46:16.110103 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 28/55] 10.222.0.15 () {56 vars in 1108 bytes} [Mon Aug 22 15:46:15 2022] GET /identity/v3/domains/default => generated 199 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 15:46:16.110612 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG dbcounter [-] [81767] Writing DB stats keystone:SELECT=100 {{(pid=81767) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 15:46:16.115800 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-514c34dd-c899-4e05-b800-ef7cd2903fde None admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 15:46:16.119989 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=100 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 15:46:16.227450 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-514c34dd-c899-4e05-b800-ef7cd2903fde None admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 15:46:16.263970 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a6e1f04-eca2-4ccf-b190-3414cac24785 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 15:46:16.264572 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a6e1f04-eca2-4ccf-b190-3414cac24785 None admin] REQUEST_METHOD: `POST` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:46:16.264856 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a6e1f04-eca2-4ccf-b190-3414cac24785 None admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:46:16.265133 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a6e1f04-eca2-4ccf-b190-3414cac24785 None admin] PATH_INFO: `/v3/projects` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:46:16.265794 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a6e1f04-eca2-4ccf-b190-3414cac24785 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 15:46:16.267336 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a6e1f04-eca2-4ccf-b190-3414cac24785 None admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 15:46:16.316648 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d5a785c-69b1-403b-9ca2-d296113179dc None admin] Validating token access rules against request {{(pid=81767) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 15:46:16.349178 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4dc06a8-76b2-4f2c-9a8f-35fa018d9bcf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81767) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 15:46:16.349777 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4dc06a8-76b2-4f2c-9a8f-35fa018d9bcf None admin] REQUEST_METHOD: `GET` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:46:16.350054 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4dc06a8-76b2-4f2c-9a8f-35fa018d9bcf None admin] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:46:16.350324 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4dc06a8-76b2-4f2c-9a8f-35fa018d9bcf None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:46:16.351037 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4dc06a8-76b2-4f2c-9a8f-35fa018d9bcf None admin] RBAC: Adding query filter params (role.id=9d9848cfd22e4b089db443c350e56670, scope.domain.id=default, user.id=6b2199ae0fb0480aa917ada391f5ec54) {{(pid=81767) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 15:46:16.351374 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4dc06a8-76b2-4f2c-9a8f-35fa018d9bcf None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 15:46:16.352802 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4dc06a8-76b2-4f2c-9a8f-35fa018d9bcf None admin] RBAC: Authorization granted {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 15:46:16.361644 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 28/56] 10.222.0.15 () {56 vars in 1322 bytes} [Mon Aug 22 15:46:16 2022] GET /identity/v3/role_assignments?user.id=6b2199ae0fb0480aa917ada391f5ec54&scope.domain.id=default&role.id=9d9848cfd22e4b089db443c350e56670 => generated 530 bytes in 255 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 22 15:46:16.401547 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 29/57] 10.222.0.15 () {60 vars in 1140 bytes} [Mon Aug 22 15:46:16 2022] POST /identity/v3/projects => generated 298 bytes in 286 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 22 15:46:18.368835 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80348351-a37a-4943-90f8-f58c590e1e28 None None] REQUEST_METHOD: `GET` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:46:18.369319 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80348351-a37a-4943-90f8-f58c590e1e28 None None] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:46:18.369401 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80348351-a37a-4943-90f8-f58c590e1e28 None None] PATH_INFO: `/` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:46:18.370709 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 29/58] 10.222.0.15 () {54 vars in 927 bytes} [Mon Aug 22 15:46:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 15:46:18.375874 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9cd758f-0144-42e3-bb0d-c42835e16aeb None None] REQUEST_METHOD: `POST` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:46:18.376285 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9cd758f-0144-42e3-bb0d-c42835e16aeb None None] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:46:18.376422 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9cd758f-0144-42e3-bb0d-c42835e16aeb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:46:18.394446 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.auth.core [None req-f9cd758f-0144-42e3-bb0d-c42835e16aeb None None] MFA Rules not processed for user `6b2199ae0fb0480aa917ada391f5ec54`. Rule list: `[]` (Enabled: `True`). {{(pid=81768) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 15:46:18.395714 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-f9cd758f-0144-42e3-bb0d-c42835e16aeb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 15:46:18.446222 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 15:46:18.462605 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 30/59] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Aug 22 15:46:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 22 15:46:18.741327 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80348351-a37a-4943-90f8-f58c590e1e28 None None] Authenticating user token {{(pid=81767) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 15:46:18.808937 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG dbcounter [-] [81767] Writing DB stats keystone:SELECT=100 {{(pid=81767) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 15:46:18.843576 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80348351-a37a-4943-90f8-f58c590e1e28 None None] Validating token access rules against request {{(pid=81767) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 15:46:18.877228 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23a9198d-e680-4b70-a094-64bb7cb445ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81767) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 15:46:18.877885 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23a9198d-e680-4b70-a094-64bb7cb445ba None admin] REQUEST_METHOD: `GET` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:46:18.878167 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23a9198d-e680-4b70-a094-64bb7cb445ba None admin] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:46:18.878446 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23a9198d-e680-4b70-a094-64bb7cb445ba None admin] PATH_INFO: `/v3/domains/default` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:46:18.879614 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23a9198d-e680-4b70-a094-64bb7cb445ba None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 15:46:18.881073 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23a9198d-e680-4b70-a094-64bb7cb445ba None admin] RBAC: Authorization granted {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 15:46:18.882638 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 30/60] 10.222.0.15 () {56 vars in 1108 bytes} [Mon Aug 22 15:46:18 2022] GET /identity/v3/domains/default => generated 199 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 15:46:18.888793 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9cd758f-0144-42e3-bb0d-c42835e16aeb None None] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 15:46:18.991293 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9cd758f-0144-42e3-bb0d-c42835e16aeb None None] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 15:46:19.027595 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51d04238-003d-4218-9e9c-387dbe8d7da6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 15:46:19.028128 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51d04238-003d-4218-9e9c-387dbe8d7da6 None admin] REQUEST_METHOD: `POST` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:46:19.028409 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51d04238-003d-4218-9e9c-387dbe8d7da6 None admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:46:19.028706 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51d04238-003d-4218-9e9c-387dbe8d7da6 None admin] PATH_INFO: `/v3/users` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:46:19.029363 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51d04238-003d-4218-9e9c-387dbe8d7da6 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 15:46:19.030812 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51d04238-003d-4218-9e9c-387dbe8d7da6 None admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 15:46:19.177163 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 31/61] 10.222.0.15 () {60 vars in 1132 bytes} [Mon Aug 22 15:46:18 2022] POST /identity/v3/users => generated 275 bytes in 289 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 22 15:46:22.135559 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceda0f40-24c7-4b18-a530-8a0b1c0bf318 None None] REQUEST_METHOD: `GET` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:46:22.136127 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceda0f40-24c7-4b18-a530-8a0b1c0bf318 None None] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:46:22.136127 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceda0f40-24c7-4b18-a530-8a0b1c0bf318 None None] PATH_INFO: `/` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:46:22.137780 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 31/62] 10.222.0.15 () {54 vars in 927 bytes} [Mon Aug 22 15:46:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 15:46:22.144065 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1a13739-b9cf-4ed8-9ced-b39e7d83c61d None None] REQUEST_METHOD: `POST` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:46:22.144495 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1a13739-b9cf-4ed8-9ced-b39e7d83c61d None None] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:46:22.144681 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1a13739-b9cf-4ed8-9ced-b39e7d83c61d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:46:22.169305 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.auth.core [None req-b1a13739-b9cf-4ed8-9ced-b39e7d83c61d None None] MFA Rules not processed for user `6b2199ae0fb0480aa917ada391f5ec54`. Rule list: `[]` (Enabled: `True`). {{(pid=81768) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 15:46:22.170518 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-b1a13739-b9cf-4ed8-9ced-b39e7d83c61d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 15:46:22.211671 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c57ed971-b58c-4fd2-a5d2-e44cecea5b4d None None] REQUEST_METHOD: `GET` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:46:22.212272 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c57ed971-b58c-4fd2-a5d2-e44cecea5b4d None None] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:46:22.212743 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c57ed971-b58c-4fd2-a5d2-e44cecea5b4d None None] PATH_INFO: `/` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:46:22.214268 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 32/63] 10.222.0.15 () {54 vars in 927 bytes} [Mon Aug 22 15:46:22 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 22 15:46:22.222156 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 15:46:22.222683 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db0e04f7-9e5f-43dd-899d-dd77345f6615 None None] REQUEST_METHOD: `POST` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:46:22.223038 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db0e04f7-9e5f-43dd-899d-dd77345f6615 None None] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:46:22.223438 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db0e04f7-9e5f-43dd-899d-dd77345f6615 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:46:22.251804 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.auth.core [None req-db0e04f7-9e5f-43dd-899d-dd77345f6615 None None] MFA Rules not processed for user `6b2199ae0fb0480aa917ada391f5ec54`. Rule list: `[]` (Enabled: `True`). {{(pid=81767) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 15:46:22.253226 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.fernet_utils [None req-db0e04f7-9e5f-43dd-899d-dd77345f6615 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81767) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 15:46:22.273152 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 32/64] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Aug 22 15:46:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 22 15:46:22.353563 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 33/65] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Aug 22 15:46:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 22 15:46:22.514575 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0a35d5a-6cce-490d-8c3d-f57e7519e3bf None None] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:46:22.514778 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0a35d5a-6cce-490d-8c3d-f57e7519e3bf None None] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:46:22.515303 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0a35d5a-6cce-490d-8c3d-f57e7519e3bf None None] PATH_INFO: `/` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:46:22.516898 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 33/66] 10.222.0.15 () {54 vars in 927 bytes} [Mon Aug 22 15:46:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 15:46:22.522249 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa356a76-3930-48d3-9598-1cec5083804f None None] REQUEST_METHOD: `POST` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:46:22.522792 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa356a76-3930-48d3-9598-1cec5083804f None None] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:46:22.523258 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa356a76-3930-48d3-9598-1cec5083804f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:46:22.546789 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.auth.core [None req-fa356a76-3930-48d3-9598-1cec5083804f None None] MFA Rules not processed for user `6b2199ae0fb0480aa917ada391f5ec54`. Rule list: `[]` (Enabled: `True`). {{(pid=81767) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 15:46:22.548375 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.fernet_utils [None req-fa356a76-3930-48d3-9598-1cec5083804f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81767) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 15:46:22.616329 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG dbcounter [-] [81767] Writing DB stats keystone:SELECT=100 {{(pid=81767) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 15:46:22.626639 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 34/67] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Aug 22 15:46:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 22 15:46:22.714851 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0a35d5a-6cce-490d-8c3d-f57e7519e3bf None None] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 15:46:22.749092 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a608c940-7645-4337-a46c-cc22ee4cda2c None None] REQUEST_METHOD: `GET` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:46:22.749381 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a608c940-7645-4337-a46c-cc22ee4cda2c None None] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:46:22.749663 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a608c940-7645-4337-a46c-cc22ee4cda2c None None] PATH_INFO: `/` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:46:22.751011 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 35/68] 10.222.0.15 () {54 vars in 927 bytes} [Mon Aug 22 15:46:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 22 15:46:22.765014 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1ef19ac-bacf-447d-ad59-21cea0eece42 None None] REQUEST_METHOD: `POST` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:46:22.765260 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1ef19ac-bacf-447d-ad59-21cea0eece42 None None] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:46:22.765485 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1ef19ac-bacf-447d-ad59-21cea0eece42 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:46:22.792294 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.auth.core [None req-d1ef19ac-bacf-447d-ad59-21cea0eece42 None None] MFA Rules not processed for user `6b2199ae0fb0480aa917ada391f5ec54`. Rule list: `[]` (Enabled: `True`). {{(pid=81767) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 15:46:22.793493 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.fernet_utils [None req-d1ef19ac-bacf-447d-ad59-21cea0eece42 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81767) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 15:46:22.864037 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0a35d5a-6cce-490d-8c3d-f57e7519e3bf None None] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 15:46:22.868701 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 36/69] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Aug 22 15:46:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 22 15:46:22.869786 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1ef19ac-bacf-447d-ad59-21cea0eece42 None None] Authenticating user token {{(pid=81767) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 15:46:22.900395 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5f864ae-0ed6-4720-8e44-c05994271da9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 15:46:22.901270 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5f864ae-0ed6-4720-8e44-c05994271da9 None admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:46:22.901699 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5f864ae-0ed6-4720-8e44-c05994271da9 None admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:46:22.902256 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5f864ae-0ed6-4720-8e44-c05994271da9 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:46:22.913965 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5f864ae-0ed6-4720-8e44-c05994271da9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 22 15:46:22.919776 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application [None req-a5f864ae-0ed6-4720-8e44-c05994271da9 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 22 15:46:22.919776 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 15:46:22.919776 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 15:46:22.919776 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 15:46:22.919776 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 15:46:22.919776 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 15:46:22.919776 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 15:46:22.919776 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 15:46:22.919776 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 15:46:22.919776 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 15:46:22.919776 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 15:46:22.919776 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 15:46:22.919776 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 22 15:46:22.919776 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 22 15:46:22.919776 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 22 15:46:22.919776 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 22 15:46:22.920614 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 22 15:46:22.920614 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 22 15:46:22.920614 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 22 15:46:22.920614 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 22 15:46:22.920614 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 15:46:22.920614 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 15:46:22.920614 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 15:46:22.920614 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 15:46:22.920614 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 15:46:22.920614 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 15:46:22.920614 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 15:46:22.920614 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application with Lock( Aug 22 15:46:22.920614 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 15:46:22.920614 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self._enter() Aug 22 15:46:22.920614 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 15:46:22.920614 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 15:46:22.921360 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 15:46:22.921360 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self.creator() Aug 22 15:46:22.921360 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 15:46:22.921360 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application created_value = creator( Aug 22 15:46:22.921360 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 22 15:46:22.921360 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 22 15:46:22.921360 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 22 15:46:22.921360 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 22 15:46:22.921360 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 22 15:46:22.921360 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 22 15:46:22.921360 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 22 15:46:22.921360 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application Aug 22 15:46:22.923980 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 34/70] 10.222.0.15 () {56 vars in 1099 bytes} [Mon Aug 22 15:46:22 2022] GET /identity/v3/roles/member => generated 84 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 15:46:22.933542 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5f864ae-0ed6-4720-8e44-c05994271da9 None admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 15:46:22.975140 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=100 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 15:46:22.987945 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1ef19ac-bacf-447d-ad59-21cea0eece42 None None] Validating token access rules against request {{(pid=81767) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 15:46:23.017561 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG dbcounter [-] [81767] Writing DB stats keystone:SELECT=100 {{(pid=81767) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 15:46:23.028091 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcf13bb2-2ca3-4718-a1a1-b4d0db72bb65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81767) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 15:46:23.029499 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcf13bb2-2ca3-4718-a1a1-b4d0db72bb65 None admin] REQUEST_METHOD: `GET` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:46:23.029923 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcf13bb2-2ca3-4718-a1a1-b4d0db72bb65 None admin] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:46:23.030381 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcf13bb2-2ca3-4718-a1a1-b4d0db72bb65 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:46:23.042324 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcf13bb2-2ca3-4718-a1a1-b4d0db72bb65 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 22 15:46:23.048412 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application [None req-fcf13bb2-2ca3-4718-a1a1-b4d0db72bb65 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 22 15:46:23.048412 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 15:46:23.048412 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 15:46:23.048412 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 15:46:23.048412 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 15:46:23.048412 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 15:46:23.048412 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 15:46:23.048412 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 15:46:23.048412 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 15:46:23.048412 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 15:46:23.048412 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 15:46:23.048412 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 15:46:23.048412 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 22 15:46:23.048412 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 22 15:46:23.048412 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 22 15:46:23.048412 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 22 15:46:23.049466 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 22 15:46:23.049466 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 22 15:46:23.049466 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 22 15:46:23.049466 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 22 15:46:23.049466 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 15:46:23.049466 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 15:46:23.049466 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 15:46:23.049466 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 15:46:23.049466 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 15:46:23.049466 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 15:46:23.049466 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 15:46:23.049466 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application with Lock( Aug 22 15:46:23.049466 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 15:46:23.049466 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return self._enter() Aug 22 15:46:23.049466 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 15:46:23.049466 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 15:46:23.050402 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 15:46:23.050402 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return self.creator() Aug 22 15:46:23.050402 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 15:46:23.050402 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application created_value = creator( Aug 22 15:46:23.050402 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 22 15:46:23.050402 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 22 15:46:23.050402 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 22 15:46:23.050402 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 22 15:46:23.050402 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 22 15:46:23.050402 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 22 15:46:23.050402 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 22 15:46:23.050402 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application Aug 22 15:46:23.052415 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 37/71] 10.222.0.15 () {56 vars in 1096 bytes} [Mon Aug 22 15:46:22 2022] GET /identity/v3/roles/admin => generated 83 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 15:46:23.058556 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcf13bb2-2ca3-4718-a1a1-b4d0db72bb65 None admin] Authenticating user token {{(pid=81767) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 15:46:23.064074 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5f864ae-0ed6-4720-8e44-c05994271da9 None admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 15:46:23.111216 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a920044b-41fb-4951-b2dc-7ca35c6e8991 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 15:46:23.112017 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a920044b-41fb-4951-b2dc-7ca35c6e8991 None admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:46:23.112405 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a920044b-41fb-4951-b2dc-7ca35c6e8991 None admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:46:23.112781 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a920044b-41fb-4951-b2dc-7ca35c6e8991 None admin] PATH_INFO: `/v3/roles` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:46:23.113585 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a920044b-41fb-4951-b2dc-7ca35c6e8991 None admin] RBAC: Adding query filter params (name=member) {{(pid=81768) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 15:46:23.113990 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a920044b-41fb-4951-b2dc-7ca35c6e8991 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 15:46:23.115584 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a920044b-41fb-4951-b2dc-7ca35c6e8991 None admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 15:46:23.125028 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 35/72] 10.222.0.15 () {56 vars in 1101 bytes} [Mon Aug 22 15:46:22 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 22 15:46:23.126027 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a920044b-41fb-4951-b2dc-7ca35c6e8991 None admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 15:46:23.172705 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcf13bb2-2ca3-4718-a1a1-b4d0db72bb65 None admin] Validating token access rules against request {{(pid=81767) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 15:46:23.210072 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9df0da24-472a-45bd-8b6a-e2f076368a69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81767) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 15:46:23.210693 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9df0da24-472a-45bd-8b6a-e2f076368a69 None admin] REQUEST_METHOD: `GET` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:46:23.210971 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9df0da24-472a-45bd-8b6a-e2f076368a69 None admin] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:46:23.211240 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9df0da24-472a-45bd-8b6a-e2f076368a69 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:46:23.222144 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9df0da24-472a-45bd-8b6a-e2f076368a69 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 22 15:46:23.226819 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application [None req-9df0da24-472a-45bd-8b6a-e2f076368a69 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 22 15:46:23.226819 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 15:46:23.226819 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 15:46:23.226819 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 15:46:23.226819 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 15:46:23.226819 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 15:46:23.226819 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 15:46:23.226819 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 15:46:23.226819 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 15:46:23.226819 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 15:46:23.226819 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 15:46:23.226819 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 15:46:23.226819 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 22 15:46:23.226819 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 22 15:46:23.226819 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 22 15:46:23.226819 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 22 15:46:23.227708 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 22 15:46:23.227708 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 22 15:46:23.227708 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 22 15:46:23.227708 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 22 15:46:23.227708 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 15:46:23.227708 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 15:46:23.227708 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 15:46:23.227708 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 15:46:23.227708 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 15:46:23.227708 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 15:46:23.227708 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 15:46:23.227708 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application with Lock( Aug 22 15:46:23.227708 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 15:46:23.227708 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return self._enter() Aug 22 15:46:23.227708 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 15:46:23.227708 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 15:46:23.228456 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 15:46:23.228456 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return self.creator() Aug 22 15:46:23.228456 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 15:46:23.228456 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application created_value = creator( Aug 22 15:46:23.228456 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 22 15:46:23.228456 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 22 15:46:23.228456 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 22 15:46:23.228456 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 22 15:46:23.228456 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 22 15:46:23.228456 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 22 15:46:23.228456 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 22 15:46:23.228456 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application Aug 22 15:46:23.228456 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 38/73] 10.222.0.15 () {56 vars in 1114 bytes} [Mon Aug 22 15:46:23 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 15:46:23.229501 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-388df2e4-81ea-4f00-a415-7694033a9a0d None None] REQUEST_METHOD: `GET` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:46:23.229782 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-388df2e4-81ea-4f00-a415-7694033a9a0d None None] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:46:23.230041 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-388df2e4-81ea-4f00-a415-7694033a9a0d None None] PATH_INFO: `/` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:46:23.231266 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 39/74] 10.222.0.15 () {54 vars in 927 bytes} [Mon Aug 22 15:46:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 15:46:23.236090 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-388df2e4-81ea-4f00-a415-7694033a9a0d None None] Authenticating user token {{(pid=81767) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 15:46:23.237337 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=100 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 15:46:23.245082 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a920044b-41fb-4951-b2dc-7ca35c6e8991 None admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 15:46:23.279532 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d4c13d5-2ecc-4589-9cda-51c6610e74de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 15:46:23.280233 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d4c13d5-2ecc-4589-9cda-51c6610e74de None admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:46:23.280920 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d4c13d5-2ecc-4589-9cda-51c6610e74de None admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:46:23.281348 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d4c13d5-2ecc-4589-9cda-51c6610e74de None admin] PATH_INFO: `/v3/roles` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:46:23.282521 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d4c13d5-2ecc-4589-9cda-51c6610e74de None admin] RBAC: Adding query filter params (name=admin) {{(pid=81768) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 15:46:23.282976 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d4c13d5-2ecc-4589-9cda-51c6610e74de None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 15:46:23.283938 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG dbcounter [-] [81767] Writing DB stats keystone:SELECT=100 {{(pid=81767) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 15:46:23.284276 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d4c13d5-2ecc-4589-9cda-51c6610e74de None admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 15:46:23.292683 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 36/75] 10.222.0.15 () {56 vars in 1099 bytes} [Mon Aug 22 15:46:23 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 15:46:23.297585 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d4c13d5-2ecc-4589-9cda-51c6610e74de None admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 15:46:23.344185 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-388df2e4-81ea-4f00-a415-7694033a9a0d None None] Validating token access rules against request {{(pid=81767) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 15:46:23.380408 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f41b6c7-1c55-48fd-a261-5829bed1f2ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81767) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 15:46:23.381039 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f41b6c7-1c55-48fd-a261-5829bed1f2ff None admin] REQUEST_METHOD: `GET` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:46:23.381323 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f41b6c7-1c55-48fd-a261-5829bed1f2ff None admin] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:46:23.381597 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f41b6c7-1c55-48fd-a261-5829bed1f2ff None admin] PATH_INFO: `/v3/users/ad6aab8e70974f939bda68c2060aace5` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:46:23.397094 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f41b6c7-1c55-48fd-a261-5829bed1f2ff None admin] RBAC: Authorizing `identity:get_user(user_id=ad6aab8e70974f939bda68c2060aace5)` {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 15:46:23.397694 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d4c13d5-2ecc-4589-9cda-51c6610e74de None admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 15:46:23.398565 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f41b6c7-1c55-48fd-a261-5829bed1f2ff None admin] RBAC: Authorization granted {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 15:46:23.400314 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 40/76] 10.222.0.15 () {56 vars in 1177 bytes} [Mon Aug 22 15:46:23 2022] GET /identity/v3/users/ad6aab8e70974f939bda68c2060aace5 => generated 275 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 15:46:23.401083 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f41b6c7-1c55-48fd-a261-5829bed1f2ff None admin] Authenticating user token {{(pid=81767) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 15:46:23.436132 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78d8fcaa-e255-462b-b2ea-8300df49dad4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 15:46:23.436768 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78d8fcaa-e255-462b-b2ea-8300df49dad4 None admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:46:23.437051 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78d8fcaa-e255-462b-b2ea-8300df49dad4 None admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:46:23.437324 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78d8fcaa-e255-462b-b2ea-8300df49dad4 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:46:23.447554 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78d8fcaa-e255-462b-b2ea-8300df49dad4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 22 15:46:23.453524 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application [None req-78d8fcaa-e255-462b-b2ea-8300df49dad4 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 22 15:46:23.453524 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 15:46:23.453524 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 15:46:23.453524 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 15:46:23.453524 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 15:46:23.453524 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 15:46:23.453524 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 15:46:23.453524 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 15:46:23.453524 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 15:46:23.453524 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 15:46:23.453524 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 15:46:23.453524 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 15:46:23.453524 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 22 15:46:23.453524 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 22 15:46:23.453524 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 22 15:46:23.453524 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 22 15:46:23.454324 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 22 15:46:23.454324 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 22 15:46:23.454324 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 22 15:46:23.454324 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 22 15:46:23.454324 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 15:46:23.454324 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 15:46:23.454324 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 15:46:23.454324 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 15:46:23.454324 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 15:46:23.454324 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 15:46:23.454324 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 15:46:23.454324 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application with Lock( Aug 22 15:46:23.454324 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 15:46:23.454324 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self._enter() Aug 22 15:46:23.454324 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 15:46:23.454324 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 15:46:23.455076 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 15:46:23.455076 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self.creator() Aug 22 15:46:23.455076 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 15:46:23.455076 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application created_value = creator( Aug 22 15:46:23.455076 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 22 15:46:23.455076 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 22 15:46:23.455076 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 22 15:46:23.455076 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 22 15:46:23.455076 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 22 15:46:23.455076 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 22 15:46:23.455076 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 22 15:46:23.455076 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application Aug 22 15:46:23.455076 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 37/77] 10.222.0.15 () {56 vars in 1099 bytes} [Mon Aug 22 15:46:23 2022] GET /identity/v3/roles/member => generated 84 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 15:46:23.456374 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d96dd0d0-36c7-43d9-8caa-ab2482014b0c None None] REQUEST_METHOD: `POST` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:46:23.456374 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d96dd0d0-36c7-43d9-8caa-ab2482014b0c None None] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:46:23.456374 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d96dd0d0-36c7-43d9-8caa-ab2482014b0c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:46:23.474201 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.auth.core [None req-d96dd0d0-36c7-43d9-8caa-ab2482014b0c None None] MFA Rules not processed for user `6b2199ae0fb0480aa917ada391f5ec54`. Rule list: `[]` (Enabled: `True`). {{(pid=81768) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 15:46:23.475441 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-d96dd0d0-36c7-43d9-8caa-ab2482014b0c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 15:46:23.486382 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=100 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 15:46:23.504349 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f41b6c7-1c55-48fd-a261-5829bed1f2ff None admin] Validating token access rules against request {{(pid=81767) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 15:46:23.506712 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG dbcounter [-] [81767] Writing DB stats keystone:SELECT=100 {{(pid=81767) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 15:46:23.538012 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c067b29f-6da4-4cd2-8ee2-3b3b6da31cfb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81767) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 15:46:23.538630 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c067b29f-6da4-4cd2-8ee2-3b3b6da31cfb None admin] REQUEST_METHOD: `GET` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:46:23.538996 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c067b29f-6da4-4cd2-8ee2-3b3b6da31cfb None admin] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:46:23.547711 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c067b29f-6da4-4cd2-8ee2-3b3b6da31cfb None admin] PATH_INFO: `/v3/roles` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:46:23.547711 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c067b29f-6da4-4cd2-8ee2-3b3b6da31cfb None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81767) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 15:46:23.547711 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c067b29f-6da4-4cd2-8ee2-3b3b6da31cfb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 15:46:23.547711 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c067b29f-6da4-4cd2-8ee2-3b3b6da31cfb None admin] RBAC: Authorization granted {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 15:46:23.548039 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 38/78] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Aug 22 15:46:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 22 15:46:23.548039 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d96dd0d0-36c7-43d9-8caa-ab2482014b0c None None] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 15:46:23.550277 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 41/79] 10.222.0.15 () {56 vars in 1111 bytes} [Mon Aug 22 15:46:23 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 15:46:23.551160 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c067b29f-6da4-4cd2-8ee2-3b3b6da31cfb None admin] Authenticating user token {{(pid=81767) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 15:46:23.665201 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c067b29f-6da4-4cd2-8ee2-3b3b6da31cfb None admin] Validating token access rules against request {{(pid=81767) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 15:46:23.666093 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d96dd0d0-36c7-43d9-8caa-ab2482014b0c None None] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 15:46:23.700287 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2e805cc-77ff-4122-b1b1-4403073a2618 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81767) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 15:46:23.701438 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2e805cc-77ff-4122-b1b1-4403073a2618 None admin] REQUEST_METHOD: `GET` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:46:23.701749 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2e805cc-77ff-4122-b1b1-4403073a2618 None admin] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:46:23.701993 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2e805cc-77ff-4122-b1b1-4403073a2618 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:46:23.716035 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-893735d5-c1af-4ea6-9a83-483bc19164b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 15:46:23.716662 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-893735d5-c1af-4ea6-9a83-483bc19164b3 None admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:46:23.716940 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-893735d5-c1af-4ea6-9a83-483bc19164b3 None admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:46:23.717245 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-893735d5-c1af-4ea6-9a83-483bc19164b3 None admin] PATH_INFO: `/v3/users/6b2199ae0fb0480aa917ada391f5ec54` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:46:23.718504 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-893735d5-c1af-4ea6-9a83-483bc19164b3 None admin] RBAC: Authorizing `identity:get_user(user_id=6b2199ae0fb0480aa917ada391f5ec54)` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 15:46:23.719994 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-893735d5-c1af-4ea6-9a83-483bc19164b3 None admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 15:46:23.726224 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 39/80] 10.222.0.15 () {56 vars in 1177 bytes} [Mon Aug 22 15:46:23 2022] GET /identity/v3/users/6b2199ae0fb0480aa917ada391f5ec54 => generated 247 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 15:46:23.726224 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-893735d5-c1af-4ea6-9a83-483bc19164b3 None admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 15:46:23.740916 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=100 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 15:46:23.743583 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG dbcounter [-] [81767] Writing DB stats keystone:SELECT=100 {{(pid=81767) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 15:46:23.777428 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2e805cc-77ff-4122-b1b1-4403073a2618 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 15:46:23.778935 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2e805cc-77ff-4122-b1b1-4403073a2618 None admin] RBAC: Authorization granted {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 15:46:23.832278 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-893735d5-c1af-4ea6-9a83-483bc19164b3 None admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 15:46:23.866218 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b517d69a-ee68-40f4-bd30-8d5320091b97 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 15:46:23.866813 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b517d69a-ee68-40f4-bd30-8d5320091b97 None admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:46:23.867093 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b517d69a-ee68-40f4-bd30-8d5320091b97 None admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:46:23.867398 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b517d69a-ee68-40f4-bd30-8d5320091b97 None admin] PATH_INFO: `/v3/roles` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:46:23.868062 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b517d69a-ee68-40f4-bd30-8d5320091b97 None admin] RBAC: Adding query filter params (name=member) {{(pid=81768) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 15:46:23.868361 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b517d69a-ee68-40f4-bd30-8d5320091b97 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 15:46:23.869738 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b517d69a-ee68-40f4-bd30-8d5320091b97 None admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 15:46:23.878448 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 40/81] 10.222.0.15 () {56 vars in 1101 bytes} [Mon Aug 22 15:46:23 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 15:46:23.879128 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 42/82] 10.222.0.15 () {58 vars in 1282 bytes} [Mon Aug 22 15:46:23 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 328 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 22 15:46:23.879424 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b517d69a-ee68-40f4-bd30-8d5320091b97 None admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 15:46:23.879920 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2e805cc-77ff-4122-b1b1-4403073a2618 None admin] Authenticating user token {{(pid=81767) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 15:46:23.947889 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG dbcounter [-] [81767] Writing DB stats keystone:SELECT=100 {{(pid=81767) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 15:46:23.966194 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=100 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 15:46:23.983475 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2e805cc-77ff-4122-b1b1-4403073a2618 None admin] Validating token access rules against request {{(pid=81767) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 15:46:23.989670 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b517d69a-ee68-40f4-bd30-8d5320091b97 None admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 15:46:24.017555 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20c6d4cd-2de0-493d-b28b-486169be8878 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81767) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 15:46:24.018161 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20c6d4cd-2de0-493d-b28b-486169be8878 None admin] REQUEST_METHOD: `GET` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:46:24.018442 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20c6d4cd-2de0-493d-b28b-486169be8878 None admin] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:46:24.018708 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20c6d4cd-2de0-493d-b28b-486169be8878 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:46:24.025961 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82db215b-5aaa-4e50-81a9-fcd15a616f70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 15:46:24.026591 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82db215b-5aaa-4e50-81a9-fcd15a616f70 None admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:46:24.026872 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82db215b-5aaa-4e50-81a9-fcd15a616f70 None admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:46:24.027153 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82db215b-5aaa-4e50-81a9-fcd15a616f70 None admin] PATH_INFO: `/v3/users/ad6aab8e70974f939bda68c2060aace5` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:46:24.028765 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82db215b-5aaa-4e50-81a9-fcd15a616f70 None admin] RBAC: Authorizing `identity:get_user(user_id=ad6aab8e70974f939bda68c2060aace5)` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 15:46:24.030939 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82db215b-5aaa-4e50-81a9-fcd15a616f70 None admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 15:46:24.032611 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 41/83] 10.222.0.15 () {56 vars in 1177 bytes} [Mon Aug 22 15:46:23 2022] GET /identity/v3/users/ad6aab8e70974f939bda68c2060aace5 => generated 275 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 15:46:24.037584 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82db215b-5aaa-4e50-81a9-fcd15a616f70 None admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 15:46:24.087007 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20c6d4cd-2de0-493d-b28b-486169be8878 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 15:46:24.088393 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20c6d4cd-2de0-493d-b28b-486169be8878 None admin] RBAC: Authorization granted {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 15:46:24.146979 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82db215b-5aaa-4e50-81a9-fcd15a616f70 None admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 15:46:24.151738 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG dbcounter [-] [81767] Writing DB stats keystone:SELECT=100 {{(pid=81767) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 15:46:24.182777 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e26042b0-bfa2-400c-88af-9c32f38741c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 15:46:24.183445 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e26042b0-bfa2-400c-88af-9c32f38741c4 None admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:46:24.183730 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e26042b0-bfa2-400c-88af-9c32f38741c4 None admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:46:24.184012 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e26042b0-bfa2-400c-88af-9c32f38741c4 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:46:24.185272 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e26042b0-bfa2-400c-88af-9c32f38741c4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 15:46:24.186733 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e26042b0-bfa2-400c-88af-9c32f38741c4 None admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 15:46:24.188317 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 43/84] 10.222.0.15 () {58 vars in 1282 bytes} [Mon Aug 22 15:46:23 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 22 15:46:24.188746 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 42/85] 10.222.0.15 () {56 vars in 1108 bytes} [Mon Aug 22 15:46:24 2022] GET /identity/v3/domains/default => generated 199 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 15:46:24.189079 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20c6d4cd-2de0-493d-b28b-486169be8878 None admin] Authenticating user token {{(pid=81767) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 15:46:24.189184 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e26042b0-bfa2-400c-88af-9c32f38741c4 None admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 15:46:24.204412 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=100 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 15:46:24.292003 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20c6d4cd-2de0-493d-b28b-486169be8878 None admin] Validating token access rules against request {{(pid=81767) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 15:46:24.292206 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e26042b0-bfa2-400c-88af-9c32f38741c4 None admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 15:46:24.327156 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a96287bf-bbc2-4eac-b0fe-2a0cff55554e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 15:46:24.327831 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a96287bf-bbc2-4eac-b0fe-2a0cff55554e None admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:46:24.328112 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a96287bf-bbc2-4eac-b0fe-2a0cff55554e None admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:46:24.328282 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f619ba9-d4dd-4e92-8bba-deb688dc746d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81767) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 15:46:24.328392 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a96287bf-bbc2-4eac-b0fe-2a0cff55554e None admin] PATH_INFO: `/v3/projects/43171f5fd3454380b88625276d7375da` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:46:24.328911 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f619ba9-d4dd-4e92-8bba-deb688dc746d None admin] REQUEST_METHOD: `GET` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:46:24.329195 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f619ba9-d4dd-4e92-8bba-deb688dc746d None admin] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:46:24.329467 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f619ba9-d4dd-4e92-8bba-deb688dc746d None admin] PATH_INFO: `/v3/users/ad6aab8e70974f939bda68c2060aace5` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:46:24.329559 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a96287bf-bbc2-4eac-b0fe-2a0cff55554e None admin] RBAC: Authorizing `identity:get_project(project_id=43171f5fd3454380b88625276d7375da)` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 15:46:24.330606 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f619ba9-d4dd-4e92-8bba-deb688dc746d None admin] RBAC: Authorizing `identity:get_user(user_id=ad6aab8e70974f939bda68c2060aace5)` {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 15:46:24.331005 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a96287bf-bbc2-4eac-b0fe-2a0cff55554e None admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 15:46:24.332066 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f619ba9-d4dd-4e92-8bba-deb688dc746d None admin] RBAC: Authorization granted {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 15:46:24.332762 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 43/86] 10.222.0.15 () {56 vars in 1186 bytes} [Mon Aug 22 15:46:24 2022] GET /identity/v3/projects/43171f5fd3454380b88625276d7375da => generated 298 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 15:46:24.335917 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 44/87] 10.222.0.15 () {56 vars in 1177 bytes} [Mon Aug 22 15:46:24 2022] GET /identity/v3/users/ad6aab8e70974f939bda68c2060aace5 => generated 275 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 15:46:24.335917 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f619ba9-d4dd-4e92-8bba-deb688dc746d None admin] Authenticating user token {{(pid=81767) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 15:46:24.338662 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a96287bf-bbc2-4eac-b0fe-2a0cff55554e None admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 15:46:24.376574 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG dbcounter [-] [81767] Writing DB stats keystone:SELECT=100 {{(pid=81767) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 15:46:24.424722 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=100 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 15:46:24.443831 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f619ba9-d4dd-4e92-8bba-deb688dc746d None admin] Validating token access rules against request {{(pid=81767) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 15:46:24.444436 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a96287bf-bbc2-4eac-b0fe-2a0cff55554e None admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 15:46:24.478948 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5add0b87-9898-49c7-ad0f-66502f5c916c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81767) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 15:46:24.479342 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2b464d4-e42d-43b1-88ca-cfee3b829078 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 15:46:24.479671 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5add0b87-9898-49c7-ad0f-66502f5c916c None admin] REQUEST_METHOD: `GET` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:46:24.479774 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2b464d4-e42d-43b1-88ca-cfee3b829078 None admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:46:24.479947 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5add0b87-9898-49c7-ad0f-66502f5c916c None admin] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:46:24.480059 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2b464d4-e42d-43b1-88ca-cfee3b829078 None admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:46:24.480216 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5add0b87-9898-49c7-ad0f-66502f5c916c None admin] PATH_INFO: `/v3/projects/43171f5fd3454380b88625276d7375da` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:46:24.480332 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2b464d4-e42d-43b1-88ca-cfee3b829078 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:46:24.481376 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5add0b87-9898-49c7-ad0f-66502f5c916c None admin] RBAC: Authorizing `identity:get_project(project_id=43171f5fd3454380b88625276d7375da)` {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 15:46:24.482821 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5add0b87-9898-49c7-ad0f-66502f5c916c None admin] RBAC: Authorization granted {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 15:46:24.484472 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 45/88] 10.222.0.15 () {56 vars in 1186 bytes} [Mon Aug 22 15:46:24 2022] GET /identity/v3/projects/43171f5fd3454380b88625276d7375da => generated 298 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 15:46:24.485217 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5add0b87-9898-49c7-ad0f-66502f5c916c None admin] Authenticating user token {{(pid=81767) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 15:46:24.549369 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2b464d4-e42d-43b1-88ca-cfee3b829078 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 15:46:24.550901 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2b464d4-e42d-43b1-88ca-cfee3b829078 None admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 15:46:24.594413 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5add0b87-9898-49c7-ad0f-66502f5c916c None admin] Validating token access rules against request {{(pid=81767) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 15:46:24.595906 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG dbcounter [-] [81767] Writing DB stats keystone:SELECT=100 {{(pid=81767) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 15:46:24.629527 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcbb491e-6ec2-4911-bb81-4f81ae284fde None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81767) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 15:46:24.630141 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcbb491e-6ec2-4911-bb81-4f81ae284fde None admin] REQUEST_METHOD: `POST` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:46:24.630441 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcbb491e-6ec2-4911-bb81-4f81ae284fde None admin] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:46:24.630736 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcbb491e-6ec2-4911-bb81-4f81ae284fde None admin] PATH_INFO: `/v3/users` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:46:24.631506 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcbb491e-6ec2-4911-bb81-4f81ae284fde None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 15:46:24.632648 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=100 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 15:46:24.632995 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcbb491e-6ec2-4911-bb81-4f81ae284fde None admin] RBAC: Authorization granted {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 15:46:24.652160 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 44/89] 10.222.0.15 () {58 vars in 1282 bytes} [Mon Aug 22 15:46:24 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 318 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 22 15:46:24.652985 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2b464d4-e42d-43b1-88ca-cfee3b829078 None admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 15:46:24.750151 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2b464d4-e42d-43b1-88ca-cfee3b829078 None admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 15:46:24.760116 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 46/90] 10.222.0.15 () {60 vars in 1132 bytes} [Mon Aug 22 15:46:24 2022] POST /identity/v3/users => generated 289 bytes in 275 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 22 15:46:24.760885 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcbb491e-6ec2-4911-bb81-4f81ae284fde None admin] Authenticating user token {{(pid=81767) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 15:46:24.788684 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-945cec82-a31d-44ae-b5c7-ebd09f1a68c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 15:46:24.789304 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-945cec82-a31d-44ae-b5c7-ebd09f1a68c8 None admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:46:24.789607 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-945cec82-a31d-44ae-b5c7-ebd09f1a68c8 None admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:46:24.789881 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-945cec82-a31d-44ae-b5c7-ebd09f1a68c8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:46:24.790626 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-945cec82-a31d-44ae-b5c7-ebd09f1a68c8 None admin] RBAC: Adding query filter params (role.id=e6fc1ee3fb974125ba9b4fe55b12b45e, scope.project.id=43171f5fd3454380b88625276d7375da, user.id=ad6aab8e70974f939bda68c2060aace5) {{(pid=81768) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 15:46:24.790930 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-945cec82-a31d-44ae-b5c7-ebd09f1a68c8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 15:46:24.792491 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-945cec82-a31d-44ae-b5c7-ebd09f1a68c8 None admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 15:46:24.811307 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 45/91] 10.222.0.15 () {56 vars in 1374 bytes} [Mon Aug 22 15:46:24 2022] GET /identity/v3/role_assignments?user.id=ad6aab8e70974f939bda68c2060aace5&scope.project.id=43171f5fd3454380b88625276d7375da&role.id=e6fc1ee3fb974125ba9b4fe55b12b45e => generated 259 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 15:46:24.811307 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-945cec82-a31d-44ae-b5c7-ebd09f1a68c8 None admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 15:46:24.887764 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=100 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 15:46:24.905172 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcbb491e-6ec2-4911-bb81-4f81ae284fde None admin] Validating token access rules against request {{(pid=81767) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 15:46:24.965141 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cddd1a1-33ee-4f71-8dd9-d5b9cb08a05b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81767) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 15:46:24.966071 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cddd1a1-33ee-4f71-8dd9-d5b9cb08a05b None admin] REQUEST_METHOD: `GET` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:46:24.966494 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cddd1a1-33ee-4f71-8dd9-d5b9cb08a05b None admin] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:46:24.967022 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cddd1a1-33ee-4f71-8dd9-d5b9cb08a05b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:46:24.977345 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-945cec82-a31d-44ae-b5c7-ebd09f1a68c8 None admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 15:46:24.996422 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG dbcounter [-] [81767] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81767) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 15:46:25.020033 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a76530ae-6d5c-4acf-bd28-2933a29cea0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 15:46:25.020815 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a76530ae-6d5c-4acf-bd28-2933a29cea0d None admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:46:25.021967 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a76530ae-6d5c-4acf-bd28-2933a29cea0d None admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:46:25.022252 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a76530ae-6d5c-4acf-bd28-2933a29cea0d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:46:25.023068 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a76530ae-6d5c-4acf-bd28-2933a29cea0d None admin] RBAC: Adding query filter params (role.id=9d9848cfd22e4b089db443c350e56670, scope.project.id=43171f5fd3454380b88625276d7375da, user.id=6b2199ae0fb0480aa917ada391f5ec54) {{(pid=81768) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 15:46:25.023574 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a76530ae-6d5c-4acf-bd28-2933a29cea0d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 15:46:25.025358 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a76530ae-6d5c-4acf-bd28-2933a29cea0d None admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 15:46:25.034817 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 46/92] 10.222.0.15 () {56 vars in 1374 bytes} [Mon Aug 22 15:46:24 2022] GET /identity/v3/role_assignments?user.id=6b2199ae0fb0480aa917ada391f5ec54&scope.project.id=43171f5fd3454380b88625276d7375da&role.id=9d9848cfd22e4b089db443c350e56670 => generated 259 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 15:46:25.035692 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a76530ae-6d5c-4acf-bd28-2933a29cea0d None admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 15:46:25.059257 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cddd1a1-33ee-4f71-8dd9-d5b9cb08a05b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 15:46:25.062495 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cddd1a1-33ee-4f71-8dd9-d5b9cb08a05b None admin] RBAC: Authorization granted {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 15:46:25.202939 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a76530ae-6d5c-4acf-bd28-2933a29cea0d None admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 15:46:25.210450 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 47/93] 10.222.0.15 () {58 vars in 1282 bytes} [Mon Aug 22 15:46:24 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 450 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 22 15:46:25.216946 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cddd1a1-33ee-4f71-8dd9-d5b9cb08a05b None admin] Authenticating user token {{(pid=81767) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 15:46:25.219405 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=100 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 15:46:25.257423 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f34da79c-db02-458d-af29-9f3fd9e8460c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 15:46:25.259232 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f34da79c-db02-458d-af29-9f3fd9e8460c None admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:46:25.259606 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f34da79c-db02-458d-af29-9f3fd9e8460c None admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:46:25.259910 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f34da79c-db02-458d-af29-9f3fd9e8460c None admin] PATH_INFO: `/v3/projects/43171f5fd3454380b88625276d7375da` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:46:25.261250 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f34da79c-db02-458d-af29-9f3fd9e8460c None admin] RBAC: Authorizing `identity:get_project(project_id=43171f5fd3454380b88625276d7375da)` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 15:46:25.263299 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f34da79c-db02-458d-af29-9f3fd9e8460c None admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 15:46:25.265825 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 47/94] 10.222.0.15 () {56 vars in 1186 bytes} [Mon Aug 22 15:46:25 2022] GET /identity/v3/projects/43171f5fd3454380b88625276d7375da => generated 298 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 15:46:25.275866 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f34da79c-db02-458d-af29-9f3fd9e8460c None admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 15:46:25.299381 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG dbcounter [-] [81767] Writing DB stats keystone:SELECT=100 {{(pid=81767) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 15:46:25.363346 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cddd1a1-33ee-4f71-8dd9-d5b9cb08a05b None admin] Validating token access rules against request {{(pid=81767) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 15:46:25.403807 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f34da79c-db02-458d-af29-9f3fd9e8460c None admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 15:46:25.413364 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54d84f3d-bf97-4acb-b7ef-dcd033dd4b04 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81767) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 15:46:25.414809 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54d84f3d-bf97-4acb-b7ef-dcd033dd4b04 None admin] REQUEST_METHOD: `GET` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:46:25.415718 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54d84f3d-bf97-4acb-b7ef-dcd033dd4b04 None admin] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:46:25.416271 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54d84f3d-bf97-4acb-b7ef-dcd033dd4b04 None admin] PATH_INFO: `/v3/projects/92cc7eea42054bbdb6e4208d8f331345` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:46:25.417965 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54d84f3d-bf97-4acb-b7ef-dcd033dd4b04 None admin] RBAC: Authorizing `identity:get_project(project_id=92cc7eea42054bbdb6e4208d8f331345)` {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 15:46:25.421314 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54d84f3d-bf97-4acb-b7ef-dcd033dd4b04 None admin] RBAC: Authorization granted {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 15:46:25.423922 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 48/95] 10.222.0.15 () {56 vars in 1186 bytes} [Mon Aug 22 15:46:25 2022] GET /identity/v3/projects/92cc7eea42054bbdb6e4208d8f331345 => generated 312 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 22 15:46:25.435666 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54d84f3d-bf97-4acb-b7ef-dcd033dd4b04 None admin] Authenticating user token {{(pid=81767) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 15:46:25.466374 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d47e359a-efc0-4d0b-b10b-3ca0486a5203 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 15:46:25.467369 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d47e359a-efc0-4d0b-b10b-3ca0486a5203 None admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:46:25.468070 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d47e359a-efc0-4d0b-b10b-3ca0486a5203 None admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:46:25.469321 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d47e359a-efc0-4d0b-b10b-3ca0486a5203 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:46:25.470460 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d47e359a-efc0-4d0b-b10b-3ca0486a5203 None admin] RBAC: Adding query filter params (role.id=1ce6b924c9604048906c07dbfb126da9, scope.project.id=43171f5fd3454380b88625276d7375da, user.id=ad6aab8e70974f939bda68c2060aace5) {{(pid=81768) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 15:46:25.472155 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d47e359a-efc0-4d0b-b10b-3ca0486a5203 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 15:46:25.475057 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d47e359a-efc0-4d0b-b10b-3ca0486a5203 None admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 15:46:25.487262 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 48/96] 10.222.0.15 () {56 vars in 1374 bytes} [Mon Aug 22 15:46:25 2022] GET /identity/v3/role_assignments?user.id=ad6aab8e70974f939bda68c2060aace5&scope.project.id=43171f5fd3454380b88625276d7375da&role.id=1ce6b924c9604048906c07dbfb126da9 => generated 259 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 15:46:25.613421 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54d84f3d-bf97-4acb-b7ef-dcd033dd4b04 None admin] Validating token access rules against request {{(pid=81767) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 15:46:25.653768 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG dbcounter [-] [81767] Writing DB stats keystone:SELECT=100 {{(pid=81767) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 15:46:25.664031 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aee2c546-d56c-4aae-ace9-08d6f3c9e971 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81767) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 15:46:25.664994 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aee2c546-d56c-4aae-ace9-08d6f3c9e971 None admin] REQUEST_METHOD: `GET` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:46:25.675388 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aee2c546-d56c-4aae-ace9-08d6f3c9e971 None admin] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:46:25.675388 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aee2c546-d56c-4aae-ace9-08d6f3c9e971 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:46:25.675388 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aee2c546-d56c-4aae-ace9-08d6f3c9e971 None admin] RBAC: Adding query filter params (role.id=e6fc1ee3fb974125ba9b4fe55b12b45e, scope.project.id=92cc7eea42054bbdb6e4208d8f331345, user.id=ad6aab8e70974f939bda68c2060aace5) {{(pid=81767) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 15:46:25.675388 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aee2c546-d56c-4aae-ace9-08d6f3c9e971 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 15:46:25.675388 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aee2c546-d56c-4aae-ace9-08d6f3c9e971 None admin] RBAC: Authorization granted {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 15:46:25.682377 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 49/97] 10.222.0.15 () {56 vars in 1374 bytes} [Mon Aug 22 15:46:25 2022] GET /identity/v3/role_assignments?user.id=ad6aab8e70974f939bda68c2060aace5&scope.project.id=92cc7eea42054bbdb6e4208d8f331345&role.id=e6fc1ee3fb974125ba9b4fe55b12b45e => generated 259 bytes in 248 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 15:46:27.191714 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb2ec716-f617-4905-99cb-c0dc1419a323 None None] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:46:27.192435 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb2ec716-f617-4905-99cb-c0dc1419a323 None None] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:46:27.192531 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb2ec716-f617-4905-99cb-c0dc1419a323 None None] PATH_INFO: `/` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:46:27.194393 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 49/98] 10.222.0.15 () {54 vars in 927 bytes} [Mon Aug 22 15:46:27 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 15:46:27.204414 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a9daf51-cae8-49f2-8bab-8d6b4e7b84aa None None] REQUEST_METHOD: `POST` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:46:27.204414 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a9daf51-cae8-49f2-8bab-8d6b4e7b84aa None None] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:46:27.204414 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a9daf51-cae8-49f2-8bab-8d6b4e7b84aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:46:27.227896 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.auth.core [None req-0a9daf51-cae8-49f2-8bab-8d6b4e7b84aa None None] MFA Rules not processed for user `6b2199ae0fb0480aa917ada391f5ec54`. Rule list: `[]` (Enabled: `True`). {{(pid=81767) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 15:46:27.229343 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.fernet_utils [None req-0a9daf51-cae8-49f2-8bab-8d6b4e7b84aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81767) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 15:46:27.300861 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 50/99] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Aug 22 15:46:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 22 15:46:27.804191 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b55260d6-ea7d-4496-9b92-d0a7b049db12 None None] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:46:27.804591 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b55260d6-ea7d-4496-9b92-d0a7b049db12 None None] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:46:27.804771 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b55260d6-ea7d-4496-9b92-d0a7b049db12 None None] PATH_INFO: `/` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:46:27.806645 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 50/100] 10.222.0.15 () {54 vars in 927 bytes} [Mon Aug 22 15:46:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 15:46:27.813061 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9ee4e63-77aa-40d6-9ea9-59a1fb2f909e None None] REQUEST_METHOD: `POST` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:46:27.813354 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9ee4e63-77aa-40d6-9ea9-59a1fb2f909e None None] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:46:27.813632 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9ee4e63-77aa-40d6-9ea9-59a1fb2f909e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:46:27.840297 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.auth.core [None req-a9ee4e63-77aa-40d6-9ea9-59a1fb2f909e None None] MFA Rules not processed for user `6b2199ae0fb0480aa917ada391f5ec54`. Rule list: `[]` (Enabled: `True`). {{(pid=81767) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 15:46:27.841883 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.fernet_utils [None req-a9ee4e63-77aa-40d6-9ea9-59a1fb2f909e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81767) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 15:46:27.863104 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd9735d9-e5da-4634-9db3-cda5e4b0c367 None None] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:46:27.863531 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd9735d9-e5da-4634-9db3-cda5e4b0c367 None None] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:46:27.863881 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd9735d9-e5da-4634-9db3-cda5e4b0c367 None None] PATH_INFO: `/` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:46:27.865565 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 51/101] 10.222.0.15 () {54 vars in 927 bytes} [Mon Aug 22 15:46:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 15:46:27.871029 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea24e166-53f3-46c3-be56-5d65e012a9b0 None None] REQUEST_METHOD: `POST` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:46:27.871446 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea24e166-53f3-46c3-be56-5d65e012a9b0 None None] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:46:27.871762 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea24e166-53f3-46c3-be56-5d65e012a9b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:46:27.898841 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.auth.core [None req-ea24e166-53f3-46c3-be56-5d65e012a9b0 None None] MFA Rules not processed for user `6b2199ae0fb0480aa917ada391f5ec54`. Rule list: `[]` (Enabled: `True`). {{(pid=81768) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 15:46:27.900199 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-ea24e166-53f3-46c3-be56-5d65e012a9b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 15:46:27.951329 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 51/102] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Aug 22 15:46:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 22 15:46:27.970785 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=100 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 15:46:27.996815 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9ee4e63-77aa-40d6-9ea9-59a1fb2f909e None None] Authenticating user token {{(pid=81767) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 15:46:28.026758 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 52/103] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Aug 22 15:46:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 22 15:46:28.072498 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG dbcounter [-] [81767] Writing DB stats keystone:SELECT=100 {{(pid=81767) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 15:46:28.129712 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9ee4e63-77aa-40d6-9ea9-59a1fb2f909e None None] Validating token access rules against request {{(pid=81767) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 15:46:28.164882 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41a3110c-0f2a-44fc-bd25-25d75b484a65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81767) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 15:46:28.165539 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41a3110c-0f2a-44fc-bd25-25d75b484a65 None admin] REQUEST_METHOD: `GET` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:46:28.165820 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41a3110c-0f2a-44fc-bd25-25d75b484a65 None admin] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:46:28.166093 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41a3110c-0f2a-44fc-bd25-25d75b484a65 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:46:28.178299 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41a3110c-0f2a-44fc-bd25-25d75b484a65 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 22 15:46:28.184000 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application [None req-41a3110c-0f2a-44fc-bd25-25d75b484a65 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 22 15:46:28.184000 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 15:46:28.184000 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 15:46:28.184000 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 15:46:28.184000 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 15:46:28.184000 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 15:46:28.184000 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 15:46:28.184000 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 15:46:28.184000 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 15:46:28.184000 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 15:46:28.184000 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 15:46:28.184000 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 15:46:28.184000 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 22 15:46:28.184000 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 22 15:46:28.184000 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 22 15:46:28.184000 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 22 15:46:28.185431 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 22 15:46:28.185431 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 22 15:46:28.185431 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 22 15:46:28.185431 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 22 15:46:28.185431 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 15:46:28.185431 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 15:46:28.185431 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 15:46:28.185431 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 15:46:28.185431 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 15:46:28.185431 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 15:46:28.185431 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 15:46:28.185431 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application with Lock( Aug 22 15:46:28.185431 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 15:46:28.185431 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return self._enter() Aug 22 15:46:28.185431 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 15:46:28.185431 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 15:46:28.186202 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 15:46:28.186202 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return self.creator() Aug 22 15:46:28.186202 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 15:46:28.186202 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application created_value = creator( Aug 22 15:46:28.186202 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 22 15:46:28.186202 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 22 15:46:28.186202 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 22 15:46:28.186202 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 22 15:46:28.186202 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 22 15:46:28.186202 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 22 15:46:28.186202 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 22 15:46:28.186202 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application Aug 22 15:46:28.186202 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 52/104] 10.222.0.15 () {56 vars in 1099 bytes} [Mon Aug 22 15:46:27 2022] GET /identity/v3/roles/member => generated 84 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 22 15:46:28.191759 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea24e166-53f3-46c3-be56-5d65e012a9b0 None None] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 15:46:28.295373 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41a3110c-0f2a-44fc-bd25-25d75b484a65 None admin] Authenticating user token {{(pid=81767) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 15:46:28.368466 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea24e166-53f3-46c3-be56-5d65e012a9b0 None None] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 15:46:28.470931 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fea87b6-53d7-48a0-9b37-0cd1765e65cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 15:46:28.471584 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fea87b6-53d7-48a0-9b37-0cd1765e65cb None admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:46:28.471869 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fea87b6-53d7-48a0-9b37-0cd1765e65cb None admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:46:28.472142 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fea87b6-53d7-48a0-9b37-0cd1765e65cb None admin] PATH_INFO: `/v3/roles` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:46:28.472825 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fea87b6-53d7-48a0-9b37-0cd1765e65cb None admin] RBAC: Adding query filter params (name=member) {{(pid=81768) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 15:46:28.473156 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fea87b6-53d7-48a0-9b37-0cd1765e65cb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 15:46:28.474554 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fea87b6-53d7-48a0-9b37-0cd1765e65cb None admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 15:46:28.494690 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 53/105] 10.222.0.15 () {56 vars in 1101 bytes} [Mon Aug 22 15:46:28 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 303 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 15:46:28.495387 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fea87b6-53d7-48a0-9b37-0cd1765e65cb None admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 15:46:28.495502 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41a3110c-0f2a-44fc-bd25-25d75b484a65 None admin] Validating token access rules against request {{(pid=81767) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 15:46:28.523568 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG dbcounter [-] [81767] Writing DB stats keystone:SELECT=100 {{(pid=81767) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 15:46:28.550607 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=100 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 15:46:28.562999 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c7259fb-28b2-4d68-a8d2-3989809d7244 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81767) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 15:46:28.564160 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c7259fb-28b2-4d68-a8d2-3989809d7244 None admin] REQUEST_METHOD: `GET` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:46:28.567512 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c7259fb-28b2-4d68-a8d2-3989809d7244 None admin] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:46:28.567512 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c7259fb-28b2-4d68-a8d2-3989809d7244 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:46:28.580824 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c7259fb-28b2-4d68-a8d2-3989809d7244 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 22 15:46:28.588660 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application [None req-8c7259fb-28b2-4d68-a8d2-3989809d7244 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 22 15:46:28.588660 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 15:46:28.588660 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 15:46:28.588660 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 15:46:28.588660 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 15:46:28.588660 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 15:46:28.588660 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 15:46:28.588660 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 15:46:28.588660 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 15:46:28.588660 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 15:46:28.588660 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 15:46:28.588660 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 15:46:28.588660 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 22 15:46:28.588660 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 22 15:46:28.588660 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 22 15:46:28.588660 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 22 15:46:28.589910 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 22 15:46:28.589910 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 22 15:46:28.589910 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 22 15:46:28.589910 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 22 15:46:28.589910 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 15:46:28.589910 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 15:46:28.589910 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 15:46:28.589910 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 15:46:28.589910 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 15:46:28.589910 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 15:46:28.589910 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 15:46:28.589910 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application with Lock( Aug 22 15:46:28.589910 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 15:46:28.589910 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return self._enter() Aug 22 15:46:28.589910 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 15:46:28.589910 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 15:46:28.590768 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 15:46:28.590768 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return self.creator() Aug 22 15:46:28.590768 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 15:46:28.590768 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application created_value = creator( Aug 22 15:46:28.590768 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 22 15:46:28.590768 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 22 15:46:28.590768 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 22 15:46:28.590768 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 22 15:46:28.590768 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 22 15:46:28.590768 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 22 15:46:28.590768 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 22 15:46:28.590768 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application Aug 22 15:46:28.590768 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 53/106] 10.222.0.15 () {56 vars in 1099 bytes} [Mon Aug 22 15:46:28 2022] GET /identity/v3/roles/reader => generated 84 bytes in 295 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 22 15:46:28.591506 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c7259fb-28b2-4d68-a8d2-3989809d7244 None admin] Authenticating user token {{(pid=81767) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 15:46:28.632576 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fea87b6-53d7-48a0-9b37-0cd1765e65cb None admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 15:46:28.676330 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-875d691e-3161-42a4-aaff-979e574fc968 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 15:46:28.677191 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-875d691e-3161-42a4-aaff-979e574fc968 None admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:46:28.677580 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-875d691e-3161-42a4-aaff-979e574fc968 None admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:46:28.677985 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-875d691e-3161-42a4-aaff-979e574fc968 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:46:28.691880 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-875d691e-3161-42a4-aaff-979e574fc968 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 22 15:46:28.697740 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application [None req-875d691e-3161-42a4-aaff-979e574fc968 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 22 15:46:28.697740 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 15:46:28.697740 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 15:46:28.697740 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 15:46:28.697740 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 15:46:28.697740 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 15:46:28.697740 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 15:46:28.697740 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 15:46:28.697740 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 15:46:28.697740 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 15:46:28.697740 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 15:46:28.697740 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 15:46:28.697740 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 22 15:46:28.697740 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 22 15:46:28.697740 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 22 15:46:28.697740 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 22 15:46:28.699427 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 22 15:46:28.699427 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 22 15:46:28.699427 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 22 15:46:28.699427 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 22 15:46:28.699427 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 15:46:28.699427 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 15:46:28.699427 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 15:46:28.699427 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 15:46:28.699427 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 15:46:28.699427 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 15:46:28.699427 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 15:46:28.699427 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application with Lock( Aug 22 15:46:28.699427 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 15:46:28.699427 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self._enter() Aug 22 15:46:28.699427 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 15:46:28.699427 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 15:46:28.701081 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 15:46:28.701081 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self.creator() Aug 22 15:46:28.701081 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 15:46:28.701081 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application created_value = creator( Aug 22 15:46:28.701081 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 22 15:46:28.701081 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 22 15:46:28.701081 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 22 15:46:28.701081 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 22 15:46:28.701081 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 22 15:46:28.701081 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 22 15:46:28.701081 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 22 15:46:28.701081 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application Aug 22 15:46:28.701081 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 54/107] 10.222.0.15 () {56 vars in 1096 bytes} [Mon Aug 22 15:46:28 2022] GET /identity/v3/roles/admin => generated 83 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 15:46:28.707567 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd7f1063-1bd8-4ca2-9690-77cc6ff6076f None None] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:46:28.707567 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd7f1063-1bd8-4ca2-9690-77cc6ff6076f None None] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:46:28.707567 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd7f1063-1bd8-4ca2-9690-77cc6ff6076f None None] PATH_INFO: `/` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:46:28.707567 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 55/108] 10.222.0.15 () {54 vars in 927 bytes} [Mon Aug 22 15:46:28 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 15:46:28.707567 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd7f1063-1bd8-4ca2-9690-77cc6ff6076f None None] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 15:46:28.738185 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c7259fb-28b2-4d68-a8d2-3989809d7244 None admin] Validating token access rules against request {{(pid=81767) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 15:46:28.783491 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e1328c5-e778-4fe0-8f6f-88d2699a311e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81767) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 15:46:28.784558 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e1328c5-e778-4fe0-8f6f-88d2699a311e None admin] REQUEST_METHOD: `GET` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:46:28.784840 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e1328c5-e778-4fe0-8f6f-88d2699a311e None admin] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:46:28.785110 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e1328c5-e778-4fe0-8f6f-88d2699a311e None admin] PATH_INFO: `/v3/users/ad6aab8e70974f939bda68c2060aace5` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:46:28.786537 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e1328c5-e778-4fe0-8f6f-88d2699a311e None admin] RBAC: Authorizing `identity:get_user(user_id=ad6aab8e70974f939bda68c2060aace5)` {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 15:46:28.788920 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e1328c5-e778-4fe0-8f6f-88d2699a311e None admin] RBAC: Authorization granted {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 15:46:28.790816 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 54/109] 10.222.0.15 () {56 vars in 1177 bytes} [Mon Aug 22 15:46:28 2022] GET /identity/v3/users/ad6aab8e70974f939bda68c2060aace5 => generated 275 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 15:46:28.791714 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e1328c5-e778-4fe0-8f6f-88d2699a311e None admin] Authenticating user token {{(pid=81767) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 15:46:28.829851 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=100 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 15:46:28.844596 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd7f1063-1bd8-4ca2-9690-77cc6ff6076f None None] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 15:46:28.847514 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG dbcounter [-] [81767] Writing DB stats keystone:SELECT=100 {{(pid=81767) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 15:46:28.892759 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf7ed7bd-fc4e-4eb7-a094-d9200c6aeb6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 15:46:28.896735 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf7ed7bd-fc4e-4eb7-a094-d9200c6aeb6d None admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:46:28.896735 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf7ed7bd-fc4e-4eb7-a094-d9200c6aeb6d None admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:46:28.896735 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf7ed7bd-fc4e-4eb7-a094-d9200c6aeb6d None admin] PATH_INFO: `/v3/roles` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:46:28.896735 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf7ed7bd-fc4e-4eb7-a094-d9200c6aeb6d None admin] RBAC: Adding query filter params (name=reader) {{(pid=81768) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 15:46:28.896735 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf7ed7bd-fc4e-4eb7-a094-d9200c6aeb6d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 15:46:28.896735 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf7ed7bd-fc4e-4eb7-a094-d9200c6aeb6d None admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 15:46:28.907135 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 56/110] 10.222.0.15 () {56 vars in 1101 bytes} [Mon Aug 22 15:46:28 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 15:46:28.908766 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-117a70c1-b456-4c67-9a19-df01141ec002 None None] REQUEST_METHOD: `POST` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:46:28.909068 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-117a70c1-b456-4c67-9a19-df01141ec002 None None] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:46:28.909339 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-117a70c1-b456-4c67-9a19-df01141ec002 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:46:28.928045 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e1328c5-e778-4fe0-8f6f-88d2699a311e None admin] Validating token access rules against request {{(pid=81767) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 15:46:28.936559 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.auth.core [None req-117a70c1-b456-4c67-9a19-df01141ec002 None None] MFA Rules not processed for user `6b2199ae0fb0480aa917ada391f5ec54`. Rule list: `[]` (Enabled: `True`). {{(pid=81768) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 15:46:28.937764 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-117a70c1-b456-4c67-9a19-df01141ec002 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 15:46:28.975486 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffc58b20-53aa-47a4-bb68-4e7f0e47eee2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81767) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 15:46:28.976108 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffc58b20-53aa-47a4-bb68-4e7f0e47eee2 None admin] REQUEST_METHOD: `GET` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:46:28.976388 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffc58b20-53aa-47a4-bb68-4e7f0e47eee2 None admin] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:46:28.976731 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffc58b20-53aa-47a4-bb68-4e7f0e47eee2 None admin] PATH_INFO: `/v3/roles` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:46:28.977409 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffc58b20-53aa-47a4-bb68-4e7f0e47eee2 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81767) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 15:46:28.977710 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffc58b20-53aa-47a4-bb68-4e7f0e47eee2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 15:46:28.979896 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffc58b20-53aa-47a4-bb68-4e7f0e47eee2 None admin] RBAC: Authorization granted {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 15:46:28.989132 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 55/111] 10.222.0.15 () {56 vars in 1099 bytes} [Mon Aug 22 15:46:28 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 15:46:28.989884 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffc58b20-53aa-47a4-bb68-4e7f0e47eee2 None admin] Authenticating user token {{(pid=81767) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 15:46:29.020376 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 57/112] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Aug 22 15:46:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 22 15:46:29.030573 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8d8ebec-3825-41f0-b6bc-4e10df64c317 None None] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:46:29.031456 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8d8ebec-3825-41f0-b6bc-4e10df64c317 None None] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:46:29.032158 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8d8ebec-3825-41f0-b6bc-4e10df64c317 None None] PATH_INFO: `/` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:46:29.033753 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 58/113] 10.222.0.15 () {54 vars in 927 bytes} [Mon Aug 22 15:46:29 2022] GET /identity => generated 270 bytes in 13 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 15:46:29.039076 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8d8ebec-3825-41f0-b6bc-4e10df64c317 None None] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 15:46:29.124593 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG dbcounter [-] [81767] Writing DB stats keystone:SELECT=100 {{(pid=81767) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 15:46:29.127362 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffc58b20-53aa-47a4-bb68-4e7f0e47eee2 None admin] Validating token access rules against request {{(pid=81767) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 15:46:29.155460 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=100 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 15:46:29.166232 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-355d84b3-77e7-40dc-9d83-8c09c7b2cb5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81767) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 15:46:29.167058 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-355d84b3-77e7-40dc-9d83-8c09c7b2cb5f None admin] REQUEST_METHOD: `GET` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:46:29.167183 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-355d84b3-77e7-40dc-9d83-8c09c7b2cb5f None admin] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:46:29.167447 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-355d84b3-77e7-40dc-9d83-8c09c7b2cb5f None admin] PATH_INFO: `/v3/projects/43171f5fd3454380b88625276d7375da` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:46:29.168643 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-355d84b3-77e7-40dc-9d83-8c09c7b2cb5f None admin] RBAC: Authorizing `identity:get_project(project_id=43171f5fd3454380b88625276d7375da)` {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 15:46:29.171455 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-355d84b3-77e7-40dc-9d83-8c09c7b2cb5f None admin] RBAC: Authorization granted {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 15:46:29.173120 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 56/114] 10.222.0.15 () {56 vars in 1186 bytes} [Mon Aug 22 15:46:28 2022] GET /identity/v3/projects/43171f5fd3454380b88625276d7375da => generated 298 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 15:46:29.173624 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-355d84b3-77e7-40dc-9d83-8c09c7b2cb5f None admin] Authenticating user token {{(pid=81767) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 15:46:29.180093 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8d8ebec-3825-41f0-b6bc-4e10df64c317 None None] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 15:46:29.220668 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4d85e06-e253-476f-ba95-f84f04e02ef6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 15:46:29.221780 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d85e06-e253-476f-ba95-f84f04e02ef6 None admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:46:29.222062 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d85e06-e253-476f-ba95-f84f04e02ef6 None admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:46:29.222322 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d85e06-e253-476f-ba95-f84f04e02ef6 None admin] PATH_INFO: `/v3/users/b22a88ebd0c340cd9b0d140552d1b84e` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:46:29.239324 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4d85e06-e253-476f-ba95-f84f04e02ef6 None admin] RBAC: Authorizing `identity:get_user(user_id=b22a88ebd0c340cd9b0d140552d1b84e)` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 15:46:29.240833 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4d85e06-e253-476f-ba95-f84f04e02ef6 None admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 15:46:29.242507 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 59/115] 10.222.0.15 () {56 vars in 1177 bytes} [Mon Aug 22 15:46:29 2022] GET /identity/v3/users/b22a88ebd0c340cd9b0d140552d1b84e => generated 289 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 15:46:29.243680 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2de9ace7-69c9-4824-ac51-c3f50b04d0f0 None None] REQUEST_METHOD: `POST` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:46:29.243957 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2de9ace7-69c9-4824-ac51-c3f50b04d0f0 None None] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:46:29.244229 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2de9ace7-69c9-4824-ac51-c3f50b04d0f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:46:29.263586 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.auth.core [None req-2de9ace7-69c9-4824-ac51-c3f50b04d0f0 None None] MFA Rules not processed for user `6b2199ae0fb0480aa917ada391f5ec54`. Rule list: `[]` (Enabled: `True`). {{(pid=81768) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 15:46:29.264977 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-2de9ace7-69c9-4824-ac51-c3f50b04d0f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 15:46:29.305470 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-355d84b3-77e7-40dc-9d83-8c09c7b2cb5f None admin] Validating token access rules against request {{(pid=81767) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 15:46:29.342347 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 60/116] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Aug 22 15:46:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 22 15:46:29.343160 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2b43023-500e-4afb-9efb-dbfa2c54487e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81767) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 15:46:29.343738 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f7cd3fc-7558-41e9-ab81-e240c70292ac None None] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:46:29.344095 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f7cd3fc-7558-41e9-ab81-e240c70292ac None None] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:46:29.344193 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2b43023-500e-4afb-9efb-dbfa2c54487e None admin] REQUEST_METHOD: `GET` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:46:29.344355 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f7cd3fc-7558-41e9-ab81-e240c70292ac None None] PATH_INFO: `/` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:46:29.344443 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2b43023-500e-4afb-9efb-dbfa2c54487e None admin] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:46:29.344892 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2b43023-500e-4afb-9efb-dbfa2c54487e None admin] PATH_INFO: `/v3/users/6b2199ae0fb0480aa917ada391f5ec54` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:46:29.345568 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 61/117] 10.222.0.15 () {54 vars in 927 bytes} [Mon Aug 22 15:46:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 15:46:29.346327 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f7cd3fc-7558-41e9-ab81-e240c70292ac None None] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 15:46:29.346675 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2b43023-500e-4afb-9efb-dbfa2c54487e None admin] RBAC: Authorizing `identity:get_user(user_id=6b2199ae0fb0480aa917ada391f5ec54)` {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 15:46:29.348126 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2b43023-500e-4afb-9efb-dbfa2c54487e None admin] RBAC: Authorization granted {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 15:46:29.350213 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 57/118] 10.222.0.15 () {56 vars in 1177 bytes} [Mon Aug 22 15:46:29 2022] GET /identity/v3/users/6b2199ae0fb0480aa917ada391f5ec54 => generated 247 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 15:46:29.351721 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2b43023-500e-4afb-9efb-dbfa2c54487e None admin] Authenticating user token {{(pid=81767) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 15:46:29.401620 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG dbcounter [-] [81767] Writing DB stats keystone:SELECT=100 {{(pid=81767) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 15:46:29.436918 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=100 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 15:46:29.476009 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f7cd3fc-7558-41e9-ab81-e240c70292ac None None] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 15:46:29.480931 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2b43023-500e-4afb-9efb-dbfa2c54487e None admin] Validating token access rules against request {{(pid=81767) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 15:46:29.511496 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a406260-7faf-4ee1-bb53-caa6464f8101 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 15:46:29.512152 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a406260-7faf-4ee1-bb53-caa6464f8101 None admin] REQUEST_METHOD: `PUT` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:46:29.512429 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a406260-7faf-4ee1-bb53-caa6464f8101 None admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:46:29.512728 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a406260-7faf-4ee1-bb53-caa6464f8101 None admin] PATH_INFO: `/v3/projects/43171f5fd3454380b88625276d7375da/users/ad6aab8e70974f939bda68c2060aace5/roles/e6fc1ee3fb974125ba9b4fe55b12b45e` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:46:29.514628 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a406260-7faf-4ee1-bb53-caa6464f8101 None admin] RBAC: Authorizing `identity:create_grant(project_id=43171f5fd3454380b88625276d7375da, user_id=ad6aab8e70974f939bda68c2060aace5, role_id=e6fc1ee3fb974125ba9b4fe55b12b45e)` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 15:46:29.516107 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a406260-7faf-4ee1-bb53-caa6464f8101 None admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 15:46:29.518805 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce904bdd-3033-419d-8368-4da3d49adaa7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81767) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 15:46:29.519436 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce904bdd-3033-419d-8368-4da3d49adaa7 None admin] REQUEST_METHOD: `GET` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:46:29.519771 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce904bdd-3033-419d-8368-4da3d49adaa7 None admin] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:46:29.520155 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce904bdd-3033-419d-8368-4da3d49adaa7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:46:29.587698 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce904bdd-3033-419d-8368-4da3d49adaa7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 15:46:29.589180 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce904bdd-3033-419d-8368-4da3d49adaa7 None admin] RBAC: Authorization granted {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 15:46:29.603310 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 62/119] 10.222.0.15 () {58 vars in 1439 bytes} [Mon Aug 22 15:46:29 2022] PUT /identity/v3/projects/43171f5fd3454380b88625276d7375da/users/ad6aab8e70974f939bda68c2060aace5/roles/e6fc1ee3fb974125ba9b4fe55b12b45e => generated 0 bytes in 248 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 15:46:29.603310 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a406260-7faf-4ee1-bb53-caa6464f8101 None admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 15:46:29.637292 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG dbcounter [-] [81767] Writing DB stats keystone:SELECT=100 {{(pid=81767) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 15:46:29.713106 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 58/120] 10.222.0.15 () {58 vars in 1282 bytes} [Mon Aug 22 15:46:29 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 362 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 22 15:46:29.714465 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c365c13-07b7-48eb-95a7-6a9d8bbcc30e None None] REQUEST_METHOD: `POST` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:46:29.714760 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c365c13-07b7-48eb-95a7-6a9d8bbcc30e None None] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:46:29.715035 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c365c13-07b7-48eb-95a7-6a9d8bbcc30e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:46:29.733066 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a406260-7faf-4ee1-bb53-caa6464f8101 None admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 15:46:29.733924 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.auth.core [None req-4c365c13-07b7-48eb-95a7-6a9d8bbcc30e None None] MFA Rules not processed for user `6b2199ae0fb0480aa917ada391f5ec54`. Rule list: `[]` (Enabled: `True`). {{(pid=81767) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 15:46:29.735165 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.fernet_utils [None req-4c365c13-07b7-48eb-95a7-6a9d8bbcc30e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81767) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 15:46:29.771020 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 15:46:29.776615 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02a7cccf-8153-4548-a1e3-a19547bdf71e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 15:46:29.777260 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02a7cccf-8153-4548-a1e3-a19547bdf71e None admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:46:29.777540 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02a7cccf-8153-4548-a1e3-a19547bdf71e None admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:46:29.777817 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02a7cccf-8153-4548-a1e3-a19547bdf71e None admin] PATH_INFO: `/v3/domains/default` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:46:29.779916 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02a7cccf-8153-4548-a1e3-a19547bdf71e None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 15:46:29.787491 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02a7cccf-8153-4548-a1e3-a19547bdf71e None admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 15:46:29.787491 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 63/121] 10.222.0.15 () {56 vars in 1108 bytes} [Mon Aug 22 15:46:29 2022] GET /identity/v3/domains/default => generated 199 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 15:46:29.787491 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02a7cccf-8153-4548-a1e3-a19547bdf71e None admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 15:46:29.817373 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 59/122] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Aug 22 15:46:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 22 15:46:29.818044 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c365c13-07b7-48eb-95a7-6a9d8bbcc30e None None] Authenticating user token {{(pid=81767) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 15:46:29.896924 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02a7cccf-8153-4548-a1e3-a19547bdf71e None admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 15:46:29.900824 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG dbcounter [-] [81767] Writing DB stats keystone:SELECT=100 {{(pid=81767) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 15:46:29.937887 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31fa19ca-cdd2-4424-a8b7-e0cd3512962d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 15:46:29.938532 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31fa19ca-cdd2-4424-a8b7-e0cd3512962d None admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:46:29.938815 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31fa19ca-cdd2-4424-a8b7-e0cd3512962d None admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:46:29.939089 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31fa19ca-cdd2-4424-a8b7-e0cd3512962d None admin] PATH_INFO: `/v3/projects/43171f5fd3454380b88625276d7375da` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:46:29.940332 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31fa19ca-cdd2-4424-a8b7-e0cd3512962d None admin] RBAC: Authorizing `identity:get_project(project_id=43171f5fd3454380b88625276d7375da)` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 15:46:29.941825 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31fa19ca-cdd2-4424-a8b7-e0cd3512962d None admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 15:46:29.944358 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 64/123] 10.222.0.15 () {56 vars in 1186 bytes} [Mon Aug 22 15:46:29 2022] GET /identity/v3/projects/43171f5fd3454380b88625276d7375da => generated 298 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 15:46:29.945334 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31fa19ca-cdd2-4424-a8b7-e0cd3512962d None admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 15:46:29.948676 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c365c13-07b7-48eb-95a7-6a9d8bbcc30e None None] Validating token access rules against request {{(pid=81767) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 15:46:29.998801 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e78592e8-fa09-46dd-bdb3-e349df0a8420 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81767) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 15:46:29.999834 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e78592e8-fa09-46dd-bdb3-e349df0a8420 None admin] REQUEST_METHOD: `GET` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:46:30.000423 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e78592e8-fa09-46dd-bdb3-e349df0a8420 None admin] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:46:30.000703 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e78592e8-fa09-46dd-bdb3-e349df0a8420 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:46:30.015682 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e78592e8-fa09-46dd-bdb3-e349df0a8420 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 22 15:46:30.022736 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application [None req-e78592e8-fa09-46dd-bdb3-e349df0a8420 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 22 15:46:30.022736 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 15:46:30.022736 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 15:46:30.022736 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 15:46:30.022736 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 15:46:30.022736 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 15:46:30.022736 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 15:46:30.022736 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 15:46:30.022736 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 15:46:30.022736 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 15:46:30.022736 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 15:46:30.022736 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 15:46:30.022736 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 22 15:46:30.022736 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 22 15:46:30.022736 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 22 15:46:30.022736 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 22 15:46:30.023620 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 22 15:46:30.023620 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 22 15:46:30.023620 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 22 15:46:30.023620 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 22 15:46:30.023620 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 15:46:30.023620 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 15:46:30.023620 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 15:46:30.023620 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 15:46:30.023620 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 15:46:30.023620 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 15:46:30.023620 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 15:46:30.023620 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application with Lock( Aug 22 15:46:30.023620 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 15:46:30.023620 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return self._enter() Aug 22 15:46:30.023620 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 15:46:30.023620 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 15:46:30.024351 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 15:46:30.024351 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return self.creator() Aug 22 15:46:30.024351 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 15:46:30.024351 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application created_value = creator( Aug 22 15:46:30.024351 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 22 15:46:30.024351 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 22 15:46:30.024351 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 22 15:46:30.024351 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 22 15:46:30.024351 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 22 15:46:30.024351 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 22 15:46:30.024351 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 22 15:46:30.024351 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application Aug 22 15:46:30.024927 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=100 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 15:46:30.027467 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 60/124] 10.222.0.15 () {56 vars in 1099 bytes} [Mon Aug 22 15:46:29 2022] GET /identity/v3/roles/member => generated 84 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 15:46:30.027467 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e78592e8-fa09-46dd-bdb3-e349df0a8420 None admin] Authenticating user token {{(pid=81767) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 15:46:30.063859 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31fa19ca-cdd2-4424-a8b7-e0cd3512962d None admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 15:46:30.104530 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-898a5df5-9e7c-498c-85b7-98a6471b30d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 15:46:30.105301 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-898a5df5-9e7c-498c-85b7-98a6471b30d1 None admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:46:30.105652 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-898a5df5-9e7c-498c-85b7-98a6471b30d1 None admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:46:30.105996 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-898a5df5-9e7c-498c-85b7-98a6471b30d1 None admin] PATH_INFO: `/v3/projects/43171f5fd3454380b88625276d7375da` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:46:30.107829 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-898a5df5-9e7c-498c-85b7-98a6471b30d1 None admin] RBAC: Authorizing `identity:get_project(project_id=43171f5fd3454380b88625276d7375da)` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 15:46:30.109537 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-898a5df5-9e7c-498c-85b7-98a6471b30d1 None admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 15:46:30.111378 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 65/125] 10.222.0.15 () {56 vars in 1186 bytes} [Mon Aug 22 15:46:29 2022] GET /identity/v3/projects/43171f5fd3454380b88625276d7375da => generated 298 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 15:46:30.112075 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-898a5df5-9e7c-498c-85b7-98a6471b30d1 None admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 15:46:30.158918 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e78592e8-fa09-46dd-bdb3-e349df0a8420 None admin] Validating token access rules against request {{(pid=81767) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 15:46:30.187333 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG dbcounter [-] [81767] Writing DB stats keystone:SELECT=100 {{(pid=81767) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 15:46:30.197636 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bb2b770-f140-448f-8a98-82f81d78f7a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81767) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 15:46:30.198118 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb2b770-f140-448f-8a98-82f81d78f7a1 None admin] REQUEST_METHOD: `POST` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:46:30.198507 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb2b770-f140-448f-8a98-82f81d78f7a1 None admin] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:46:30.199224 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb2b770-f140-448f-8a98-82f81d78f7a1 None admin] PATH_INFO: `/v3/projects` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:46:30.199658 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bb2b770-f140-448f-8a98-82f81d78f7a1 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 15:46:30.205388 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bb2b770-f140-448f-8a98-82f81d78f7a1 None admin] RBAC: Authorization granted {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 15:46:30.224084 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-898a5df5-9e7c-498c-85b7-98a6471b30d1 None admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 15:46:30.259321 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f88940a1-d853-4f7b-adaf-be149b55d617 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 15:46:30.259899 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f88940a1-d853-4f7b-adaf-be149b55d617 None admin] REQUEST_METHOD: `PUT` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:46:30.260164 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f88940a1-d853-4f7b-adaf-be149b55d617 None admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:46:30.260437 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f88940a1-d853-4f7b-adaf-be149b55d617 None admin] PATH_INFO: `/v3/projects/43171f5fd3454380b88625276d7375da/users/6b2199ae0fb0480aa917ada391f5ec54/roles/9d9848cfd22e4b089db443c350e56670` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:46:30.262371 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f88940a1-d853-4f7b-adaf-be149b55d617 None admin] RBAC: Authorizing `identity:create_grant(project_id=43171f5fd3454380b88625276d7375da, user_id=6b2199ae0fb0480aa917ada391f5ec54, role_id=9d9848cfd22e4b089db443c350e56670)` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 15:46:30.263919 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f88940a1-d853-4f7b-adaf-be149b55d617 None admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 15:46:30.266955 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 15:46:30.304111 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 61/126] 10.222.0.15 () {60 vars in 1141 bytes} [Mon Aug 22 15:46:30 2022] POST /identity/v3/projects => generated 302 bytes in 278 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 22 15:46:30.305219 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bb2b770-f140-448f-8a98-82f81d78f7a1 None admin] Authenticating user token {{(pid=81767) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 15:46:30.430298 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 66/127] 10.222.0.15 () {58 vars in 1439 bytes} [Mon Aug 22 15:46:30 2022] PUT /identity/v3/projects/43171f5fd3454380b88625276d7375da/users/6b2199ae0fb0480aa917ada391f5ec54/roles/9d9848cfd22e4b089db443c350e56670 => generated 0 bytes in 318 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 15:46:30.431224 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f88940a1-d853-4f7b-adaf-be149b55d617 None admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 15:46:30.434233 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bb2b770-f140-448f-8a98-82f81d78f7a1 None admin] Validating token access rules against request {{(pid=81767) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 15:46:30.497753 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c47bc53-4587-464e-9b8f-34f22ad15e20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81767) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 15:46:30.498780 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c47bc53-4587-464e-9b8f-34f22ad15e20 None admin] REQUEST_METHOD: `GET` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:46:30.499306 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c47bc53-4587-464e-9b8f-34f22ad15e20 None admin] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:46:30.499769 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c47bc53-4587-464e-9b8f-34f22ad15e20 None admin] PATH_INFO: `/v3/roles` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:46:30.500691 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c47bc53-4587-464e-9b8f-34f22ad15e20 None admin] RBAC: Adding query filter params (name=member) {{(pid=81767) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 15:46:30.501962 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c47bc53-4587-464e-9b8f-34f22ad15e20 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 15:46:30.504409 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c47bc53-4587-464e-9b8f-34f22ad15e20 None admin] RBAC: Authorization granted {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 15:46:30.517868 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 62/128] 10.222.0.15 () {56 vars in 1101 bytes} [Mon Aug 22 15:46:30 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 15:46:30.519070 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c47bc53-4587-464e-9b8f-34f22ad15e20 None admin] Authenticating user token {{(pid=81767) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 15:46:30.572287 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG dbcounter [-] [81767] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81767) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 15:46:30.590871 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f88940a1-d853-4f7b-adaf-be149b55d617 None admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 15:46:30.631196 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73717d06-10c8-4a30-a998-dc5733b6bb0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 15:46:30.631871 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73717d06-10c8-4a30-a998-dc5733b6bb0d None admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:46:30.632153 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73717d06-10c8-4a30-a998-dc5733b6bb0d None admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:46:30.632436 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73717d06-10c8-4a30-a998-dc5733b6bb0d None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:46:30.642147 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c47bc53-4587-464e-9b8f-34f22ad15e20 None admin] Validating token access rules against request {{(pid=81767) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 15:46:30.644859 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73717d06-10c8-4a30-a998-dc5733b6bb0d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 22 15:46:30.650212 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application [None req-73717d06-10c8-4a30-a998-dc5733b6bb0d None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 22 15:46:30.650212 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 15:46:30.650212 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 15:46:30.650212 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 15:46:30.650212 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 15:46:30.650212 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 15:46:30.650212 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 15:46:30.650212 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 15:46:30.650212 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 15:46:30.650212 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 15:46:30.650212 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 15:46:30.650212 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 15:46:30.650212 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 22 15:46:30.650212 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 22 15:46:30.650212 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 22 15:46:30.650212 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 22 15:46:30.651087 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 22 15:46:30.651087 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 22 15:46:30.651087 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 22 15:46:30.651087 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 22 15:46:30.651087 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 15:46:30.651087 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 15:46:30.651087 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 15:46:30.651087 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 15:46:30.651087 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 15:46:30.651087 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 15:46:30.651087 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 15:46:30.651087 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application with Lock( Aug 22 15:46:30.651087 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 15:46:30.651087 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self._enter() Aug 22 15:46:30.651087 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 15:46:30.651087 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 15:46:30.651853 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 15:46:30.651853 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self.creator() Aug 22 15:46:30.651853 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 15:46:30.651853 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application created_value = creator( Aug 22 15:46:30.651853 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 22 15:46:30.651853 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 22 15:46:30.651853 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 22 15:46:30.651853 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 22 15:46:30.651853 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 22 15:46:30.651853 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 22 15:46:30.651853 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 22 15:46:30.651853 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application Aug 22 15:46:30.651853 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 67/129] 10.222.0.15 () {56 vars in 1114 bytes} [Mon Aug 22 15:46:30 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 15:46:30.652463 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73717d06-10c8-4a30-a998-dc5733b6bb0d None admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 15:46:30.689991 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3bc4098-fe51-4154-96b2-6039b6e081c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81767) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 15:46:30.691539 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3bc4098-fe51-4154-96b2-6039b6e081c3 None admin] REQUEST_METHOD: `GET` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:46:30.692030 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3bc4098-fe51-4154-96b2-6039b6e081c3 None admin] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:46:30.692555 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3bc4098-fe51-4154-96b2-6039b6e081c3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:46:30.693488 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3bc4098-fe51-4154-96b2-6039b6e081c3 None admin] RBAC: Adding query filter params (role.id=640209879fb24ac0ae3f65dd107697c3, scope.project.id=43171f5fd3454380b88625276d7375da, user.id=b22a88ebd0c340cd9b0d140552d1b84e) {{(pid=81767) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 15:46:30.694073 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3bc4098-fe51-4154-96b2-6039b6e081c3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 15:46:30.695979 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3bc4098-fe51-4154-96b2-6039b6e081c3 None admin] RBAC: Authorization granted {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 15:46:30.702704 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 63/130] 10.222.0.15 () {56 vars in 1374 bytes} [Mon Aug 22 15:46:30 2022] GET /identity/v3/role_assignments?user.id=b22a88ebd0c340cd9b0d140552d1b84e&scope.project.id=43171f5fd3454380b88625276d7375da&role.id=640209879fb24ac0ae3f65dd107697c3 => generated 259 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 15:46:30.703884 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3bc4098-fe51-4154-96b2-6039b6e081c3 None admin] Authenticating user token {{(pid=81767) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 15:46:30.755653 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=100 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 15:46:30.845277 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73717d06-10c8-4a30-a998-dc5733b6bb0d None admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 15:46:30.896215 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3bc4098-fe51-4154-96b2-6039b6e081c3 None admin] Validating token access rules against request {{(pid=81767) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 15:46:30.899621 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG dbcounter [-] [81767] Writing DB stats keystone:SELECT=100 {{(pid=81767) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 15:46:30.933637 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0084e65-f8b4-4526-98d1-d9b05e891e3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81767) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 15:46:30.934282 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0084e65-f8b4-4526-98d1-d9b05e891e3f None admin] REQUEST_METHOD: `GET` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:46:30.934562 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0084e65-f8b4-4526-98d1-d9b05e891e3f None admin] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:46:30.934894 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0084e65-f8b4-4526-98d1-d9b05e891e3f None admin] PATH_INFO: `/v3/roles` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:46:30.935604 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0084e65-f8b4-4526-98d1-d9b05e891e3f None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81767) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 15:46:30.935963 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0084e65-f8b4-4526-98d1-d9b05e891e3f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 15:46:30.937426 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0084e65-f8b4-4526-98d1-d9b05e891e3f None admin] RBAC: Authorization granted {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 15:46:30.946730 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 64/131] 10.222.0.15 () {56 vars in 1111 bytes} [Mon Aug 22 15:46:30 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 243 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 15:46:30.952911 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0084e65-f8b4-4526-98d1-d9b05e891e3f None admin] Authenticating user token {{(pid=81767) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 15:46:30.956077 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4e0993b-8bff-4627-9bf9-2efa1e8d97fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 15:46:30.956823 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4e0993b-8bff-4627-9bf9-2efa1e8d97fb None admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:46:30.957112 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4e0993b-8bff-4627-9bf9-2efa1e8d97fb None admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:46:30.957395 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4e0993b-8bff-4627-9bf9-2efa1e8d97fb None admin] PATH_INFO: `/v3/users/ad6aab8e70974f939bda68c2060aace5` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:46:30.958768 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4e0993b-8bff-4627-9bf9-2efa1e8d97fb None admin] RBAC: Authorizing `identity:get_user(user_id=ad6aab8e70974f939bda68c2060aace5)` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 15:46:30.960365 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4e0993b-8bff-4627-9bf9-2efa1e8d97fb None admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 15:46:30.962034 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 68/132] 10.222.0.15 () {56 vars in 1177 bytes} [Mon Aug 22 15:46:30 2022] GET /identity/v3/users/ad6aab8e70974f939bda68c2060aace5 => generated 275 bytes in 310 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 15:46:30.968442 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4e0993b-8bff-4627-9bf9-2efa1e8d97fb None admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 15:46:31.097631 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4e0993b-8bff-4627-9bf9-2efa1e8d97fb None admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 15:46:31.123727 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=100 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 15:46:31.136560 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c477121e-e190-489b-b89e-d234365eb7b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 15:46:31.137211 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c477121e-e190-489b-b89e-d234365eb7b0 None admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:46:31.137521 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c477121e-e190-489b-b89e-d234365eb7b0 None admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:46:31.137797 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c477121e-e190-489b-b89e-d234365eb7b0 None admin] PATH_INFO: `/v3/projects/92cc7eea42054bbdb6e4208d8f331345` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:46:31.139246 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c477121e-e190-489b-b89e-d234365eb7b0 None admin] RBAC: Authorizing `identity:get_project(project_id=92cc7eea42054bbdb6e4208d8f331345)` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 15:46:31.140745 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c477121e-e190-489b-b89e-d234365eb7b0 None admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 15:46:31.145768 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 69/133] 10.222.0.15 () {56 vars in 1186 bytes} [Mon Aug 22 15:46:30 2022] GET /identity/v3/projects/92cc7eea42054bbdb6e4208d8f331345 => generated 312 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 22 15:46:31.152342 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c477121e-e190-489b-b89e-d234365eb7b0 None admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 15:46:31.162034 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0084e65-f8b4-4526-98d1-d9b05e891e3f None admin] Validating token access rules against request {{(pid=81767) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 15:46:31.235166 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfe47e25-ba3c-4dc6-8bb2-bd9236e63d17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81767) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 15:46:31.235844 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfe47e25-ba3c-4dc6-8bb2-bd9236e63d17 None admin] REQUEST_METHOD: `GET` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:46:31.236127 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfe47e25-ba3c-4dc6-8bb2-bd9236e63d17 None admin] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:46:31.236399 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfe47e25-ba3c-4dc6-8bb2-bd9236e63d17 None admin] PATH_INFO: `/v3/users/ad6aab8e70974f939bda68c2060aace5` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:46:31.237553 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfe47e25-ba3c-4dc6-8bb2-bd9236e63d17 None admin] RBAC: Authorizing `identity:get_user(user_id=ad6aab8e70974f939bda68c2060aace5)` {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 15:46:31.238997 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfe47e25-ba3c-4dc6-8bb2-bd9236e63d17 None admin] RBAC: Authorization granted {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 15:46:31.245433 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 65/134] 10.222.0.15 () {56 vars in 1177 bytes} [Mon Aug 22 15:46:30 2022] GET /identity/v3/users/ad6aab8e70974f939bda68c2060aace5 => generated 275 bytes in 293 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 22 15:46:31.251907 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfe47e25-ba3c-4dc6-8bb2-bd9236e63d17 None admin] Authenticating user token {{(pid=81767) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 15:46:31.294093 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG dbcounter [-] [81767] Writing DB stats keystone:SELECT=100 {{(pid=81767) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 15:46:31.372288 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfe47e25-ba3c-4dc6-8bb2-bd9236e63d17 None admin] Validating token access rules against request {{(pid=81767) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 15:46:31.411674 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c477121e-e190-489b-b89e-d234365eb7b0 None admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 15:46:31.412300 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e7c15da-ebb1-4891-b6b3-efb4ec6bedcb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81767) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 15:46:31.412911 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e7c15da-ebb1-4891-b6b3-efb4ec6bedcb None admin] REQUEST_METHOD: `GET` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:46:31.413192 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e7c15da-ebb1-4891-b6b3-efb4ec6bedcb None admin] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:46:31.413647 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e7c15da-ebb1-4891-b6b3-efb4ec6bedcb None admin] PATH_INFO: `/v3/projects/43171f5fd3454380b88625276d7375da` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:46:31.417644 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e7c15da-ebb1-4891-b6b3-efb4ec6bedcb None admin] RBAC: Authorizing `identity:get_project(project_id=43171f5fd3454380b88625276d7375da)` {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 15:46:31.422347 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e7c15da-ebb1-4891-b6b3-efb4ec6bedcb None admin] RBAC: Authorization granted {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 15:46:31.424075 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 66/135] 10.222.0.15 () {56 vars in 1186 bytes} [Mon Aug 22 15:46:31 2022] GET /identity/v3/projects/43171f5fd3454380b88625276d7375da => generated 298 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 15:46:31.430740 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e7c15da-ebb1-4891-b6b3-efb4ec6bedcb None admin] Authenticating user token {{(pid=81767) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 15:46:31.484101 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e573607c-1003-47eb-9a5a-0501f8a012e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 15:46:31.484770 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e573607c-1003-47eb-9a5a-0501f8a012e5 None admin] REQUEST_METHOD: `PUT` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:46:31.485047 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e573607c-1003-47eb-9a5a-0501f8a012e5 None admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:46:31.485317 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e573607c-1003-47eb-9a5a-0501f8a012e5 None admin] PATH_INFO: `/v3/projects/92cc7eea42054bbdb6e4208d8f331345/users/ad6aab8e70974f939bda68c2060aace5/roles/e6fc1ee3fb974125ba9b4fe55b12b45e` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:46:31.491963 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e573607c-1003-47eb-9a5a-0501f8a012e5 None admin] RBAC: Authorizing `identity:create_grant(project_id=92cc7eea42054bbdb6e4208d8f331345, user_id=ad6aab8e70974f939bda68c2060aace5, role_id=e6fc1ee3fb974125ba9b4fe55b12b45e)` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 15:46:31.494705 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e573607c-1003-47eb-9a5a-0501f8a012e5 None admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 15:46:31.571842 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e7c15da-ebb1-4891-b6b3-efb4ec6bedcb None admin] Validating token access rules against request {{(pid=81767) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 15:46:31.573446 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG dbcounter [-] [81767] Writing DB stats keystone:SELECT=100 {{(pid=81767) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 15:46:31.600436 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 70/136] 10.222.0.15 () {58 vars in 1439 bytes} [Mon Aug 22 15:46:31 2022] PUT /identity/v3/projects/92cc7eea42054bbdb6e4208d8f331345/users/ad6aab8e70974f939bda68c2060aace5/roles/e6fc1ee3fb974125ba9b4fe55b12b45e => generated 0 bytes in 448 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 15:46:31.614859 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eed15bb6-3af9-4619-839f-9c0f3416a0ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81767) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 15:46:31.615571 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eed15bb6-3af9-4619-839f-9c0f3416a0ca None admin] REQUEST_METHOD: `PUT` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:46:31.615922 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eed15bb6-3af9-4619-839f-9c0f3416a0ca None admin] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:46:31.616195 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eed15bb6-3af9-4619-839f-9c0f3416a0ca None admin] PATH_INFO: `/v3/projects/43171f5fd3454380b88625276d7375da/users/ad6aab8e70974f939bda68c2060aace5/roles/1ce6b924c9604048906c07dbfb126da9` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:46:31.618528 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eed15bb6-3af9-4619-839f-9c0f3416a0ca None admin] RBAC: Authorizing `identity:create_grant(project_id=43171f5fd3454380b88625276d7375da, user_id=ad6aab8e70974f939bda68c2060aace5, role_id=1ce6b924c9604048906c07dbfb126da9)` {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 15:46:31.620150 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eed15bb6-3af9-4619-839f-9c0f3416a0ca None admin] RBAC: Authorization granted {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 15:46:31.680138 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d6f35c6-27e6-4f97-93fd-011532fcc462 None None] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:46:31.680415 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d6f35c6-27e6-4f97-93fd-011532fcc462 None None] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:46:31.680651 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d6f35c6-27e6-4f97-93fd-011532fcc462 None None] PATH_INFO: `/` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:46:31.682379 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 71/137] 10.222.0.15 () {54 vars in 927 bytes} [Mon Aug 22 15:46:31 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 22 15:46:31.688389 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-322757ab-c89d-49c8-ac6a-6d317c8cce61 None None] REQUEST_METHOD: `POST` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:46:31.688688 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-322757ab-c89d-49c8-ac6a-6d317c8cce61 None None] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:46:31.688986 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-322757ab-c89d-49c8-ac6a-6d317c8cce61 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:46:31.714410 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.auth.core [None req-322757ab-c89d-49c8-ac6a-6d317c8cce61 None None] MFA Rules not processed for user `6b2199ae0fb0480aa917ada391f5ec54`. Rule list: `[]` (Enabled: `True`). {{(pid=81768) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 15:46:31.716200 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-322757ab-c89d-49c8-ac6a-6d317c8cce61 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 15:46:31.760716 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 67/138] 10.222.0.15 () {58 vars in 1439 bytes} [Mon Aug 22 15:46:31 2022] PUT /identity/v3/projects/43171f5fd3454380b88625276d7375da/users/ad6aab8e70974f939bda68c2060aace5/roles/1ce6b924c9604048906c07dbfb126da9 => generated 0 bytes in 330 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 15:46:31.798871 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 15:46:31.826827 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 72/139] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Aug 22 15:46:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 22 15:46:32.261246 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eed15bb6-3af9-4619-839f-9c0f3416a0ca None admin] Authenticating user token {{(pid=81767) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 15:46:32.367739 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eed15bb6-3af9-4619-839f-9c0f3416a0ca None admin] Validating token access rules against request {{(pid=81767) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 15:46:32.406939 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b9fb8ef-9506-4462-ad49-eb0907f800c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81767) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 15:46:32.407607 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b9fb8ef-9506-4462-ad49-eb0907f800c4 None admin] REQUEST_METHOD: `GET` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:46:32.407885 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b9fb8ef-9506-4462-ad49-eb0907f800c4 None admin] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:46:32.408240 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b9fb8ef-9506-4462-ad49-eb0907f800c4 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:46:32.419020 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b9fb8ef-9506-4462-ad49-eb0907f800c4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 22 15:46:32.423867 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application [None req-4b9fb8ef-9506-4462-ad49-eb0907f800c4 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 22 15:46:32.423867 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 15:46:32.423867 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 15:46:32.423867 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 15:46:32.423867 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 15:46:32.423867 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 15:46:32.423867 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 15:46:32.423867 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 15:46:32.423867 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 15:46:32.423867 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 15:46:32.423867 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 15:46:32.423867 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 15:46:32.423867 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 22 15:46:32.423867 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 22 15:46:32.423867 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 22 15:46:32.423867 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 22 15:46:32.424963 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 22 15:46:32.424963 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 22 15:46:32.424963 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 22 15:46:32.424963 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 22 15:46:32.424963 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 15:46:32.424963 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 15:46:32.424963 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 15:46:32.424963 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 15:46:32.424963 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 15:46:32.424963 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 15:46:32.424963 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 15:46:32.424963 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application with Lock( Aug 22 15:46:32.424963 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 15:46:32.424963 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return self._enter() Aug 22 15:46:32.424963 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 15:46:32.424963 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 15:46:32.425832 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 15:46:32.425832 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return self.creator() Aug 22 15:46:32.425832 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 15:46:32.425832 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application created_value = creator( Aug 22 15:46:32.425832 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 22 15:46:32.425832 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 22 15:46:32.425832 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 22 15:46:32.425832 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 22 15:46:32.425832 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 22 15:46:32.425832 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 22 15:46:32.425832 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 22 15:46:32.425832 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application Aug 22 15:46:32.425832 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 68/140] 10.222.0.15 () {56 vars in 1099 bytes} [Mon Aug 22 15:46:32 2022] GET /identity/v3/roles/member => generated 84 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 15:46:32.431239 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-322757ab-c89d-49c8-ac6a-6d317c8cce61 None None] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 15:46:32.550248 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-322757ab-c89d-49c8-ac6a-6d317c8cce61 None None] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 15:46:32.588043 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7528f9cb-7cd7-4a5c-8090-fb6a4953c58c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 15:46:32.588668 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7528f9cb-7cd7-4a5c-8090-fb6a4953c58c None admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:46:32.588941 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7528f9cb-7cd7-4a5c-8090-fb6a4953c58c None admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:46:32.589212 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7528f9cb-7cd7-4a5c-8090-fb6a4953c58c None admin] PATH_INFO: `/v3/roles` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:46:32.589878 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7528f9cb-7cd7-4a5c-8090-fb6a4953c58c None admin] RBAC: Adding query filter params (name=member) {{(pid=81768) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 15:46:32.590172 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7528f9cb-7cd7-4a5c-8090-fb6a4953c58c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 15:46:32.592123 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7528f9cb-7cd7-4a5c-8090-fb6a4953c58c None admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 15:46:32.600934 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 73/141] 10.222.0.15 () {56 vars in 1101 bytes} [Mon Aug 22 15:46:32 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 15:46:32.608679 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b9fb8ef-9506-4462-ad49-eb0907f800c4 None admin] Authenticating user token {{(pid=81767) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 15:46:32.636602 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG dbcounter [-] [81767] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81767) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 15:46:32.714978 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b9fb8ef-9506-4462-ad49-eb0907f800c4 None admin] Validating token access rules against request {{(pid=81767) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 15:46:32.753817 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d6f3e81-ccfd-4ff4-985e-dc08e48cef6e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81767) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 15:46:32.754518 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d6f3e81-ccfd-4ff4-985e-dc08e48cef6e None admin] REQUEST_METHOD: `GET` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:46:32.754796 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d6f3e81-ccfd-4ff4-985e-dc08e48cef6e None admin] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:46:32.755062 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d6f3e81-ccfd-4ff4-985e-dc08e48cef6e None admin] PATH_INFO: `/v3/users/ad6aab8e70974f939bda68c2060aace5` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:46:32.756281 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d6f3e81-ccfd-4ff4-985e-dc08e48cef6e None admin] RBAC: Authorizing `identity:get_user(user_id=ad6aab8e70974f939bda68c2060aace5)` {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 15:46:32.757722 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d6f3e81-ccfd-4ff4-985e-dc08e48cef6e None admin] RBAC: Authorization granted {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 15:46:32.759408 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 69/142] 10.222.0.15 () {56 vars in 1177 bytes} [Mon Aug 22 15:46:32 2022] GET /identity/v3/users/ad6aab8e70974f939bda68c2060aace5 => generated 275 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 15:46:32.766042 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7528f9cb-7cd7-4a5c-8090-fb6a4953c58c None admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 15:46:32.831786 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=100 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 15:46:32.876574 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7528f9cb-7cd7-4a5c-8090-fb6a4953c58c None admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 15:46:32.920438 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8118a004-2a02-4549-90a7-4ccdd7326b71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 15:46:32.921051 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8118a004-2a02-4549-90a7-4ccdd7326b71 None admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:46:32.921326 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8118a004-2a02-4549-90a7-4ccdd7326b71 None admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:46:32.921653 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8118a004-2a02-4549-90a7-4ccdd7326b71 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:46:32.993586 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8118a004-2a02-4549-90a7-4ccdd7326b71 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 15:46:32.995126 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8118a004-2a02-4549-90a7-4ccdd7326b71 None admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 15:46:33.041420 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=100 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 15:46:33.098395 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 74/143] 10.222.0.15 () {58 vars in 1282 bytes} [Mon Aug 22 15:46:32 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 333 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 22 15:46:33.103953 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d6f3e81-ccfd-4ff4-985e-dc08e48cef6e None admin] Authenticating user token {{(pid=81767) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 15:46:33.202391 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG dbcounter [-] [81767] Writing DB stats keystone:SELECT=100 {{(pid=81767) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 15:46:33.211959 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d6f3e81-ccfd-4ff4-985e-dc08e48cef6e None admin] Validating token access rules against request {{(pid=81767) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 15:46:33.248421 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cae41d1-b6c9-4f20-a63c-8b449fbfbb3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81767) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 15:46:33.249054 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cae41d1-b6c9-4f20-a63c-8b449fbfbb3d None admin] REQUEST_METHOD: `GET` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:46:33.249331 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cae41d1-b6c9-4f20-a63c-8b449fbfbb3d None admin] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:46:33.249604 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cae41d1-b6c9-4f20-a63c-8b449fbfbb3d None admin] PATH_INFO: `/v3/projects/43171f5fd3454380b88625276d7375da` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:46:33.250733 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cae41d1-b6c9-4f20-a63c-8b449fbfbb3d None admin] RBAC: Authorizing `identity:get_project(project_id=43171f5fd3454380b88625276d7375da)` {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 15:46:33.252206 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cae41d1-b6c9-4f20-a63c-8b449fbfbb3d None admin] RBAC: Authorization granted {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 15:46:33.254006 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 70/144] 10.222.0.15 () {56 vars in 1186 bytes} [Mon Aug 22 15:46:33 2022] GET /identity/v3/projects/43171f5fd3454380b88625276d7375da => generated 298 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 15:46:33.260170 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8118a004-2a02-4549-90a7-4ccdd7326b71 None admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 15:46:33.368132 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8118a004-2a02-4549-90a7-4ccdd7326b71 None admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 15:46:33.405016 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c65bf2c-0183-42e9-b9db-7e0ca3c4c913 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 15:46:33.405622 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c65bf2c-0183-42e9-b9db-7e0ca3c4c913 None admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:46:33.405893 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c65bf2c-0183-42e9-b9db-7e0ca3c4c913 None admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:46:33.406157 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c65bf2c-0183-42e9-b9db-7e0ca3c4c913 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:46:33.406875 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c65bf2c-0183-42e9-b9db-7e0ca3c4c913 None admin] RBAC: Adding query filter params (role.id=e6fc1ee3fb974125ba9b4fe55b12b45e, scope.project.id=43171f5fd3454380b88625276d7375da, user.id=ad6aab8e70974f939bda68c2060aace5) {{(pid=81768) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 15:46:33.407174 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c65bf2c-0183-42e9-b9db-7e0ca3c4c913 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 15:46:33.408649 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c65bf2c-0183-42e9-b9db-7e0ca3c4c913 None admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 15:46:33.414923 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 75/145] 10.222.0.15 () {56 vars in 1374 bytes} [Mon Aug 22 15:46:33 2022] GET /identity/v3/role_assignments?user.id=ad6aab8e70974f939bda68c2060aace5&scope.project.id=43171f5fd3454380b88625276d7375da&role.id=e6fc1ee3fb974125ba9b4fe55b12b45e => generated 608 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 15:46:33.428507 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50d105eb-9d01-45b6-8a3d-0eb03eb9a741 None None] REQUEST_METHOD: `GET` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:46:33.428828 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50d105eb-9d01-45b6-8a3d-0eb03eb9a741 None None] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:46:33.429136 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50d105eb-9d01-45b6-8a3d-0eb03eb9a741 None None] PATH_INFO: `/` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:46:33.430550 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 71/146] 10.222.0.15 () {54 vars in 927 bytes} [Mon Aug 22 15:46:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 15:46:33.449898 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca42a95-f6a7-4d6c-ac9e-d77eedb0144f None None] REQUEST_METHOD: `POST` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:46:33.450900 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca42a95-f6a7-4d6c-ac9e-d77eedb0144f None None] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:46:33.451022 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca42a95-f6a7-4d6c-ac9e-d77eedb0144f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:46:33.481303 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.auth.core [None req-8ca42a95-f6a7-4d6c-ac9e-d77eedb0144f None None] MFA Rules not processed for user `6b2199ae0fb0480aa917ada391f5ec54`. Rule list: `[]` (Enabled: `True`). {{(pid=81768) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 15:46:33.482540 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-8ca42a95-f6a7-4d6c-ac9e-d77eedb0144f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 15:46:33.484900 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=100 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 15:46:33.589318 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 76/147] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Aug 22 15:46:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 22 15:46:33.821878 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea77ff29-1752-4745-9a29-f3925e7c1ac3 None None] REQUEST_METHOD: `GET` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:46:33.822405 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea77ff29-1752-4745-9a29-f3925e7c1ac3 None None] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:46:33.822479 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea77ff29-1752-4745-9a29-f3925e7c1ac3 None None] PATH_INFO: `/` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:46:33.823958 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 72/148] 10.222.0.15 () {54 vars in 927 bytes} [Mon Aug 22 15:46:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 15:46:33.831783 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d85aea68-cbb2-4a4e-9eb3-1b50dd11e875 None None] REQUEST_METHOD: `POST` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:46:33.832071 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d85aea68-cbb2-4a4e-9eb3-1b50dd11e875 None None] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:46:33.832350 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d85aea68-cbb2-4a4e-9eb3-1b50dd11e875 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:46:33.860341 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.auth.core [None req-d85aea68-cbb2-4a4e-9eb3-1b50dd11e875 None None] MFA Rules not processed for user `6b2199ae0fb0480aa917ada391f5ec54`. Rule list: `[]` (Enabled: `True`). {{(pid=81768) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 15:46:33.861642 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-d85aea68-cbb2-4a4e-9eb3-1b50dd11e875 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 15:46:33.862811 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a98a121-4c07-4150-b941-6a1ea66aadb2 None None] REQUEST_METHOD: `GET` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:46:33.863078 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a98a121-4c07-4150-b941-6a1ea66aadb2 None None] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:46:33.863371 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a98a121-4c07-4150-b941-6a1ea66aadb2 None None] PATH_INFO: `/` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:46:33.864707 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 73/149] 10.222.0.15 () {54 vars in 927 bytes} [Mon Aug 22 15:46:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 22 15:46:33.872085 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba776fd4-c68c-4a53-9f4d-70870b4adfaa None None] REQUEST_METHOD: `POST` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:46:33.872370 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba776fd4-c68c-4a53-9f4d-70870b4adfaa None None] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:46:33.872627 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba776fd4-c68c-4a53-9f4d-70870b4adfaa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:46:33.893978 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.auth.core [None req-ba776fd4-c68c-4a53-9f4d-70870b4adfaa None None] MFA Rules not processed for user `6b2199ae0fb0480aa917ada391f5ec54`. Rule list: `[]` (Enabled: `True`). {{(pid=81767) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 15:46:33.895147 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.fernet_utils [None req-ba776fd4-c68c-4a53-9f4d-70870b4adfaa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81767) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 15:46:33.978070 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 74/150] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Aug 22 15:46:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 22 15:46:34.021680 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 77/151] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Aug 22 15:46:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 22 15:46:34.074839 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba776fd4-c68c-4a53-9f4d-70870b4adfaa None None] Authenticating user token {{(pid=81767) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 15:46:34.180204 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG dbcounter [-] [81767] Writing DB stats keystone:SELECT=100 {{(pid=81767) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 15:46:34.192957 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba776fd4-c68c-4a53-9f4d-70870b4adfaa None None] Validating token access rules against request {{(pid=81767) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 15:46:34.228723 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a187e6f-9e19-47eb-ae08-ed50597a0596 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81767) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 15:46:34.229353 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a187e6f-9e19-47eb-ae08-ed50597a0596 None admin] REQUEST_METHOD: `GET` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:46:34.229623 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a187e6f-9e19-47eb-ae08-ed50597a0596 None admin] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:46:34.229894 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a187e6f-9e19-47eb-ae08-ed50597a0596 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:46:34.231142 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a187e6f-9e19-47eb-ae08-ed50597a0596 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 15:46:34.232605 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a187e6f-9e19-47eb-ae08-ed50597a0596 None admin] RBAC: Authorization granted {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 15:46:34.234387 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 75/152] 10.222.0.15 () {56 vars in 1108 bytes} [Mon Aug 22 15:46:34 2022] GET /identity/v3/domains/default => generated 199 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 15:46:34.239761 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d85aea68-cbb2-4a4e-9eb3-1b50dd11e875 None None] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 15:46:34.261856 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f422492-f3c2-430b-83eb-7906ff56b409 None None] REQUEST_METHOD: `GET` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:46:34.262172 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f422492-f3c2-430b-83eb-7906ff56b409 None None] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:46:34.262437 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f422492-f3c2-430b-83eb-7906ff56b409 None None] PATH_INFO: `/` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:46:34.264487 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 76/153] 10.222.0.15 () {54 vars in 927 bytes} [Mon Aug 22 15:46:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 22 15:46:34.269403 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28b44f9c-b1a8-4aca-acf9-af9f3fa77be8 None None] REQUEST_METHOD: `POST` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:46:34.269673 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28b44f9c-b1a8-4aca-acf9-af9f3fa77be8 None None] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:46:34.269941 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28b44f9c-b1a8-4aca-acf9-af9f3fa77be8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:46:34.291657 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.auth.core [None req-28b44f9c-b1a8-4aca-acf9-af9f3fa77be8 None None] MFA Rules not processed for user `6b2199ae0fb0480aa917ada391f5ec54`. Rule list: `[]` (Enabled: `True`). {{(pid=81767) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 15:46:34.292876 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.fernet_utils [None req-28b44f9c-b1a8-4aca-acf9-af9f3fa77be8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81767) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 15:46:34.361122 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=100 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 15:46:34.371492 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 77/154] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Aug 22 15:46:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 22 15:46:34.404598 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d85aea68-cbb2-4a4e-9eb3-1b50dd11e875 None None] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 15:46:34.440775 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee158d0a-ae8a-4104-9d34-97f8aa828d79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 15:46:34.441658 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee158d0a-ae8a-4104-9d34-97f8aa828d79 None admin] REQUEST_METHOD: `POST` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:46:34.442069 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee158d0a-ae8a-4104-9d34-97f8aa828d79 None admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:46:34.442512 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee158d0a-ae8a-4104-9d34-97f8aa828d79 None admin] PATH_INFO: `/v3/users` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:46:34.443310 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee158d0a-ae8a-4104-9d34-97f8aa828d79 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 15:46:34.444954 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee158d0a-ae8a-4104-9d34-97f8aa828d79 None admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 15:46:34.462853 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28b44f9c-b1a8-4aca-acf9-af9f3fa77be8 None None] Authenticating user token {{(pid=81767) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 15:46:34.555743 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG dbcounter [-] [81767] Writing DB stats keystone:SELECT=100 {{(pid=81767) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 15:46:34.572110 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28b44f9c-b1a8-4aca-acf9-af9f3fa77be8 None None] Validating token access rules against request {{(pid=81767) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 15:46:34.578415 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 78/155] 10.222.0.15 () {60 vars in 1132 bytes} [Mon Aug 22 15:46:34 2022] POST /identity/v3/users => generated 283 bytes in 339 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 22 15:46:34.579289 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee158d0a-ae8a-4104-9d34-97f8aa828d79 None admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 15:46:34.625307 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f34efbc2-09ad-493a-857a-21a73c15b473 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81767) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 15:46:34.625878 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f34efbc2-09ad-493a-857a-21a73c15b473 None admin] REQUEST_METHOD: `GET` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:46:34.626156 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f34efbc2-09ad-493a-857a-21a73c15b473 None admin] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:46:34.626426 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f34efbc2-09ad-493a-857a-21a73c15b473 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:46:34.660420 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f34efbc2-09ad-493a-857a-21a73c15b473 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 22 15:46:34.665354 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application [None req-f34efbc2-09ad-493a-857a-21a73c15b473 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 22 15:46:34.665354 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 15:46:34.665354 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 15:46:34.665354 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 15:46:34.665354 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 15:46:34.665354 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 15:46:34.665354 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 15:46:34.665354 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 15:46:34.665354 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 15:46:34.665354 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 15:46:34.665354 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 15:46:34.665354 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 15:46:34.665354 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 22 15:46:34.665354 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 22 15:46:34.665354 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 22 15:46:34.665354 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 22 15:46:34.666207 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 22 15:46:34.666207 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 22 15:46:34.666207 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 22 15:46:34.666207 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 22 15:46:34.666207 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 15:46:34.666207 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 15:46:34.666207 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 15:46:34.666207 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 15:46:34.666207 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 15:46:34.666207 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 15:46:34.666207 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 15:46:34.666207 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application with Lock( Aug 22 15:46:34.666207 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 15:46:34.666207 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return self._enter() Aug 22 15:46:34.666207 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 15:46:34.666207 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 15:46:34.667139 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 15:46:34.667139 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return self.creator() Aug 22 15:46:34.667139 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 15:46:34.667139 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application created_value = creator( Aug 22 15:46:34.667139 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 22 15:46:34.667139 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 22 15:46:34.667139 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 22 15:46:34.667139 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 22 15:46:34.667139 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 22 15:46:34.667139 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 22 15:46:34.667139 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 22 15:46:34.667139 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application Aug 22 15:46:34.671699 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 78/156] 10.222.0.15 () {56 vars in 1099 bytes} [Mon Aug 22 15:46:34 2022] GET /identity/v3/roles/reader => generated 84 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 15:46:34.676412 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f34efbc2-09ad-493a-857a-21a73c15b473 None admin] Authenticating user token {{(pid=81767) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 15:46:34.721628 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee158d0a-ae8a-4104-9d34-97f8aa828d79 None admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 15:46:34.752423 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 15:46:34.765882 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a8865ec-0265-4d16-b522-6aeaafae1dbc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 15:46:34.774656 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a8865ec-0265-4d16-b522-6aeaafae1dbc None admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:46:34.774656 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a8865ec-0265-4d16-b522-6aeaafae1dbc None admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:46:34.774656 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a8865ec-0265-4d16-b522-6aeaafae1dbc None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:46:34.785067 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a8865ec-0265-4d16-b522-6aeaafae1dbc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 22 15:46:34.791238 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application [None req-9a8865ec-0265-4d16-b522-6aeaafae1dbc None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 22 15:46:34.791238 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 15:46:34.791238 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 15:46:34.791238 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 15:46:34.791238 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 15:46:34.791238 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 15:46:34.791238 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 15:46:34.791238 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 15:46:34.791238 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 15:46:34.791238 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 15:46:34.791238 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 15:46:34.791238 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 15:46:34.791238 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 22 15:46:34.791238 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 22 15:46:34.791238 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 22 15:46:34.791238 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 22 15:46:34.792220 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 22 15:46:34.792220 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 22 15:46:34.792220 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 22 15:46:34.792220 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 22 15:46:34.792220 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 15:46:34.792220 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 15:46:34.792220 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 15:46:34.792220 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 15:46:34.792220 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 15:46:34.792220 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 15:46:34.792220 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 15:46:34.792220 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application with Lock( Aug 22 15:46:34.792220 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 15:46:34.792220 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self._enter() Aug 22 15:46:34.792220 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 15:46:34.792220 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 15:46:34.792982 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 15:46:34.792982 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self.creator() Aug 22 15:46:34.792982 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 15:46:34.792982 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application created_value = creator( Aug 22 15:46:34.792982 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 22 15:46:34.792982 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 22 15:46:34.792982 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 22 15:46:34.792982 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 22 15:46:34.792982 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 22 15:46:34.792982 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 22 15:46:34.792982 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 22 15:46:34.792982 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application Aug 22 15:46:34.793634 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 79/157] 10.222.0.15 () {56 vars in 1096 bytes} [Mon Aug 22 15:46:34 2022] GET /identity/v3/roles/admin => generated 83 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 15:46:34.794349 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a8865ec-0265-4d16-b522-6aeaafae1dbc None admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 15:46:34.847648 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f34efbc2-09ad-493a-857a-21a73c15b473 None admin] Validating token access rules against request {{(pid=81767) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 15:46:34.895329 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ab1d166-9c86-49f9-a435-cee5c07b1ed0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81767) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 15:46:34.895329 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab1d166-9c86-49f9-a435-cee5c07b1ed0 None admin] REQUEST_METHOD: `GET` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:46:34.895329 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab1d166-9c86-49f9-a435-cee5c07b1ed0 None admin] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:46:34.895329 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab1d166-9c86-49f9-a435-cee5c07b1ed0 None admin] PATH_INFO: `/v3/roles` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:46:34.895725 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ab1d166-9c86-49f9-a435-cee5c07b1ed0 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81767) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 15:46:34.895725 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ab1d166-9c86-49f9-a435-cee5c07b1ed0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 15:46:34.895725 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ab1d166-9c86-49f9-a435-cee5c07b1ed0 None admin] RBAC: Authorization granted {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 15:46:34.902535 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 79/158] 10.222.0.15 () {56 vars in 1101 bytes} [Mon Aug 22 15:46:34 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 15:46:34.903469 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ab1d166-9c86-49f9-a435-cee5c07b1ed0 None admin] Authenticating user token {{(pid=81767) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 15:46:34.917444 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG dbcounter [-] [81767] Writing DB stats keystone:SELECT=100 {{(pid=81767) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 15:46:34.920750 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a8865ec-0265-4d16-b522-6aeaafae1dbc None admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 15:46:34.966311 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0de2a54b-0477-417b-acb1-13faf8c012b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 15:46:34.967387 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0de2a54b-0477-417b-acb1-13faf8c012b2 None admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:46:34.967387 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0de2a54b-0477-417b-acb1-13faf8c012b2 None admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:46:34.967648 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0de2a54b-0477-417b-acb1-13faf8c012b2 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:46:34.979441 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0de2a54b-0477-417b-acb1-13faf8c012b2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 22 15:46:34.985901 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application [None req-0de2a54b-0477-417b-acb1-13faf8c012b2 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 22 15:46:34.985901 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 15:46:34.985901 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 15:46:34.985901 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 15:46:34.985901 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 15:46:34.985901 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 15:46:34.985901 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 15:46:34.985901 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 15:46:34.985901 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 15:46:34.985901 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 15:46:34.985901 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 15:46:34.985901 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 15:46:34.985901 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 22 15:46:34.985901 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 22 15:46:34.985901 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 22 15:46:34.985901 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 22 15:46:34.986994 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 22 15:46:34.986994 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 22 15:46:34.986994 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 22 15:46:34.986994 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 22 15:46:34.986994 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 15:46:34.986994 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 15:46:34.986994 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 15:46:34.986994 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 15:46:34.986994 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 15:46:34.986994 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 15:46:34.986994 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 15:46:34.986994 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application with Lock( Aug 22 15:46:34.986994 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 15:46:34.986994 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self._enter() Aug 22 15:46:34.986994 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 15:46:34.986994 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 15:46:34.988133 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 15:46:34.988133 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self.creator() Aug 22 15:46:34.988133 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 15:46:34.988133 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application created_value = creator( Aug 22 15:46:34.988133 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 22 15:46:34.988133 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 22 15:46:34.988133 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 22 15:46:34.988133 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 22 15:46:34.988133 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 22 15:46:34.988133 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 22 15:46:34.988133 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 22 15:46:34.988133 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application Aug 22 15:46:34.989511 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 80/159] 10.222.0.15 () {56 vars in 1099 bytes} [Mon Aug 22 15:46:34 2022] GET /identity/v3/roles/member => generated 84 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 15:46:34.990785 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed4938d-27c6-4b02-990c-be5609e46db3 None None] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:46:34.991068 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed4938d-27c6-4b02-990c-be5609e46db3 None None] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:46:34.991363 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed4938d-27c6-4b02-990c-be5609e46db3 None None] PATH_INFO: `/` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:46:34.992946 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 81/160] 10.222.0.15 () {54 vars in 927 bytes} [Mon Aug 22 15:46:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 15:46:34.993667 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fed4938d-27c6-4b02-990c-be5609e46db3 None None] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 15:46:35.044786 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ab1d166-9c86-49f9-a435-cee5c07b1ed0 None admin] Validating token access rules against request {{(pid=81767) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 15:46:35.047087 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=100 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 15:46:35.095318 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2a1252f-06ba-43d2-9e79-ca8745a94842 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81767) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 15:46:35.096109 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a1252f-06ba-43d2-9e79-ca8745a94842 None admin] REQUEST_METHOD: `GET` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:46:35.096388 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a1252f-06ba-43d2-9e79-ca8745a94842 None admin] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:46:35.096657 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a1252f-06ba-43d2-9e79-ca8745a94842 None admin] PATH_INFO: `/v3/roles` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:46:35.098022 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2a1252f-06ba-43d2-9e79-ca8745a94842 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81767) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 15:46:35.098022 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2a1252f-06ba-43d2-9e79-ca8745a94842 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 15:46:35.099060 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2a1252f-06ba-43d2-9e79-ca8745a94842 None admin] RBAC: Authorization granted {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 15:46:35.110466 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 80/161] 10.222.0.15 () {56 vars in 1099 bytes} [Mon Aug 22 15:46:34 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 15:46:35.110466 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2a1252f-06ba-43d2-9e79-ca8745a94842 None admin] Authenticating user token {{(pid=81767) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 15:46:35.124389 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fed4938d-27c6-4b02-990c-be5609e46db3 None None] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 15:46:35.174938 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c426a2d-1505-4c5b-a0b1-8d99580efb2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 15:46:35.175245 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c426a2d-1505-4c5b-a0b1-8d99580efb2e None admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:46:35.175489 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c426a2d-1505-4c5b-a0b1-8d99580efb2e None admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:46:35.175766 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c426a2d-1505-4c5b-a0b1-8d99580efb2e None admin] PATH_INFO: `/v3/users/b22a88ebd0c340cd9b0d140552d1b84e` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:46:35.178402 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c426a2d-1505-4c5b-a0b1-8d99580efb2e None admin] RBAC: Authorizing `identity:get_user(user_id=b22a88ebd0c340cd9b0d140552d1b84e)` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 15:46:35.180769 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c426a2d-1505-4c5b-a0b1-8d99580efb2e None admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 15:46:35.182724 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 82/162] 10.222.0.15 () {56 vars in 1177 bytes} [Mon Aug 22 15:46:34 2022] GET /identity/v3/users/b22a88ebd0c340cd9b0d140552d1b84e => generated 289 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 15:46:35.187299 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b70b75c7-9bf3-4328-a9bf-129dcdca92aa None None] REQUEST_METHOD: `POST` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:46:35.187299 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b70b75c7-9bf3-4328-a9bf-129dcdca92aa None None] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:46:35.187299 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b70b75c7-9bf3-4328-a9bf-129dcdca92aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:46:35.202587 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG dbcounter [-] [81767] Writing DB stats keystone:SELECT=100 {{(pid=81767) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 15:46:35.209684 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.auth.core [None req-b70b75c7-9bf3-4328-a9bf-129dcdca92aa None None] MFA Rules not processed for user `6b2199ae0fb0480aa917ada391f5ec54`. Rule list: `[]` (Enabled: `True`). {{(pid=81768) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 15:46:35.211007 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-b70b75c7-9bf3-4328-a9bf-129dcdca92aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 15:46:35.245996 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2a1252f-06ba-43d2-9e79-ca8745a94842 None admin] Validating token access rules against request {{(pid=81767) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 15:46:35.288023 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db3eaf51-a986-4544-a8c1-649a6934bfc3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81767) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 15:46:35.288665 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db3eaf51-a986-4544-a8c1-649a6934bfc3 None admin] REQUEST_METHOD: `GET` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:46:35.289197 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db3eaf51-a986-4544-a8c1-649a6934bfc3 None admin] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:46:35.289389 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db3eaf51-a986-4544-a8c1-649a6934bfc3 None admin] PATH_INFO: `/v3/roles` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:46:35.290294 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db3eaf51-a986-4544-a8c1-649a6934bfc3 None admin] RBAC: Adding query filter params (name=member) {{(pid=81767) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 15:46:35.290518 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db3eaf51-a986-4544-a8c1-649a6934bfc3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 15:46:35.293281 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db3eaf51-a986-4544-a8c1-649a6934bfc3 None admin] RBAC: Authorization granted {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 15:46:35.301902 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 83/163] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Aug 22 15:46:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 22 15:46:35.303000 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b70b75c7-9bf3-4328-a9bf-129dcdca92aa None None] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 15:46:35.306214 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 81/164] 10.222.0.15 () {56 vars in 1101 bytes} [Mon Aug 22 15:46:35 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 15:46:35.308204 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db3eaf51-a986-4544-a8c1-649a6934bfc3 None admin] Authenticating user token {{(pid=81767) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 15:46:35.365837 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=100 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 15:46:35.432739 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b70b75c7-9bf3-4328-a9bf-129dcdca92aa None None] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 15:46:35.471580 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30a51675-b8ed-43c1-8681-1777b5b49801 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 15:46:35.472304 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30a51675-b8ed-43c1-8681-1777b5b49801 None admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:46:35.472586 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30a51675-b8ed-43c1-8681-1777b5b49801 None admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:46:35.472857 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30a51675-b8ed-43c1-8681-1777b5b49801 None admin] PATH_INFO: `/v3/users/6b2199ae0fb0480aa917ada391f5ec54` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:46:35.474298 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30a51675-b8ed-43c1-8681-1777b5b49801 None admin] RBAC: Authorizing `identity:get_user(user_id=6b2199ae0fb0480aa917ada391f5ec54)` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 15:46:35.475976 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30a51675-b8ed-43c1-8681-1777b5b49801 None admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 15:46:35.477888 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 84/165] 10.222.0.15 () {56 vars in 1177 bytes} [Mon Aug 22 15:46:35 2022] GET /identity/v3/users/6b2199ae0fb0480aa917ada391f5ec54 => generated 247 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 15:46:35.478639 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30a51675-b8ed-43c1-8681-1777b5b49801 None admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 15:46:35.575428 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db3eaf51-a986-4544-a8c1-649a6934bfc3 None admin] Validating token access rules against request {{(pid=81767) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 15:46:35.593237 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30a51675-b8ed-43c1-8681-1777b5b49801 None admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 15:46:35.627324 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG dbcounter [-] [81767] Writing DB stats keystone:SELECT=100 {{(pid=81767) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 15:46:35.631110 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0501254-0e6b-453b-a30f-3e9ab417c6ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81767) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 15:46:35.632054 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0501254-0e6b-453b-a30f-3e9ab417c6ca None admin] REQUEST_METHOD: `GET` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:46:35.632499 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0501254-0e6b-453b-a30f-3e9ab417c6ca None admin] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:46:35.642748 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0501254-0e6b-453b-a30f-3e9ab417c6ca None admin] PATH_INFO: `/v3/projects/43171f5fd3454380b88625276d7375da` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:46:35.642748 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0501254-0e6b-453b-a30f-3e9ab417c6ca None admin] RBAC: Authorizing `identity:get_project(project_id=43171f5fd3454380b88625276d7375da)` {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 15:46:35.642922 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=100 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 15:46:35.642993 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0501254-0e6b-453b-a30f-3e9ab417c6ca None admin] RBAC: Authorization granted {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 15:46:35.642993 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 82/166] 10.222.0.15 () {56 vars in 1186 bytes} [Mon Aug 22 15:46:35 2022] GET /identity/v3/projects/43171f5fd3454380b88625276d7375da => generated 298 bytes in 330 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 15:46:35.642993 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0501254-0e6b-453b-a30f-3e9ab417c6ca None admin] Authenticating user token {{(pid=81767) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 15:46:35.668093 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99cc4093-e9fa-4ad8-95ce-f149d88c33d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 15:46:35.675901 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99cc4093-e9fa-4ad8-95ce-f149d88c33d0 None admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:46:35.675901 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99cc4093-e9fa-4ad8-95ce-f149d88c33d0 None admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:46:35.675901 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99cc4093-e9fa-4ad8-95ce-f149d88c33d0 None admin] PATH_INFO: `/v3/users/ad6aab8e70974f939bda68c2060aace5` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:46:35.675901 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99cc4093-e9fa-4ad8-95ce-f149d88c33d0 None admin] RBAC: Authorizing `identity:get_user(user_id=ad6aab8e70974f939bda68c2060aace5)` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 15:46:35.675901 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99cc4093-e9fa-4ad8-95ce-f149d88c33d0 None admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 15:46:35.675901 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 85/167] 10.222.0.15 () {56 vars in 1177 bytes} [Mon Aug 22 15:46:35 2022] GET /identity/v3/users/ad6aab8e70974f939bda68c2060aace5 => generated 275 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 15:46:35.676309 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99cc4093-e9fa-4ad8-95ce-f149d88c33d0 None admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 15:46:35.830867 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99cc4093-e9fa-4ad8-95ce-f149d88c33d0 None admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 15:46:35.866043 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0501254-0e6b-453b-a30f-3e9ab417c6ca None admin] Validating token access rules against request {{(pid=81767) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 15:46:35.874936 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed352e5d-977e-4a21-baae-5c38a8175fac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 15:46:35.875659 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed352e5d-977e-4a21-baae-5c38a8175fac None admin] REQUEST_METHOD: `PUT` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:46:35.876038 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed352e5d-977e-4a21-baae-5c38a8175fac None admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:46:35.876319 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed352e5d-977e-4a21-baae-5c38a8175fac None admin] PATH_INFO: `/v3/projects/43171f5fd3454380b88625276d7375da/users/b22a88ebd0c340cd9b0d140552d1b84e/roles/640209879fb24ac0ae3f65dd107697c3` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:46:35.878768 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed352e5d-977e-4a21-baae-5c38a8175fac None admin] RBAC: Authorizing `identity:create_grant(project_id=43171f5fd3454380b88625276d7375da, user_id=b22a88ebd0c340cd9b0d140552d1b84e, role_id=640209879fb24ac0ae3f65dd107697c3)` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 15:46:35.880198 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed352e5d-977e-4a21-baae-5c38a8175fac None admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 15:46:35.914402 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6461db1-c9ab-4ded-9838-7bbc78ff04ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81767) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 15:46:35.915007 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6461db1-c9ab-4ded-9838-7bbc78ff04ef None admin] REQUEST_METHOD: `GET` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:46:35.915298 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6461db1-c9ab-4ded-9838-7bbc78ff04ef None admin] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:46:35.915573 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6461db1-c9ab-4ded-9838-7bbc78ff04ef None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:46:35.954639 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 86/168] 10.222.0.15 () {58 vars in 1439 bytes} [Mon Aug 22 15:46:35 2022] PUT /identity/v3/projects/43171f5fd3454380b88625276d7375da/users/b22a88ebd0c340cd9b0d140552d1b84e/roles/640209879fb24ac0ae3f65dd107697c3 => generated 0 bytes in 280 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 15:46:35.955493 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed352e5d-977e-4a21-baae-5c38a8175fac None admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 15:46:36.035975 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG dbcounter [-] [81767] Writing DB stats keystone:SELECT=100 {{(pid=81767) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 15:46:36.040030 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 15:46:36.040254 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6461db1-c9ab-4ded-9838-7bbc78ff04ef None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 15:46:36.041633 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6461db1-c9ab-4ded-9838-7bbc78ff04ef None admin] RBAC: Authorization granted {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 15:46:36.125660 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed352e5d-977e-4a21-baae-5c38a8175fac None admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 15:46:36.175516 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4d73848-0674-4b09-a330-faf82490e4a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 15:46:36.176198 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d73848-0674-4b09-a330-faf82490e4a9 None admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:46:36.176525 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d73848-0674-4b09-a330-faf82490e4a9 None admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:46:36.176795 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d73848-0674-4b09-a330-faf82490e4a9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:46:36.291611 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 83/169] 10.222.0.15 () {58 vars in 1282 bytes} [Mon Aug 22 15:46:35 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 653 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 22 15:46:36.292802 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6461db1-c9ab-4ded-9838-7bbc78ff04ef None admin] Authenticating user token {{(pid=81767) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 15:46:36.305745 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4d73848-0674-4b09-a330-faf82490e4a9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 15:46:36.307948 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4d73848-0674-4b09-a330-faf82490e4a9 None admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 15:46:36.389281 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=100 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 15:46:36.488607 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG dbcounter [-] [81767] Writing DB stats keystone:SELECT=100 {{(pid=81767) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 15:46:36.493842 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6461db1-c9ab-4ded-9838-7bbc78ff04ef None admin] Validating token access rules against request {{(pid=81767) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 15:46:36.522293 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 87/170] 10.222.0.15 () {58 vars in 1282 bytes} [Mon Aug 22 15:46:35 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 567 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 22 15:46:36.524013 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4d73848-0674-4b09-a330-faf82490e4a9 None admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 15:46:36.577718 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf6f799b-89d9-4f85-80b5-c3dc458a2490 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81767) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 15:46:36.578380 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf6f799b-89d9-4f85-80b5-c3dc458a2490 None admin] REQUEST_METHOD: `GET` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:46:36.578662 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf6f799b-89d9-4f85-80b5-c3dc458a2490 None admin] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:46:36.578932 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf6f799b-89d9-4f85-80b5-c3dc458a2490 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:46:36.603991 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf6f799b-89d9-4f85-80b5-c3dc458a2490 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 22 15:46:36.610012 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application [None req-cf6f799b-89d9-4f85-80b5-c3dc458a2490 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 22 15:46:36.610012 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 15:46:36.610012 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 15:46:36.610012 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 15:46:36.610012 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 15:46:36.610012 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 15:46:36.610012 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 15:46:36.610012 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 15:46:36.610012 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 15:46:36.610012 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 15:46:36.610012 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 15:46:36.610012 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 15:46:36.610012 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 22 15:46:36.610012 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 22 15:46:36.610012 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 22 15:46:36.610012 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 22 15:46:36.610872 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 22 15:46:36.610872 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 22 15:46:36.610872 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 22 15:46:36.610872 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 22 15:46:36.610872 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 15:46:36.610872 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 15:46:36.610872 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 15:46:36.610872 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 15:46:36.610872 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 15:46:36.610872 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 15:46:36.610872 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 15:46:36.610872 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application with Lock( Aug 22 15:46:36.610872 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 15:46:36.610872 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return self._enter() Aug 22 15:46:36.610872 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 15:46:36.610872 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 15:46:36.611654 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 15:46:36.611654 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return self.creator() Aug 22 15:46:36.611654 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 15:46:36.611654 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application created_value = creator( Aug 22 15:46:36.611654 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 22 15:46:36.611654 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 22 15:46:36.611654 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 22 15:46:36.611654 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 22 15:46:36.611654 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 22 15:46:36.611654 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 22 15:46:36.611654 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 22 15:46:36.611654 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application Aug 22 15:46:36.611654 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 84/171] 10.222.0.15 () {56 vars in 1114 bytes} [Mon Aug 22 15:46:36 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 319 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 15:46:36.617022 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf6f799b-89d9-4f85-80b5-c3dc458a2490 None admin] Authenticating user token {{(pid=81767) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 15:46:36.775185 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4d73848-0674-4b09-a330-faf82490e4a9 None admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 15:46:36.840833 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf6f799b-89d9-4f85-80b5-c3dc458a2490 None admin] Validating token access rules against request {{(pid=81767) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 15:46:36.854691 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58ad2be7-54c5-495a-882d-6b8d0722d739 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 15:46:36.855400 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58ad2be7-54c5-495a-882d-6b8d0722d739 None admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:46:36.855775 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58ad2be7-54c5-495a-882d-6b8d0722d739 None admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:46:36.856582 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58ad2be7-54c5-495a-882d-6b8d0722d739 None admin] PATH_INFO: `/v3/projects/43171f5fd3454380b88625276d7375da` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:46:36.857746 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58ad2be7-54c5-495a-882d-6b8d0722d739 None admin] RBAC: Authorizing `identity:get_project(project_id=43171f5fd3454380b88625276d7375da)` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 15:46:36.859356 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58ad2be7-54c5-495a-882d-6b8d0722d739 None admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 15:46:36.861088 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 88/172] 10.222.0.15 () {56 vars in 1186 bytes} [Mon Aug 22 15:46:36 2022] GET /identity/v3/projects/43171f5fd3454380b88625276d7375da => generated 298 bytes in 338 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 15:46:36.862212 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58ad2be7-54c5-495a-882d-6b8d0722d739 None admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 15:46:36.879385 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=100 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 15:46:36.944279 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f762ce7a-56d6-4b61-9c10-c8b139db1f18 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81767) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 15:46:36.944944 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f762ce7a-56d6-4b61-9c10-c8b139db1f18 None admin] REQUEST_METHOD: `GET` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:46:36.945204 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f762ce7a-56d6-4b61-9c10-c8b139db1f18 None admin] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:46:36.945475 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f762ce7a-56d6-4b61-9c10-c8b139db1f18 None admin] PATH_INFO: `/v3/projects/92cc7eea42054bbdb6e4208d8f331345` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:46:36.946661 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f762ce7a-56d6-4b61-9c10-c8b139db1f18 None admin] RBAC: Authorizing `identity:get_project(project_id=92cc7eea42054bbdb6e4208d8f331345)` {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 15:46:36.948169 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f762ce7a-56d6-4b61-9c10-c8b139db1f18 None admin] RBAC: Authorization granted {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 15:46:36.949936 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 85/173] 10.222.0.15 () {56 vars in 1186 bytes} [Mon Aug 22 15:46:36 2022] GET /identity/v3/projects/92cc7eea42054bbdb6e4208d8f331345 => generated 312 bytes in 338 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 15:46:36.950628 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f762ce7a-56d6-4b61-9c10-c8b139db1f18 None admin] Authenticating user token {{(pid=81767) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 15:46:37.031727 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG dbcounter [-] [81767] Writing DB stats keystone:SELECT=100 {{(pid=81767) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 15:46:37.079393 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58ad2be7-54c5-495a-882d-6b8d0722d739 None admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 15:46:37.142275 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc2e7209-4563-448a-a77b-a9973db8c9cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 15:46:37.142892 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc2e7209-4563-448a-a77b-a9973db8c9cd None admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:46:37.143169 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc2e7209-4563-448a-a77b-a9973db8c9cd None admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:46:37.143472 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc2e7209-4563-448a-a77b-a9973db8c9cd None admin] PATH_INFO: `/v3/roles` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:46:37.144173 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc2e7209-4563-448a-a77b-a9973db8c9cd None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81768) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 15:46:37.145437 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc2e7209-4563-448a-a77b-a9973db8c9cd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 15:46:37.146851 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc2e7209-4563-448a-a77b-a9973db8c9cd None admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 15:46:37.156728 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 89/174] 10.222.0.15 () {56 vars in 1111 bytes} [Mon Aug 22 15:46:36 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 295 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 15:46:37.158135 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc2e7209-4563-448a-a77b-a9973db8c9cd None admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 15:46:37.206586 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f762ce7a-56d6-4b61-9c10-c8b139db1f18 None admin] Validating token access rules against request {{(pid=81767) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 15:46:37.256471 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=100 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 15:46:37.277589 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a7b71ff-d0ce-4c22-b899-e6048052f91d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81767) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 15:46:37.278269 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a7b71ff-d0ce-4c22-b899-e6048052f91d None admin] REQUEST_METHOD: `GET` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:46:37.278544 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a7b71ff-d0ce-4c22-b899-e6048052f91d None admin] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:46:37.278904 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a7b71ff-d0ce-4c22-b899-e6048052f91d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:46:37.279662 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a7b71ff-d0ce-4c22-b899-e6048052f91d None admin] RBAC: Adding query filter params (role.id=9d9848cfd22e4b089db443c350e56670, scope.project.id=43171f5fd3454380b88625276d7375da, user.id=6b2199ae0fb0480aa917ada391f5ec54) {{(pid=81767) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 15:46:37.280029 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a7b71ff-d0ce-4c22-b899-e6048052f91d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 15:46:37.281702 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a7b71ff-d0ce-4c22-b899-e6048052f91d None admin] RBAC: Authorization granted {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 15:46:37.295620 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 86/175] 10.222.0.15 () {56 vars in 1374 bytes} [Mon Aug 22 15:46:36 2022] GET /identity/v3/role_assignments?user.id=6b2199ae0fb0480aa917ada391f5ec54&scope.project.id=43171f5fd3454380b88625276d7375da&role.id=9d9848cfd22e4b089db443c350e56670 => generated 608 bytes in 345 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 15:46:37.296480 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a7b71ff-d0ce-4c22-b899-e6048052f91d None admin] Authenticating user token {{(pid=81767) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 15:46:37.324302 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc2e7209-4563-448a-a77b-a9973db8c9cd None admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 15:46:37.431566 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG dbcounter [-] [81767] Writing DB stats keystone:SELECT=100 {{(pid=81767) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 15:46:37.442533 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a7b71ff-d0ce-4c22-b899-e6048052f91d None admin] Validating token access rules against request {{(pid=81767) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 15:46:37.446221 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-273ba4a0-4414-47c6-9299-042d175d1df6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 15:46:37.449359 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-273ba4a0-4414-47c6-9299-042d175d1df6 None admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:46:37.449359 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-273ba4a0-4414-47c6-9299-042d175d1df6 None admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:46:37.449359 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-273ba4a0-4414-47c6-9299-042d175d1df6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:46:37.449359 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-273ba4a0-4414-47c6-9299-042d175d1df6 None admin] RBAC: Adding query filter params (role.id=e6fc1ee3fb974125ba9b4fe55b12b45e, scope.project.id=92cc7eea42054bbdb6e4208d8f331345, user.id=ad6aab8e70974f939bda68c2060aace5) {{(pid=81768) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 15:46:37.449359 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-273ba4a0-4414-47c6-9299-042d175d1df6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 15:46:37.450259 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-273ba4a0-4414-47c6-9299-042d175d1df6 None admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 15:46:37.466100 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 90/176] 10.222.0.15 () {56 vars in 1374 bytes} [Mon Aug 22 15:46:37 2022] GET /identity/v3/role_assignments?user.id=ad6aab8e70974f939bda68c2060aace5&scope.project.id=92cc7eea42054bbdb6e4208d8f331345&role.id=e6fc1ee3fb974125ba9b4fe55b12b45e => generated 608 bytes in 309 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 15:46:37.467527 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9e83b6d-18df-45ce-9022-5e2b0036e6fa None None] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:46:37.467786 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9e83b6d-18df-45ce-9022-5e2b0036e6fa None None] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:46:37.468060 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9e83b6d-18df-45ce-9022-5e2b0036e6fa None None] PATH_INFO: `/` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:46:37.475836 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 91/177] 10.222.0.15 () {54 vars in 927 bytes} [Mon Aug 22 15:46:37 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 15:46:37.481912 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e995cf9c-b60d-4183-bc73-8559438c3f6e None None] REQUEST_METHOD: `POST` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:46:37.482304 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e995cf9c-b60d-4183-bc73-8559438c3f6e None None] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:46:37.482468 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e995cf9c-b60d-4183-bc73-8559438c3f6e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:46:37.494287 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-711810b6-4e03-4411-831a-810aaa6f0b5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81767) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 15:46:37.495328 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-711810b6-4e03-4411-831a-810aaa6f0b5b None admin] REQUEST_METHOD: `GET` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:46:37.496020 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-711810b6-4e03-4411-831a-810aaa6f0b5b None admin] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:46:37.496487 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-711810b6-4e03-4411-831a-810aaa6f0b5b None admin] PATH_INFO: `/v3/users/ad6aab8e70974f939bda68c2060aace5` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:46:37.505300 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-711810b6-4e03-4411-831a-810aaa6f0b5b None admin] RBAC: Authorizing `identity:get_user(user_id=ad6aab8e70974f939bda68c2060aace5)` {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 15:46:37.507162 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-711810b6-4e03-4411-831a-810aaa6f0b5b None admin] RBAC: Authorization granted {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 15:46:37.509383 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 87/178] 10.222.0.15 () {56 vars in 1177 bytes} [Mon Aug 22 15:46:37 2022] GET /identity/v3/users/ad6aab8e70974f939bda68c2060aace5 => generated 275 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 15:46:37.515689 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-711810b6-4e03-4411-831a-810aaa6f0b5b None admin] Authenticating user token {{(pid=81767) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 15:46:37.521873 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.auth.core [None req-e995cf9c-b60d-4183-bc73-8559438c3f6e None None] MFA Rules not processed for user `6b2199ae0fb0480aa917ada391f5ec54`. Rule list: `[]` (Enabled: `True`). {{(pid=81768) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 15:46:37.523750 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-e995cf9c-b60d-4183-bc73-8559438c3f6e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 15:46:37.691681 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 92/179] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Aug 22 15:46:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 211 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 22 15:46:37.694220 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddb039a8-a271-4d7a-ae6a-ee986e723df3 None None] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:46:37.695473 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddb039a8-a271-4d7a-ae6a-ee986e723df3 None None] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:46:37.696099 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddb039a8-a271-4d7a-ae6a-ee986e723df3 None None] PATH_INFO: `/` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:46:37.698910 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 93/180] 10.222.0.15 () {54 vars in 927 bytes} [Mon Aug 22 15:46:37 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 15:46:37.704840 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c797a11-0304-4d38-bebf-20e13612a5ff None None] REQUEST_METHOD: `POST` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:46:37.705370 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c797a11-0304-4d38-bebf-20e13612a5ff None None] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:46:37.705824 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c797a11-0304-4d38-bebf-20e13612a5ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:46:37.712009 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-711810b6-4e03-4411-831a-810aaa6f0b5b None admin] Validating token access rules against request {{(pid=81767) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 15:46:37.737501 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.auth.core [None req-9c797a11-0304-4d38-bebf-20e13612a5ff None None] MFA Rules not processed for user `6b2199ae0fb0480aa917ada391f5ec54`. Rule list: `[]` (Enabled: `True`). {{(pid=81768) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 15:46:37.739687 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-9c797a11-0304-4d38-bebf-20e13612a5ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 15:46:37.804301 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-456adcdf-1616-4d62-94e7-5fb8b11dbbea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81767) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 15:46:37.804656 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-456adcdf-1616-4d62-94e7-5fb8b11dbbea None admin] REQUEST_METHOD: `GET` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:46:37.804935 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-456adcdf-1616-4d62-94e7-5fb8b11dbbea None admin] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:46:37.805201 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-456adcdf-1616-4d62-94e7-5fb8b11dbbea None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:46:37.827739 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 94/181] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Aug 22 15:46:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 22 15:46:37.828362 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=100 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 15:46:37.866341 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG dbcounter [-] [81767] Writing DB stats keystone:SELECT=100 {{(pid=81767) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 15:46:37.913545 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-456adcdf-1616-4d62-94e7-5fb8b11dbbea None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 15:46:37.915055 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-456adcdf-1616-4d62-94e7-5fb8b11dbbea None admin] RBAC: Authorization granted {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 15:46:38.034083 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 88/182] 10.222.0.15 () {58 vars in 1282 bytes} [Mon Aug 22 15:46:37 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 519 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 22 15:46:38.040329 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c797a11-0304-4d38-bebf-20e13612a5ff None None] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 15:46:38.048619 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-456adcdf-1616-4d62-94e7-5fb8b11dbbea None admin] Authenticating user token {{(pid=81767) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 15:46:38.176836 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG dbcounter [-] [81767] Writing DB stats keystone:SELECT=100 {{(pid=81767) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 15:46:38.187431 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c797a11-0304-4d38-bebf-20e13612a5ff None None] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 15:46:38.230016 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41beb014-c7f4-4501-8ee2-19550637ce8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 15:46:38.231688 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41beb014-c7f4-4501-8ee2-19550637ce8e None admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:46:38.232198 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41beb014-c7f4-4501-8ee2-19550637ce8e None admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:46:38.232668 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41beb014-c7f4-4501-8ee2-19550637ce8e None admin] PATH_INFO: `/v3/projects/43171f5fd3454380b88625276d7375da` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:46:38.234027 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41beb014-c7f4-4501-8ee2-19550637ce8e None admin] RBAC: Authorizing `identity:get_project(project_id=43171f5fd3454380b88625276d7375da)` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 15:46:38.235723 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41beb014-c7f4-4501-8ee2-19550637ce8e None admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 15:46:38.238384 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 95/183] 10.222.0.15 () {56 vars in 1186 bytes} [Mon Aug 22 15:46:38 2022] GET /identity/v3/projects/43171f5fd3454380b88625276d7375da => generated 298 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 15:46:38.248310 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41beb014-c7f4-4501-8ee2-19550637ce8e None admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 15:46:38.259989 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-456adcdf-1616-4d62-94e7-5fb8b11dbbea None admin] Validating token access rules against request {{(pid=81767) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 15:46:38.297852 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6d71389-80e5-4bb9-9e5f-11591422bf73 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81767) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 15:46:38.298860 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6d71389-80e5-4bb9-9e5f-11591422bf73 None admin] REQUEST_METHOD: `GET` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:46:38.299146 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6d71389-80e5-4bb9-9e5f-11591422bf73 None admin] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:46:38.299448 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6d71389-80e5-4bb9-9e5f-11591422bf73 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:46:38.311154 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6d71389-80e5-4bb9-9e5f-11591422bf73 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 22 15:46:38.316547 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application [None req-a6d71389-80e5-4bb9-9e5f-11591422bf73 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 22 15:46:38.316547 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 15:46:38.316547 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 15:46:38.316547 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 15:46:38.316547 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 15:46:38.316547 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 15:46:38.316547 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 15:46:38.316547 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 15:46:38.316547 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 15:46:38.316547 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 15:46:38.316547 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 15:46:38.316547 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 15:46:38.316547 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 22 15:46:38.316547 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 22 15:46:38.316547 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 22 15:46:38.316547 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 22 15:46:38.317626 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 22 15:46:38.317626 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 22 15:46:38.317626 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 22 15:46:38.317626 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 22 15:46:38.317626 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 15:46:38.317626 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 15:46:38.317626 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 15:46:38.317626 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 15:46:38.317626 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 15:46:38.317626 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 15:46:38.317626 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 15:46:38.317626 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application with Lock( Aug 22 15:46:38.317626 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 15:46:38.317626 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return self._enter() Aug 22 15:46:38.317626 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 15:46:38.317626 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 15:46:38.318401 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 15:46:38.318401 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return self.creator() Aug 22 15:46:38.318401 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 15:46:38.318401 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application created_value = creator( Aug 22 15:46:38.318401 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 22 15:46:38.318401 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 22 15:46:38.318401 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 22 15:46:38.318401 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 22 15:46:38.318401 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 22 15:46:38.318401 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 22 15:46:38.318401 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 22 15:46:38.318401 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application Aug 22 15:46:38.318401 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 89/184] 10.222.0.15 () {56 vars in 1096 bytes} [Mon Aug 22 15:46:38 2022] GET /identity/v3/roles/admin => generated 83 bytes in 270 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 15:46:38.319238 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8460bde1-5261-463b-b246-74f6290d7eff None None] REQUEST_METHOD: `GET` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:46:38.319382 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8460bde1-5261-463b-b246-74f6290d7eff None None] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:46:38.319679 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8460bde1-5261-463b-b246-74f6290d7eff None None] PATH_INFO: `/` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:46:38.323339 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 90/185] 10.222.0.15 () {54 vars in 927 bytes} [Mon Aug 22 15:46:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 15:46:38.323339 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8460bde1-5261-463b-b246-74f6290d7eff None None] Authenticating user token {{(pid=81767) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 15:46:38.328688 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=100 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 15:46:38.382954 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41beb014-c7f4-4501-8ee2-19550637ce8e None admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 15:46:38.421155 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-785cc485-bd7f-4b84-a833-e939a52c490b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 15:46:38.421773 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-785cc485-bd7f-4b84-a833-e939a52c490b None admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:46:38.422049 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-785cc485-bd7f-4b84-a833-e939a52c490b None admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:46:38.422327 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-785cc485-bd7f-4b84-a833-e939a52c490b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:46:38.423050 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-785cc485-bd7f-4b84-a833-e939a52c490b None admin] RBAC: Adding query filter params (role.id=1ce6b924c9604048906c07dbfb126da9, scope.project.id=43171f5fd3454380b88625276d7375da, user.id=ad6aab8e70974f939bda68c2060aace5) {{(pid=81768) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 15:46:38.423386 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-785cc485-bd7f-4b84-a833-e939a52c490b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 15:46:38.424929 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-785cc485-bd7f-4b84-a833-e939a52c490b None admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 15:46:38.439474 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 96/186] 10.222.0.15 () {56 vars in 1374 bytes} [Mon Aug 22 15:46:38 2022] GET /identity/v3/role_assignments?user.id=ad6aab8e70974f939bda68c2060aace5&scope.project.id=43171f5fd3454380b88625276d7375da&role.id=1ce6b924c9604048906c07dbfb126da9 => generated 608 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 15:46:38.439474 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-785cc485-bd7f-4b84-a833-e939a52c490b None admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 15:46:38.446586 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8460bde1-5261-463b-b246-74f6290d7eff None None] Validating token access rules against request {{(pid=81767) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 15:46:38.470313 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG dbcounter [-] [81767] Writing DB stats keystone:SELECT=100 {{(pid=81767) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 15:46:38.487415 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08717fec-13ed-42b9-be40-516ea41844dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81767) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 15:46:38.488112 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08717fec-13ed-42b9-be40-516ea41844dd None admin] REQUEST_METHOD: `GET` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:46:38.488384 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08717fec-13ed-42b9-be40-516ea41844dd None admin] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:46:38.488649 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08717fec-13ed-42b9-be40-516ea41844dd None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:46:38.502183 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08717fec-13ed-42b9-be40-516ea41844dd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 22 15:46:38.507185 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application [None req-08717fec-13ed-42b9-be40-516ea41844dd None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 22 15:46:38.507185 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 15:46:38.507185 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 15:46:38.507185 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 15:46:38.507185 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 15:46:38.507185 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 15:46:38.507185 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 15:46:38.507185 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 15:46:38.507185 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 15:46:38.507185 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 15:46:38.507185 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 15:46:38.507185 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 15:46:38.507185 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 22 15:46:38.507185 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 22 15:46:38.507185 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 22 15:46:38.507185 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 22 15:46:38.508239 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 22 15:46:38.508239 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 22 15:46:38.508239 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 22 15:46:38.508239 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 22 15:46:38.508239 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 15:46:38.508239 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 15:46:38.508239 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 15:46:38.508239 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 15:46:38.508239 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 15:46:38.508239 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 15:46:38.508239 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 15:46:38.508239 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application with Lock( Aug 22 15:46:38.508239 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 15:46:38.508239 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return self._enter() Aug 22 15:46:38.508239 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 15:46:38.508239 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 15:46:38.509010 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 15:46:38.509010 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return self.creator() Aug 22 15:46:38.509010 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 15:46:38.509010 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application created_value = creator( Aug 22 15:46:38.509010 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 22 15:46:38.509010 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 22 15:46:38.509010 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 22 15:46:38.509010 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 22 15:46:38.509010 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 22 15:46:38.509010 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 22 15:46:38.509010 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 22 15:46:38.509010 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application Aug 22 15:46:38.509010 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 91/187] 10.222.0.15 () {56 vars in 1114 bytes} [Mon Aug 22 15:46:38 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 15:46:38.516868 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3541d6f9-3aca-4912-9f8b-209e6d8e7a20 None None] REQUEST_METHOD: `POST` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:46:38.516868 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3541d6f9-3aca-4912-9f8b-209e6d8e7a20 None None] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:46:38.516868 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3541d6f9-3aca-4912-9f8b-209e6d8e7a20 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:46:38.536590 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.auth.core [None req-3541d6f9-3aca-4912-9f8b-209e6d8e7a20 None None] MFA Rules not processed for user `6b2199ae0fb0480aa917ada391f5ec54`. Rule list: `[]` (Enabled: `True`). {{(pid=81767) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 15:46:38.537810 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.fernet_utils [None req-3541d6f9-3aca-4912-9f8b-209e6d8e7a20 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81767) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 15:46:38.575691 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-785cc485-bd7f-4b84-a833-e939a52c490b None admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 15:46:38.605254 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=100 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 15:46:38.617018 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 92/188] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Aug 22 15:46:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 22 15:46:38.617975 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3541d6f9-3aca-4912-9f8b-209e6d8e7a20 None None] Authenticating user token {{(pid=81767) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 15:46:38.618218 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b025621f-5f5b-4d77-a78d-b370e56e320c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 15:46:38.618834 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b025621f-5f5b-4d77-a78d-b370e56e320c None admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:46:38.619114 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b025621f-5f5b-4d77-a78d-b370e56e320c None admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:46:38.619445 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b025621f-5f5b-4d77-a78d-b370e56e320c None admin] PATH_INFO: `/v3/roles` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:46:38.620219 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b025621f-5f5b-4d77-a78d-b370e56e320c None admin] RBAC: Adding query filter params (name=admin) {{(pid=81768) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 15:46:38.620599 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b025621f-5f5b-4d77-a78d-b370e56e320c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 15:46:38.622791 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b025621f-5f5b-4d77-a78d-b370e56e320c None admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 15:46:38.633328 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 97/189] 10.222.0.15 () {56 vars in 1099 bytes} [Mon Aug 22 15:46:38 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 15:46:38.640605 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b025621f-5f5b-4d77-a78d-b370e56e320c None admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 15:46:38.763103 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3541d6f9-3aca-4912-9f8b-209e6d8e7a20 None None] Validating token access rules against request {{(pid=81767) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 15:46:38.766664 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b025621f-5f5b-4d77-a78d-b370e56e320c None admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 15:46:38.778810 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG dbcounter [-] [81767] Writing DB stats keystone:SELECT=100 {{(pid=81767) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 15:46:38.811345 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38f8870d-0c41-46b4-8737-85b59d23173e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81767) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 15:46:38.811927 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38f8870d-0c41-46b4-8737-85b59d23173e None admin] REQUEST_METHOD: `GET` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:46:38.812205 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38f8870d-0c41-46b4-8737-85b59d23173e None admin] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:46:38.812474 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38f8870d-0c41-46b4-8737-85b59d23173e None admin] PATH_INFO: `/v3/roles` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:46:38.813146 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38f8870d-0c41-46b4-8737-85b59d23173e None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81767) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 15:46:38.813443 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38f8870d-0c41-46b4-8737-85b59d23173e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 15:46:38.814603 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5336110-1dae-4c77-89b6-73f07356b1b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 15:46:38.814822 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38f8870d-0c41-46b4-8737-85b59d23173e None admin] RBAC: Authorization granted {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 15:46:38.816656 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5336110-1dae-4c77-89b6-73f07356b1b9 None admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:46:38.816656 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5336110-1dae-4c77-89b6-73f07356b1b9 None admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:46:38.816656 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5336110-1dae-4c77-89b6-73f07356b1b9 None admin] PATH_INFO: `/v3/users/bfcf841e2f9e42fbb981ad7584d890cb` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:46:38.823522 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 93/190] 10.222.0.15 () {56 vars in 1111 bytes} [Mon Aug 22 15:46:38 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 15:46:38.824443 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b87a2c1-ca3f-4466-a8ea-55dc24668486 None None] REQUEST_METHOD: `GET` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:46:38.824754 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b87a2c1-ca3f-4466-a8ea-55dc24668486 None None] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:46:38.825328 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b87a2c1-ca3f-4466-a8ea-55dc24668486 None None] PATH_INFO: `/` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:46:38.826327 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 94/191] 10.222.0.15 () {54 vars in 927 bytes} [Mon Aug 22 15:46:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 15:46:38.831746 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b87a2c1-ca3f-4466-a8ea-55dc24668486 None None] Authenticating user token {{(pid=81767) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 15:46:38.837711 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5336110-1dae-4c77-89b6-73f07356b1b9 None admin] RBAC: Authorizing `identity:get_user(user_id=bfcf841e2f9e42fbb981ad7584d890cb)` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 15:46:38.839248 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5336110-1dae-4c77-89b6-73f07356b1b9 None admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 15:46:38.841174 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 98/192] 10.222.0.15 () {56 vars in 1177 bytes} [Mon Aug 22 15:46:38 2022] GET /identity/v3/users/bfcf841e2f9e42fbb981ad7584d890cb => generated 283 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 22 15:46:38.843193 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b691206-bf7a-40d6-add3-ede903d51fd1 None None] REQUEST_METHOD: `POST` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:46:38.843507 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b691206-bf7a-40d6-add3-ede903d51fd1 None None] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:46:38.843838 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b691206-bf7a-40d6-add3-ede903d51fd1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:46:38.863633 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.auth.core [None req-4b691206-bf7a-40d6-add3-ede903d51fd1 None None] MFA Rules not processed for user `6b2199ae0fb0480aa917ada391f5ec54`. Rule list: `[]` (Enabled: `True`). {{(pid=81768) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 15:46:38.868454 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-4b691206-bf7a-40d6-add3-ede903d51fd1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 15:46:38.907895 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=100 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 15:46:38.936795 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 99/193] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Aug 22 15:46:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 22 15:46:38.937644 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b691206-bf7a-40d6-add3-ede903d51fd1 None None] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 15:46:38.950358 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b87a2c1-ca3f-4466-a8ea-55dc24668486 None None] Validating token access rules against request {{(pid=81767) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 15:46:38.990989 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c133fab-74e9-4300-a3e4-fd13d7384173 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81767) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 15:46:38.991725 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c133fab-74e9-4300-a3e4-fd13d7384173 None admin] REQUEST_METHOD: `GET` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:46:38.991976 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c133fab-74e9-4300-a3e4-fd13d7384173 None admin] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:46:38.992244 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c133fab-74e9-4300-a3e4-fd13d7384173 None admin] PATH_INFO: `/v3/users/bfcf841e2f9e42fbb981ad7584d890cb` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:46:38.993378 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c133fab-74e9-4300-a3e4-fd13d7384173 None admin] RBAC: Authorizing `identity:get_user(user_id=bfcf841e2f9e42fbb981ad7584d890cb)` {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 15:46:38.994818 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c133fab-74e9-4300-a3e4-fd13d7384173 None admin] RBAC: Authorization granted {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 15:46:38.996457 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 95/194] 10.222.0.15 () {56 vars in 1177 bytes} [Mon Aug 22 15:46:38 2022] GET /identity/v3/users/bfcf841e2f9e42fbb981ad7584d890cb => generated 283 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 15:46:38.997786 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c133fab-74e9-4300-a3e4-fd13d7384173 None admin] Authenticating user token {{(pid=81767) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 15:46:39.044124 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG dbcounter [-] [81767] Writing DB stats keystone:SELECT=100 {{(pid=81767) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 15:46:39.055138 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b691206-bf7a-40d6-add3-ede903d51fd1 None None] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 15:46:39.093361 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bde0d6cf-e7fe-4b39-bff3-80b6e4dfe27a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 15:46:39.094340 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde0d6cf-e7fe-4b39-bff3-80b6e4dfe27a None admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:46:39.094763 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde0d6cf-e7fe-4b39-bff3-80b6e4dfe27a None admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:46:39.095206 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde0d6cf-e7fe-4b39-bff3-80b6e4dfe27a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:46:39.114203 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c133fab-74e9-4300-a3e4-fd13d7384173 None admin] Validating token access rules against request {{(pid=81767) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 15:46:39.144179 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=100 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 15:46:39.152472 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc92ef7c-5b40-4424-8a94-9398c1992aa9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81767) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 15:46:39.153840 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc92ef7c-5b40-4424-8a94-9398c1992aa9 None admin] REQUEST_METHOD: `GET` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:46:39.154282 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc92ef7c-5b40-4424-8a94-9398c1992aa9 None admin] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:46:39.154710 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc92ef7c-5b40-4424-8a94-9398c1992aa9 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:46:39.156014 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc92ef7c-5b40-4424-8a94-9398c1992aa9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 15:46:39.158962 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc92ef7c-5b40-4424-8a94-9398c1992aa9 None admin] RBAC: Authorization granted {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 15:46:39.159653 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 96/195] 10.222.0.15 () {56 vars in 1108 bytes} [Mon Aug 22 15:46:38 2022] GET /identity/v3/domains/default => generated 199 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 15:46:39.160578 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc92ef7c-5b40-4424-8a94-9398c1992aa9 None admin] Authenticating user token {{(pid=81767) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 15:46:39.176220 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bde0d6cf-e7fe-4b39-bff3-80b6e4dfe27a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 15:46:39.178023 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bde0d6cf-e7fe-4b39-bff3-80b6e4dfe27a None admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 15:46:39.280332 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc92ef7c-5b40-4424-8a94-9398c1992aa9 None admin] Validating token access rules against request {{(pid=81767) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 15:46:39.289151 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG dbcounter [-] [81767] Writing DB stats keystone:SELECT=100 {{(pid=81767) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 15:46:39.294472 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 100/196] 10.222.0.15 () {58 vars in 1282 bytes} [Mon Aug 22 15:46:38 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 357 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 22 15:46:39.295297 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bde0d6cf-e7fe-4b39-bff3-80b6e4dfe27a None admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 15:46:39.315684 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a920993-b2da-4c8f-a5a7-59b922d385b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81767) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 15:46:39.316285 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a920993-b2da-4c8f-a5a7-59b922d385b9 None admin] REQUEST_METHOD: `GET` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:46:39.316557 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a920993-b2da-4c8f-a5a7-59b922d385b9 None admin] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:46:39.316818 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a920993-b2da-4c8f-a5a7-59b922d385b9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:46:39.380737 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=100 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 15:46:39.386504 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a920993-b2da-4c8f-a5a7-59b922d385b9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 15:46:39.388092 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a920993-b2da-4c8f-a5a7-59b922d385b9 None admin] RBAC: Authorization granted {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 15:46:39.406031 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bde0d6cf-e7fe-4b39-bff3-80b6e4dfe27a None admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 15:46:39.440064 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40484698-348f-4d21-a0af-eb7e5ac30dc6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 15:46:39.440679 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40484698-348f-4d21-a0af-eb7e5ac30dc6 None admin] REQUEST_METHOD: `POST` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:46:39.440956 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40484698-348f-4d21-a0af-eb7e5ac30dc6 None admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:46:39.442086 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40484698-348f-4d21-a0af-eb7e5ac30dc6 None admin] PATH_INFO: `/v3/users` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:46:39.442086 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40484698-348f-4d21-a0af-eb7e5ac30dc6 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 15:46:39.443417 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40484698-348f-4d21-a0af-eb7e5ac30dc6 None admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 15:46:39.492954 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 97/197] 10.222.0.15 () {58 vars in 1282 bytes} [Mon Aug 22 15:46:39 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 333 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 22 15:46:39.493847 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a920993-b2da-4c8f-a5a7-59b922d385b9 None admin] Authenticating user token {{(pid=81767) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 15:46:39.504341 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG dbcounter [-] [81767] Writing DB stats keystone:SELECT=100 {{(pid=81767) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 15:46:39.598800 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a920993-b2da-4c8f-a5a7-59b922d385b9 None admin] Validating token access rules against request {{(pid=81767) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 15:46:39.633740 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7be720e-e371-4605-8448-43813714a02a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81767) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 15:46:39.634285 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7be720e-e371-4605-8448-43813714a02a None admin] REQUEST_METHOD: `GET` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:46:39.634566 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7be720e-e371-4605-8448-43813714a02a None admin] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:46:39.634865 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7be720e-e371-4605-8448-43813714a02a None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:46:39.646007 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7be720e-e371-4605-8448-43813714a02a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 22 15:46:39.650827 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application [None req-b7be720e-e371-4605-8448-43813714a02a None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 22 15:46:39.650827 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 15:46:39.650827 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 15:46:39.650827 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 15:46:39.650827 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 15:46:39.650827 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 15:46:39.650827 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 15:46:39.650827 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 15:46:39.650827 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 15:46:39.650827 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 15:46:39.650827 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 15:46:39.650827 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 15:46:39.650827 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 22 15:46:39.650827 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 22 15:46:39.650827 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 22 15:46:39.650827 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 22 15:46:39.658261 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 22 15:46:39.658261 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 22 15:46:39.658261 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 22 15:46:39.658261 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 22 15:46:39.658261 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 15:46:39.658261 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 15:46:39.658261 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 15:46:39.658261 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 15:46:39.658261 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 15:46:39.658261 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 15:46:39.658261 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 15:46:39.658261 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application with Lock( Aug 22 15:46:39.658261 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 15:46:39.658261 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return self._enter() Aug 22 15:46:39.658261 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 15:46:39.658261 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 15:46:39.659800 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 15:46:39.659800 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return self.creator() Aug 22 15:46:39.659800 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 15:46:39.659800 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application created_value = creator( Aug 22 15:46:39.659800 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 22 15:46:39.659800 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 22 15:46:39.659800 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 22 15:46:39.659800 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 22 15:46:39.659800 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 22 15:46:39.659800 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 22 15:46:39.659800 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 22 15:46:39.659800 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application Aug 22 15:46:39.661057 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 101/198] 10.222.0.15 () {60 vars in 1132 bytes} [Mon Aug 22 15:46:39 2022] POST /identity/v3/users => generated 297 bytes in 357 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 22 15:46:39.661214 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 98/199] 10.222.0.15 () {56 vars in 1099 bytes} [Mon Aug 22 15:46:39 2022] GET /identity/v3/roles/reader => generated 84 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 15:46:39.661214 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7be720e-e371-4605-8448-43813714a02a None admin] Authenticating user token {{(pid=81767) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 15:46:39.661488 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40484698-348f-4d21-a0af-eb7e5ac30dc6 None admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 15:46:39.747123 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG dbcounter [-] [81767] Writing DB stats keystone:SELECT=100 {{(pid=81767) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 15:46:39.787635 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7be720e-e371-4605-8448-43813714a02a None admin] Validating token access rules against request {{(pid=81767) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 15:46:39.789297 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40484698-348f-4d21-a0af-eb7e5ac30dc6 None admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 15:46:39.830018 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 15:46:39.834087 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dac1f6e-599e-486b-bd03-efd7e55fa8ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81767) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 15:46:39.834805 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dac1f6e-599e-486b-bd03-efd7e55fa8ca None admin] REQUEST_METHOD: `GET` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:46:39.835081 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dac1f6e-599e-486b-bd03-efd7e55fa8ca None admin] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:46:39.835404 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dac1f6e-599e-486b-bd03-efd7e55fa8ca None admin] PATH_INFO: `/v3/projects/c05dae9fd0dc44f9a2ab2d373bbc913f` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:46:39.839311 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dac1f6e-599e-486b-bd03-efd7e55fa8ca None admin] RBAC: Authorizing `identity:get_project(project_id=c05dae9fd0dc44f9a2ab2d373bbc913f)` {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 15:46:39.839455 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7633946-27ea-4d10-ae18-fc347ee2db45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 15:46:39.839862 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dac1f6e-599e-486b-bd03-efd7e55fa8ca None admin] RBAC: Authorization granted {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 15:46:39.839935 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7633946-27ea-4d10-ae18-fc347ee2db45 None admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:46:39.839935 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7633946-27ea-4d10-ae18-fc347ee2db45 None admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:46:39.839935 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7633946-27ea-4d10-ae18-fc347ee2db45 None admin] PATH_INFO: `/v3/projects/c05dae9fd0dc44f9a2ab2d373bbc913f` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:46:39.847783 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 99/200] 10.222.0.15 () {56 vars in 1186 bytes} [Mon Aug 22 15:46:39 2022] GET /identity/v3/projects/c05dae9fd0dc44f9a2ab2d373bbc913f => generated 302 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 15:46:39.847923 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7633946-27ea-4d10-ae18-fc347ee2db45 None admin] RBAC: Authorizing `identity:get_project(project_id=c05dae9fd0dc44f9a2ab2d373bbc913f)` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 15:46:39.847923 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7633946-27ea-4d10-ae18-fc347ee2db45 None admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 15:46:39.847923 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 102/201] 10.222.0.15 () {56 vars in 1186 bytes} [Mon Aug 22 15:46:39 2022] GET /identity/v3/projects/c05dae9fd0dc44f9a2ab2d373bbc913f => generated 302 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 15:46:39.848255 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dac1f6e-599e-486b-bd03-efd7e55fa8ca None admin] Authenticating user token {{(pid=81767) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 15:46:39.850654 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7633946-27ea-4d10-ae18-fc347ee2db45 None admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 15:46:39.979850 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dac1f6e-599e-486b-bd03-efd7e55fa8ca None admin] Validating token access rules against request {{(pid=81767) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 15:46:39.983158 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7633946-27ea-4d10-ae18-fc347ee2db45 None admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 15:46:40.020446 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aab7d885-5f67-4961-a276-9b7b712be683 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81767) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 15:46:40.020446 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aab7d885-5f67-4961-a276-9b7b712be683 None admin] REQUEST_METHOD: `GET` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:46:40.020910 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aab7d885-5f67-4961-a276-9b7b712be683 None admin] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:46:40.021069 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aab7d885-5f67-4961-a276-9b7b712be683 None admin] PATH_INFO: `/v3/roles` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:46:40.021716 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aab7d885-5f67-4961-a276-9b7b712be683 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81767) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 15:46:40.022010 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aab7d885-5f67-4961-a276-9b7b712be683 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 15:46:40.023958 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aab7d885-5f67-4961-a276-9b7b712be683 None admin] RBAC: Authorization granted {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 15:46:40.027708 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG dbcounter [-] [81767] Writing DB stats keystone:SELECT=100 {{(pid=81767) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 15:46:40.030110 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f028f19f-b5c0-4a56-9177-13f8733d6d7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 15:46:40.030739 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f028f19f-b5c0-4a56-9177-13f8733d6d7c None admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:46:40.031013 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f028f19f-b5c0-4a56-9177-13f8733d6d7c None admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:46:40.031322 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f028f19f-b5c0-4a56-9177-13f8733d6d7c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:46:40.032080 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f028f19f-b5c0-4a56-9177-13f8733d6d7c None admin] RBAC: Adding query filter params (role.id=9d9848cfd22e4b089db443c350e56670, scope.project.id=c05dae9fd0dc44f9a2ab2d373bbc913f, user.id=bfcf841e2f9e42fbb981ad7584d890cb) {{(pid=81768) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 15:46:40.032455 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f028f19f-b5c0-4a56-9177-13f8733d6d7c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 15:46:40.034022 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f028f19f-b5c0-4a56-9177-13f8733d6d7c None admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 15:46:40.036056 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 100/202] 10.222.0.15 () {56 vars in 1101 bytes} [Mon Aug 22 15:46:39 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 15:46:40.041208 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aab7d885-5f67-4961-a276-9b7b712be683 None admin] Authenticating user token {{(pid=81767) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 15:46:40.041334 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 103/203] 10.222.0.15 () {56 vars in 1374 bytes} [Mon Aug 22 15:46:39 2022] GET /identity/v3/role_assignments?user.id=bfcf841e2f9e42fbb981ad7584d890cb&scope.project.id=c05dae9fd0dc44f9a2ab2d373bbc913f&role.id=9d9848cfd22e4b089db443c350e56670 => generated 259 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 15:46:40.041761 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f028f19f-b5c0-4a56-9177-13f8733d6d7c None admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 15:46:40.144095 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=100 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 15:46:40.195092 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f028f19f-b5c0-4a56-9177-13f8733d6d7c None admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 15:46:40.245208 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-215f1a0d-c273-4a37-87ad-3726d5892785 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 15:46:40.246201 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-215f1a0d-c273-4a37-87ad-3726d5892785 None admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:46:40.247193 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-215f1a0d-c273-4a37-87ad-3726d5892785 None admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:46:40.247681 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-215f1a0d-c273-4a37-87ad-3726d5892785 None admin] PATH_INFO: `/v3/users/b22a88ebd0c340cd9b0d140552d1b84e` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:46:40.249028 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-215f1a0d-c273-4a37-87ad-3726d5892785 None admin] RBAC: Authorizing `identity:get_user(user_id=b22a88ebd0c340cd9b0d140552d1b84e)` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 15:46:40.250794 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-215f1a0d-c273-4a37-87ad-3726d5892785 None admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 15:46:40.253009 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 104/204] 10.222.0.15 () {56 vars in 1177 bytes} [Mon Aug 22 15:46:40 2022] GET /identity/v3/users/b22a88ebd0c340cd9b0d140552d1b84e => generated 289 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 15:46:40.260329 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-215f1a0d-c273-4a37-87ad-3726d5892785 None admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 15:46:40.269927 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aab7d885-5f67-4961-a276-9b7b712be683 None admin] Validating token access rules against request {{(pid=81767) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 15:46:40.343958 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c3571d5-c3d8-4766-81b4-c8fae48b7cd9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81767) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 15:46:40.344588 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c3571d5-c3d8-4766-81b4-c8fae48b7cd9 None admin] REQUEST_METHOD: `GET` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:46:40.344862 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c3571d5-c3d8-4766-81b4-c8fae48b7cd9 None admin] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:46:40.345132 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c3571d5-c3d8-4766-81b4-c8fae48b7cd9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:46:40.345874 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c3571d5-c3d8-4766-81b4-c8fae48b7cd9 None admin] RBAC: Adding query filter params (role.id=1ce6b924c9604048906c07dbfb126da9, scope.project.id=c05dae9fd0dc44f9a2ab2d373bbc913f, user.id=bfcf841e2f9e42fbb981ad7584d890cb) {{(pid=81767) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 15:46:40.346177 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c3571d5-c3d8-4766-81b4-c8fae48b7cd9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 15:46:40.349497 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c3571d5-c3d8-4766-81b4-c8fae48b7cd9 None admin] RBAC: Authorization granted {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 15:46:40.367478 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 101/205] 10.222.0.15 () {56 vars in 1374 bytes} [Mon Aug 22 15:46:40 2022] GET /identity/v3/role_assignments?user.id=bfcf841e2f9e42fbb981ad7584d890cb&scope.project.id=c05dae9fd0dc44f9a2ab2d373bbc913f&role.id=1ce6b924c9604048906c07dbfb126da9 => generated 259 bytes in 327 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 15:46:40.447425 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-215f1a0d-c273-4a37-87ad-3726d5892785 None admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 15:46:40.483166 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=100 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 15:46:40.489896 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bf504d1-eb69-4bbf-89d7-f0b8e6a6f34d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 15:46:40.498522 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf504d1-eb69-4bbf-89d7-f0b8e6a6f34d None admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:46:40.498522 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf504d1-eb69-4bbf-89d7-f0b8e6a6f34d None admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:46:40.498522 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf504d1-eb69-4bbf-89d7-f0b8e6a6f34d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:46:40.572283 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bf504d1-eb69-4bbf-89d7-f0b8e6a6f34d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 15:46:40.574058 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bf504d1-eb69-4bbf-89d7-f0b8e6a6f34d None admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 15:46:40.682117 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 105/206] 10.222.0.15 () {58 vars in 1282 bytes} [Mon Aug 22 15:46:40 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 422 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 22 15:46:40.687924 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c3571d5-c3d8-4766-81b4-c8fae48b7cd9 None admin] Authenticating user token {{(pid=81767) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 15:46:40.789259 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG dbcounter [-] [81767] Writing DB stats keystone:SELECT=100 {{(pid=81767) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 15:46:40.839461 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c3571d5-c3d8-4766-81b4-c8fae48b7cd9 None admin] Validating token access rules against request {{(pid=81767) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 15:46:40.886970 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-890d965f-bf0c-4feb-bea8-11643d43342c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81767) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 15:46:40.887660 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-890d965f-bf0c-4feb-bea8-11643d43342c None admin] REQUEST_METHOD: `GET` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:46:40.887939 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-890d965f-bf0c-4feb-bea8-11643d43342c None admin] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:46:40.888212 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-890d965f-bf0c-4feb-bea8-11643d43342c None admin] PATH_INFO: `/v3/projects/43171f5fd3454380b88625276d7375da` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:46:40.892044 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-890d965f-bf0c-4feb-bea8-11643d43342c None admin] RBAC: Authorizing `identity:get_project(project_id=43171f5fd3454380b88625276d7375da)` {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 15:46:40.893537 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-890d965f-bf0c-4feb-bea8-11643d43342c None admin] RBAC: Authorization granted {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 15:46:40.895311 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 102/207] 10.222.0.15 () {56 vars in 1186 bytes} [Mon Aug 22 15:46:40 2022] GET /identity/v3/projects/43171f5fd3454380b88625276d7375da => generated 298 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 15:46:40.901404 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bf504d1-eb69-4bbf-89d7-f0b8e6a6f34d None admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 15:46:40.941542 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=100 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 15:46:41.036442 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bf504d1-eb69-4bbf-89d7-f0b8e6a6f34d None admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 15:46:41.086998 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61b71bb2-3a80-45b8-83c6-1d26360f0acd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 15:46:41.087565 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b71bb2-3a80-45b8-83c6-1d26360f0acd None admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:46:41.087849 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b71bb2-3a80-45b8-83c6-1d26360f0acd None admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:46:41.088112 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b71bb2-3a80-45b8-83c6-1d26360f0acd None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:46:41.088849 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61b71bb2-3a80-45b8-83c6-1d26360f0acd None admin] RBAC: Adding query filter params (role.id=640209879fb24ac0ae3f65dd107697c3, scope.project.id=43171f5fd3454380b88625276d7375da, user.id=b22a88ebd0c340cd9b0d140552d1b84e) {{(pid=81768) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 15:46:41.089153 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61b71bb2-3a80-45b8-83c6-1d26360f0acd None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 15:46:41.091181 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61b71bb2-3a80-45b8-83c6-1d26360f0acd None admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 15:46:41.100010 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 106/208] 10.222.0.15 () {56 vars in 1374 bytes} [Mon Aug 22 15:46:40 2022] GET /identity/v3/role_assignments?user.id=b22a88ebd0c340cd9b0d140552d1b84e&scope.project.id=43171f5fd3454380b88625276d7375da&role.id=640209879fb24ac0ae3f65dd107697c3 => generated 608 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 15:46:42.075311 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2de7b6e9-269a-4eb1-8d60-dcff0517025b None None] REQUEST_METHOD: `GET` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:46:42.075828 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2de7b6e9-269a-4eb1-8d60-dcff0517025b None None] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:46:42.075828 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2de7b6e9-269a-4eb1-8d60-dcff0517025b None None] PATH_INFO: `/` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:46:42.077195 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 103/209] 10.222.0.15 () {54 vars in 927 bytes} [Mon Aug 22 15:46:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 15:46:42.078870 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f299e693-7d14-4bea-ab7b-7807af74e83b None None] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:46:42.079297 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f299e693-7d14-4bea-ab7b-7807af74e83b None None] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:46:42.079464 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f299e693-7d14-4bea-ab7b-7807af74e83b None None] PATH_INFO: `/` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:46:42.080899 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 107/210] 10.222.0.15 () {54 vars in 927 bytes} [Mon Aug 22 15:46:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 15:46:42.086326 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0259029e-f485-483d-96e4-1e95b5615deb None None] REQUEST_METHOD: `POST` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:46:42.086610 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0259029e-f485-483d-96e4-1e95b5615deb None None] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:46:42.086885 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0259029e-f485-483d-96e4-1e95b5615deb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:46:42.091505 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e117ac-954d-4b66-8f04-41fa475d475f None None] REQUEST_METHOD: `POST` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:46:42.091782 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e117ac-954d-4b66-8f04-41fa475d475f None None] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:46:42.092072 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e117ac-954d-4b66-8f04-41fa475d475f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:46:42.109634 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.auth.core [None req-0259029e-f485-483d-96e4-1e95b5615deb None None] MFA Rules not processed for user `6b2199ae0fb0480aa917ada391f5ec54`. Rule list: `[]` (Enabled: `True`). {{(pid=81767) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 15:46:42.110843 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.fernet_utils [None req-0259029e-f485-483d-96e4-1e95b5615deb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81767) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 15:46:42.113159 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.auth.core [None req-c6e117ac-954d-4b66-8f04-41fa475d475f None None] MFA Rules not processed for user `6b2199ae0fb0480aa917ada391f5ec54`. Rule list: `[]` (Enabled: `True`). {{(pid=81768) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 15:46:42.114955 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-c6e117ac-954d-4b66-8f04-41fa475d475f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 15:46:42.194781 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 104/211] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Aug 22 15:46:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 22 15:46:42.216395 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 108/212] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Aug 22 15:46:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 22 15:46:42.430684 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-894b2a0b-15fa-494c-9d8b-a8c8f7b7ee21 None None] REQUEST_METHOD: `GET` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:46:42.431454 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-894b2a0b-15fa-494c-9d8b-a8c8f7b7ee21 None None] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:46:42.431920 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-894b2a0b-15fa-494c-9d8b-a8c8f7b7ee21 None None] PATH_INFO: `/` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:46:42.435197 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 105/213] 10.222.0.15 () {54 vars in 927 bytes} [Mon Aug 22 15:46:42 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 15:46:42.441956 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47eeab58-2336-4186-9da3-6bddc7b098c8 None None] REQUEST_METHOD: `POST` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:46:42.442516 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47eeab58-2336-4186-9da3-6bddc7b098c8 None None] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:46:42.442984 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47eeab58-2336-4186-9da3-6bddc7b098c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:46:42.463775 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.auth.core [None req-47eeab58-2336-4186-9da3-6bddc7b098c8 None None] MFA Rules not processed for user `6b2199ae0fb0480aa917ada391f5ec54`. Rule list: `[]` (Enabled: `True`). {{(pid=81768) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 15:46:42.465948 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-47eeab58-2336-4186-9da3-6bddc7b098c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 15:46:42.476656 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-894b2a0b-15fa-494c-9d8b-a8c8f7b7ee21 None None] Authenticating user token {{(pid=81767) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 15:46:42.492775 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=100 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 15:46:42.546860 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG dbcounter [-] [81767] Writing DB stats keystone:SELECT=100 {{(pid=81767) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 15:46:42.563880 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 109/214] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Aug 22 15:46:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 22 15:46:42.565392 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47eeab58-2336-4186-9da3-6bddc7b098c8 None None] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 15:46:42.595024 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-894b2a0b-15fa-494c-9d8b-a8c8f7b7ee21 None None] Validating token access rules against request {{(pid=81767) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 15:46:42.632427 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b18d5ada-87ea-43ee-9985-09f0322c4277 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81767) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 15:46:42.633115 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b18d5ada-87ea-43ee-9985-09f0322c4277 None admin] REQUEST_METHOD: `GET` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:46:42.633392 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b18d5ada-87ea-43ee-9985-09f0322c4277 None admin] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:46:42.633769 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b18d5ada-87ea-43ee-9985-09f0322c4277 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:46:42.635006 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b18d5ada-87ea-43ee-9985-09f0322c4277 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 15:46:42.636488 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b18d5ada-87ea-43ee-9985-09f0322c4277 None admin] RBAC: Authorization granted {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 15:46:42.638283 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 106/215] 10.222.0.15 () {56 vars in 1108 bytes} [Mon Aug 22 15:46:42 2022] GET /identity/v3/domains/default => generated 199 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 22 15:46:42.643695 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b18d5ada-87ea-43ee-9985-09f0322c4277 None admin] Authenticating user token {{(pid=81767) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 15:46:42.691507 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47eeab58-2336-4186-9da3-6bddc7b098c8 None None] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 15:46:42.729101 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dbb98f9-9f6c-46da-b849-c96be227e2ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 15:46:42.729709 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dbb98f9-9f6c-46da-b849-c96be227e2ef None admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:46:42.729942 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dbb98f9-9f6c-46da-b849-c96be227e2ef None admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:46:42.730208 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dbb98f9-9f6c-46da-b849-c96be227e2ef None admin] PATH_INFO: `/v3/roles/member` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:46:42.741250 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dbb98f9-9f6c-46da-b849-c96be227e2ef None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 22 15:46:42.745989 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application [None req-4dbb98f9-9f6c-46da-b849-c96be227e2ef None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 22 15:46:42.745989 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 15:46:42.745989 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 15:46:42.745989 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 15:46:42.745989 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 15:46:42.745989 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 15:46:42.745989 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 15:46:42.745989 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 15:46:42.745989 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 15:46:42.745989 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 15:46:42.745989 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 15:46:42.745989 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 15:46:42.745989 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 22 15:46:42.745989 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 22 15:46:42.745989 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 22 15:46:42.745989 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 22 15:46:42.746951 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 22 15:46:42.746951 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 22 15:46:42.746951 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 22 15:46:42.746951 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 22 15:46:42.746951 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 15:46:42.746951 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 15:46:42.746951 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 15:46:42.746951 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 15:46:42.746951 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 15:46:42.746951 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 15:46:42.746951 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 15:46:42.746951 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application with Lock( Aug 22 15:46:42.746951 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 15:46:42.746951 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self._enter() Aug 22 15:46:42.746951 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 15:46:42.746951 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 15:46:42.748628 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 15:46:42.748628 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self.creator() Aug 22 15:46:42.748628 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 15:46:42.748628 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application created_value = creator( Aug 22 15:46:42.748628 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 22 15:46:42.748628 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 22 15:46:42.748628 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 22 15:46:42.748628 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 22 15:46:42.748628 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 22 15:46:42.748628 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 22 15:46:42.748628 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 22 15:46:42.748628 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application Aug 22 15:46:42.748628 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 110/216] 10.222.0.15 () {56 vars in 1099 bytes} [Mon Aug 22 15:46:42 2022] GET /identity/v3/roles/member => generated 84 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 15:46:42.748628 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c771ad4-c3f5-4258-88dd-42b38d484fe8 None None] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:46:42.749341 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c771ad4-c3f5-4258-88dd-42b38d484fe8 None None] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:46:42.749341 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c771ad4-c3f5-4258-88dd-42b38d484fe8 None None] PATH_INFO: `/` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:46:42.750270 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 111/217] 10.222.0.15 () {54 vars in 927 bytes} [Mon Aug 22 15:46:42 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 15:46:42.752324 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b18d5ada-87ea-43ee-9985-09f0322c4277 None admin] Validating token access rules against request {{(pid=81767) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 15:46:42.755725 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c771ad4-c3f5-4258-88dd-42b38d484fe8 None None] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 15:46:42.776891 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=100 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 15:46:42.781114 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG dbcounter [-] [81767] Writing DB stats keystone:SELECT=100 {{(pid=81767) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 15:46:42.788987 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18243ce0-c5b1-47de-a2b1-9dd488a9650d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81767) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 15:46:42.789600 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18243ce0-c5b1-47de-a2b1-9dd488a9650d None admin] REQUEST_METHOD: `POST` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:46:42.789881 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18243ce0-c5b1-47de-a2b1-9dd488a9650d None admin] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:46:42.790152 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18243ce0-c5b1-47de-a2b1-9dd488a9650d None admin] PATH_INFO: `/v3/users` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:46:42.790798 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18243ce0-c5b1-47de-a2b1-9dd488a9650d None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 15:46:42.792258 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18243ce0-c5b1-47de-a2b1-9dd488a9650d None admin] RBAC: Authorization granted {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 15:46:42.879292 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c771ad4-c3f5-4258-88dd-42b38d484fe8 None None] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 15:46:42.914317 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 107/218] 10.222.0.15 () {60 vars in 1132 bytes} [Mon Aug 22 15:46:42 2022] POST /identity/v3/users => generated 297 bytes in 271 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 22 15:46:42.915639 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fdeeeb7-6e28-4f5e-b85d-041599dd28d2 None None] REQUEST_METHOD: `POST` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:46:42.915929 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fdeeeb7-6e28-4f5e-b85d-041599dd28d2 None None] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:46:42.916243 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fdeeeb7-6e28-4f5e-b85d-041599dd28d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:46:42.925732 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51207365-746a-4409-95ad-813cd569971e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 15:46:42.926885 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51207365-746a-4409-95ad-813cd569971e None admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:46:42.927380 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51207365-746a-4409-95ad-813cd569971e None admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:46:42.927852 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51207365-746a-4409-95ad-813cd569971e None admin] PATH_INFO: `/v3/roles` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:46:42.928576 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51207365-746a-4409-95ad-813cd569971e None admin] RBAC: Adding query filter params (name=member) {{(pid=81768) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 15:46:42.928878 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51207365-746a-4409-95ad-813cd569971e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 15:46:42.930446 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51207365-746a-4409-95ad-813cd569971e None admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 15:46:42.943052 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.auth.core [None req-8fdeeeb7-6e28-4f5e-b85d-041599dd28d2 None None] MFA Rules not processed for user `6b2199ae0fb0480aa917ada391f5ec54`. Rule list: `[]` (Enabled: `True`). {{(pid=81767) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 15:46:42.943844 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 112/219] 10.222.0.15 () {56 vars in 1101 bytes} [Mon Aug 22 15:46:42 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 15:46:42.945006 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51207365-746a-4409-95ad-813cd569971e None admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 15:46:42.948658 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.fernet_utils [None req-8fdeeeb7-6e28-4f5e-b85d-041599dd28d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81767) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 15:46:43.040727 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 108/220] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Aug 22 15:46:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 22 15:46:43.041861 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fdeeeb7-6e28-4f5e-b85d-041599dd28d2 None None] Authenticating user token {{(pid=81767) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 15:46:43.048732 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=100 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 15:46:43.082553 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51207365-746a-4409-95ad-813cd569971e None admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 15:46:43.139121 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73007441-0a89-469f-bac9-2778e412b2d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 15:46:43.139712 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73007441-0a89-469f-bac9-2778e412b2d0 None admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:46:43.139991 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73007441-0a89-469f-bac9-2778e412b2d0 None admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:46:43.140260 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73007441-0a89-469f-bac9-2778e412b2d0 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:46:43.164203 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73007441-0a89-469f-bac9-2778e412b2d0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 22 15:46:43.171996 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application [None req-73007441-0a89-469f-bac9-2778e412b2d0 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 22 15:46:43.171996 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 15:46:43.171996 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 15:46:43.171996 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 15:46:43.171996 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 15:46:43.171996 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 15:46:43.171996 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 15:46:43.171996 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 15:46:43.171996 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 15:46:43.171996 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 15:46:43.171996 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 15:46:43.171996 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 15:46:43.171996 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 22 15:46:43.171996 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 22 15:46:43.171996 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 22 15:46:43.171996 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 22 15:46:43.172889 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 22 15:46:43.172889 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 22 15:46:43.172889 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 22 15:46:43.172889 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 22 15:46:43.172889 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 15:46:43.172889 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 15:46:43.172889 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 15:46:43.172889 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 15:46:43.172889 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 15:46:43.172889 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 15:46:43.172889 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 15:46:43.172889 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application with Lock( Aug 22 15:46:43.172889 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 15:46:43.172889 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self._enter() Aug 22 15:46:43.172889 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 15:46:43.172889 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 15:46:43.173659 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 15:46:43.173659 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self.creator() Aug 22 15:46:43.173659 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 15:46:43.173659 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application created_value = creator( Aug 22 15:46:43.173659 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 22 15:46:43.173659 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 22 15:46:43.173659 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 22 15:46:43.173659 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 22 15:46:43.173659 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 22 15:46:43.173659 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 22 15:46:43.173659 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 22 15:46:43.173659 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application Aug 22 15:46:43.174236 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 113/221] 10.222.0.15 () {56 vars in 1096 bytes} [Mon Aug 22 15:46:42 2022] GET /identity/v3/roles/admin => generated 83 bytes in 229 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 15:46:43.180688 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73007441-0a89-469f-bac9-2778e412b2d0 None admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 15:46:43.202080 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fdeeeb7-6e28-4f5e-b85d-041599dd28d2 None None] Validating token access rules against request {{(pid=81767) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 15:46:43.227925 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG dbcounter [-] [81767] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81767) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 15:46:43.255034 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1024e0e-4511-432e-a542-929537d805dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81767) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 15:46:43.255711 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1024e0e-4511-432e-a542-929537d805dc None admin] REQUEST_METHOD: `GET` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:46:43.255981 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1024e0e-4511-432e-a542-929537d805dc None admin] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:46:43.256275 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1024e0e-4511-432e-a542-929537d805dc None admin] PATH_INFO: `/v3/users/038404f0743144d7bbc039278710e57e` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:46:43.271097 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1024e0e-4511-432e-a542-929537d805dc None admin] RBAC: Authorizing `identity:get_user(user_id=038404f0743144d7bbc039278710e57e)` {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 15:46:43.272615 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1024e0e-4511-432e-a542-929537d805dc None admin] RBAC: Authorization granted {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 15:46:43.274388 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 109/222] 10.222.0.15 () {56 vars in 1177 bytes} [Mon Aug 22 15:46:43 2022] GET /identity/v3/users/038404f0743144d7bbc039278710e57e => generated 297 bytes in 233 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 15:46:43.281018 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1024e0e-4511-432e-a542-929537d805dc None admin] Authenticating user token {{(pid=81767) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 15:46:43.314956 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73007441-0a89-469f-bac9-2778e412b2d0 None admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 15:46:43.372140 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2db76e56-ccdb-46fe-a986-b376766d0bb3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 15:46:43.373127 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2db76e56-ccdb-46fe-a986-b376766d0bb3 None admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:46:43.373606 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2db76e56-ccdb-46fe-a986-b376766d0bb3 None admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:46:43.373933 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2db76e56-ccdb-46fe-a986-b376766d0bb3 None admin] PATH_INFO: `/v3/roles` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:46:43.374556 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2db76e56-ccdb-46fe-a986-b376766d0bb3 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81768) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 15:46:43.374856 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2db76e56-ccdb-46fe-a986-b376766d0bb3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 15:46:43.376338 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2db76e56-ccdb-46fe-a986-b376766d0bb3 None admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 15:46:43.379560 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=100 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 15:46:43.398511 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 114/223] 10.222.0.15 () {56 vars in 1099 bytes} [Mon Aug 22 15:46:43 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 15:46:43.398511 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2db76e56-ccdb-46fe-a986-b376766d0bb3 None admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 15:46:43.417049 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1024e0e-4511-432e-a542-929537d805dc None admin] Validating token access rules against request {{(pid=81767) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 15:46:43.470786 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81406f82-180a-454c-907a-ba0509269f9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81767) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 15:46:43.471967 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81406f82-180a-454c-907a-ba0509269f9d None admin] REQUEST_METHOD: `GET` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:46:43.472942 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81406f82-180a-454c-907a-ba0509269f9d None admin] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:46:43.473410 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81406f82-180a-454c-907a-ba0509269f9d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:46:43.530033 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2db76e56-ccdb-46fe-a986-b376766d0bb3 None admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 15:46:43.532018 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG dbcounter [-] [81767] Writing DB stats keystone:SELECT=100 {{(pid=81767) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 15:46:43.563634 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81406f82-180a-454c-907a-ba0509269f9d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 15:46:43.565608 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81406f82-180a-454c-907a-ba0509269f9d None admin] RBAC: Authorization granted {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 15:46:43.571830 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bfe00bf-a65f-46a6-99c6-f1f90ca840ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 15:46:43.572582 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bfe00bf-a65f-46a6-99c6-f1f90ca840ea None admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:46:43.572954 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bfe00bf-a65f-46a6-99c6-f1f90ca840ea None admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:46:43.573306 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bfe00bf-a65f-46a6-99c6-f1f90ca840ea None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:46:43.585869 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bfe00bf-a65f-46a6-99c6-f1f90ca840ea None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 22 15:46:43.593330 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application [None req-9bfe00bf-a65f-46a6-99c6-f1f90ca840ea None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 22 15:46:43.593330 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 15:46:43.593330 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 15:46:43.593330 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 15:46:43.593330 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 15:46:43.593330 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 15:46:43.593330 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 15:46:43.593330 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 15:46:43.593330 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 15:46:43.593330 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 15:46:43.593330 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 15:46:43.593330 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 15:46:43.593330 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 22 15:46:43.593330 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 22 15:46:43.593330 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 22 15:46:43.593330 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 22 15:46:43.594247 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 22 15:46:43.594247 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 22 15:46:43.594247 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 22 15:46:43.594247 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 22 15:46:43.594247 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 15:46:43.594247 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 15:46:43.594247 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 15:46:43.594247 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 15:46:43.594247 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 15:46:43.594247 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 15:46:43.594247 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 15:46:43.594247 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application with Lock( Aug 22 15:46:43.594247 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 15:46:43.594247 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self._enter() Aug 22 15:46:43.594247 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 15:46:43.594247 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 15:46:43.595045 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 15:46:43.595045 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self.creator() Aug 22 15:46:43.595045 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 15:46:43.595045 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application created_value = creator( Aug 22 15:46:43.595045 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 22 15:46:43.595045 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 22 15:46:43.595045 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 22 15:46:43.595045 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 22 15:46:43.595045 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 22 15:46:43.595045 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 22 15:46:43.595045 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 22 15:46:43.595045 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application Aug 22 15:46:43.595045 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 115/224] 10.222.0.15 () {56 vars in 1114 bytes} [Mon Aug 22 15:46:43 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 15:46:43.595045 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bfe00bf-a65f-46a6-99c6-f1f90ca840ea None admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 15:46:43.671162 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=100 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 15:46:43.712119 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 110/225] 10.222.0.15 () {58 vars in 1282 bytes} [Mon Aug 22 15:46:43 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 431 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 22 15:46:43.715329 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81406f82-180a-454c-907a-ba0509269f9d None admin] Authenticating user token {{(pid=81767) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 15:46:43.723974 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bfe00bf-a65f-46a6-99c6-f1f90ca840ea None admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 15:46:43.759943 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3afda3eb-0610-406b-9e0e-2ce23150bf11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 15:46:43.760615 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3afda3eb-0610-406b-9e0e-2ce23150bf11 None admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:46:43.760887 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3afda3eb-0610-406b-9e0e-2ce23150bf11 None admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:46:43.761162 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3afda3eb-0610-406b-9e0e-2ce23150bf11 None admin] PATH_INFO: `/v3/users/bfcf841e2f9e42fbb981ad7584d890cb` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:46:43.762887 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3afda3eb-0610-406b-9e0e-2ce23150bf11 None admin] RBAC: Authorizing `identity:get_user(user_id=bfcf841e2f9e42fbb981ad7584d890cb)` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 15:46:43.764577 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3afda3eb-0610-406b-9e0e-2ce23150bf11 None admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 15:46:43.766573 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 116/226] 10.222.0.15 () {56 vars in 1177 bytes} [Mon Aug 22 15:46:43 2022] GET /identity/v3/users/bfcf841e2f9e42fbb981ad7584d890cb => generated 283 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 15:46:43.767411 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3afda3eb-0610-406b-9e0e-2ce23150bf11 None admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 15:46:43.803865 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG dbcounter [-] [81767] Writing DB stats keystone:SELECT=100 {{(pid=81767) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 15:46:43.837592 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81406f82-180a-454c-907a-ba0509269f9d None admin] Validating token access rules against request {{(pid=81767) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 15:46:43.875319 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3afda3eb-0610-406b-9e0e-2ce23150bf11 None admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 15:46:43.882818 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f6a857d-0a36-41fd-80e9-dab1b7f561cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81767) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 15:46:43.883531 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f6a857d-0a36-41fd-80e9-dab1b7f561cf None admin] REQUEST_METHOD: `GET` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:46:43.883813 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f6a857d-0a36-41fd-80e9-dab1b7f561cf None admin] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:46:43.884090 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f6a857d-0a36-41fd-80e9-dab1b7f561cf None admin] PATH_INFO: `/v3/roles` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:46:43.884834 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f6a857d-0a36-41fd-80e9-dab1b7f561cf None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81767) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 15:46:43.885261 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f6a857d-0a36-41fd-80e9-dab1b7f561cf None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 15:46:43.886787 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f6a857d-0a36-41fd-80e9-dab1b7f561cf None admin] RBAC: Authorization granted {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 15:46:43.897117 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 111/227] 10.222.0.15 () {56 vars in 1111 bytes} [Mon Aug 22 15:46:43 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 15:46:43.898371 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f6a857d-0a36-41fd-80e9-dab1b7f561cf None admin] Authenticating user token {{(pid=81767) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 15:46:43.910049 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=100 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 15:46:43.921585 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4955f6b-2895-432e-a5b6-17621921dcd2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 15:46:43.922096 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4955f6b-2895-432e-a5b6-17621921dcd2 None admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:46:43.922378 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4955f6b-2895-432e-a5b6-17621921dcd2 None admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:46:43.922649 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4955f6b-2895-432e-a5b6-17621921dcd2 None admin] PATH_INFO: `/v3/projects/c05dae9fd0dc44f9a2ab2d373bbc913f` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:46:43.923980 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4955f6b-2895-432e-a5b6-17621921dcd2 None admin] RBAC: Authorizing `identity:get_project(project_id=c05dae9fd0dc44f9a2ab2d373bbc913f)` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 15:46:43.925673 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4955f6b-2895-432e-a5b6-17621921dcd2 None admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 15:46:43.927571 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 117/228] 10.222.0.15 () {56 vars in 1186 bytes} [Mon Aug 22 15:46:43 2022] GET /identity/v3/projects/c05dae9fd0dc44f9a2ab2d373bbc913f => generated 302 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 15:46:43.928164 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4955f6b-2895-432e-a5b6-17621921dcd2 None admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 15:46:44.022972 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f6a857d-0a36-41fd-80e9-dab1b7f561cf None admin] Validating token access rules against request {{(pid=81767) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 15:46:44.040538 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4955f6b-2895-432e-a5b6-17621921dcd2 None admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 15:46:44.062382 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f72e97b-7235-4f54-bc20-9fc3d8c1d167 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81767) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 15:46:44.063462 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f72e97b-7235-4f54-bc20-9fc3d8c1d167 None admin] REQUEST_METHOD: `GET` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:46:44.064043 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f72e97b-7235-4f54-bc20-9fc3d8c1d167 None admin] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:46:44.064546 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f72e97b-7235-4f54-bc20-9fc3d8c1d167 None admin] PATH_INFO: `/v3/projects/c05dae9fd0dc44f9a2ab2d373bbc913f` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:46:44.065918 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f72e97b-7235-4f54-bc20-9fc3d8c1d167 None admin] RBAC: Authorizing `identity:get_project(project_id=c05dae9fd0dc44f9a2ab2d373bbc913f)` {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 15:46:44.067643 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f72e97b-7235-4f54-bc20-9fc3d8c1d167 None admin] RBAC: Authorization granted {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 15:46:44.069618 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 112/229] 10.222.0.15 () {56 vars in 1186 bytes} [Mon Aug 22 15:46:43 2022] GET /identity/v3/projects/c05dae9fd0dc44f9a2ab2d373bbc913f => generated 302 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 15:46:44.071103 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f72e97b-7235-4f54-bc20-9fc3d8c1d167 None admin] Authenticating user token {{(pid=81767) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 15:46:44.076192 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG dbcounter [-] [81767] Writing DB stats keystone:SELECT=100 {{(pid=81767) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 15:46:44.078679 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-821789b1-9388-403e-87ee-287f62072538 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 15:46:44.079506 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-821789b1-9388-403e-87ee-287f62072538 None admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:46:44.079919 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-821789b1-9388-403e-87ee-287f62072538 None admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:46:44.080324 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-821789b1-9388-403e-87ee-287f62072538 None admin] PATH_INFO: `/v3/users/bfcf841e2f9e42fbb981ad7584d890cb` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:46:44.081620 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-821789b1-9388-403e-87ee-287f62072538 None admin] RBAC: Authorizing `identity:get_user(user_id=bfcf841e2f9e42fbb981ad7584d890cb)` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 15:46:44.083417 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-821789b1-9388-403e-87ee-287f62072538 None admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 15:46:44.085605 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 118/230] 10.222.0.15 () {56 vars in 1177 bytes} [Mon Aug 22 15:46:43 2022] GET /identity/v3/users/bfcf841e2f9e42fbb981ad7584d890cb => generated 283 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 15:46:44.086557 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-821789b1-9388-403e-87ee-287f62072538 None admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 15:46:44.175772 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=100 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 15:46:44.210745 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-821789b1-9388-403e-87ee-287f62072538 None admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 15:46:44.212791 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f72e97b-7235-4f54-bc20-9fc3d8c1d167 None admin] Validating token access rules against request {{(pid=81767) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 15:46:44.254602 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce6e08d3-3e13-4382-8ef9-165ef07bfe71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 15:46:44.256146 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce6e08d3-3e13-4382-8ef9-165ef07bfe71 None admin] REQUEST_METHOD: `PUT` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:46:44.256657 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce6e08d3-3e13-4382-8ef9-165ef07bfe71 None admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:46:44.257526 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce6e08d3-3e13-4382-8ef9-165ef07bfe71 None admin] PATH_INFO: `/v3/projects/c05dae9fd0dc44f9a2ab2d373bbc913f/users/bfcf841e2f9e42fbb981ad7584d890cb/roles/9d9848cfd22e4b089db443c350e56670` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:46:44.260326 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce6e08d3-3e13-4382-8ef9-165ef07bfe71 None admin] RBAC: Authorizing `identity:create_grant(project_id=c05dae9fd0dc44f9a2ab2d373bbc913f, user_id=bfcf841e2f9e42fbb981ad7584d890cb, role_id=9d9848cfd22e4b089db443c350e56670)` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 15:46:44.262641 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce6e08d3-3e13-4382-8ef9-165ef07bfe71 None admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 15:46:44.269087 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d678341b-82ab-45ac-8b79-baf35005a0bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81767) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 15:46:44.270001 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d678341b-82ab-45ac-8b79-baf35005a0bc None admin] REQUEST_METHOD: `GET` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:46:44.270452 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d678341b-82ab-45ac-8b79-baf35005a0bc None admin] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:46:44.270887 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d678341b-82ab-45ac-8b79-baf35005a0bc None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:46:44.272116 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d678341b-82ab-45ac-8b79-baf35005a0bc None admin] RBAC: Adding query filter params (role.id=e6fc1ee3fb974125ba9b4fe55b12b45e, scope.project.id=c05dae9fd0dc44f9a2ab2d373bbc913f, user.id=038404f0743144d7bbc039278710e57e) {{(pid=81767) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 15:46:44.272601 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d678341b-82ab-45ac-8b79-baf35005a0bc None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 15:46:44.274838 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d678341b-82ab-45ac-8b79-baf35005a0bc None admin] RBAC: Authorization granted {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 15:46:44.281267 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 113/231] 10.222.0.15 () {56 vars in 1374 bytes} [Mon Aug 22 15:46:44 2022] GET /identity/v3/role_assignments?user.id=038404f0743144d7bbc039278710e57e&scope.project.id=c05dae9fd0dc44f9a2ab2d373bbc913f&role.id=e6fc1ee3fb974125ba9b4fe55b12b45e => generated 259 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 15:46:44.281843 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d678341b-82ab-45ac-8b79-baf35005a0bc None admin] Authenticating user token {{(pid=81767) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 15:46:44.373439 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 119/232] 10.222.0.15 () {58 vars in 1439 bytes} [Mon Aug 22 15:46:44 2022] PUT /identity/v3/projects/c05dae9fd0dc44f9a2ab2d373bbc913f/users/bfcf841e2f9e42fbb981ad7584d890cb/roles/9d9848cfd22e4b089db443c350e56670 => generated 0 bytes in 287 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 15:46:44.383628 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG dbcounter [-] [81767] Writing DB stats keystone:SELECT=100 {{(pid=81767) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 15:46:44.432107 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d678341b-82ab-45ac-8b79-baf35005a0bc None admin] Validating token access rules against request {{(pid=81767) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 15:46:44.471359 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de736eff-af82-4327-86f7-ecfb09e6b12b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81767) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 15:46:44.472342 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de736eff-af82-4327-86f7-ecfb09e6b12b None admin] REQUEST_METHOD: `GET` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:46:44.472814 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de736eff-af82-4327-86f7-ecfb09e6b12b None admin] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:46:44.473350 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de736eff-af82-4327-86f7-ecfb09e6b12b None admin] PATH_INFO: `/v3/projects/c05dae9fd0dc44f9a2ab2d373bbc913f` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:46:44.475450 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de736eff-af82-4327-86f7-ecfb09e6b12b None admin] RBAC: Authorizing `identity:get_project(project_id=c05dae9fd0dc44f9a2ab2d373bbc913f)` {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 15:46:44.477265 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de736eff-af82-4327-86f7-ecfb09e6b12b None admin] RBAC: Authorization granted {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 15:46:44.479245 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 114/233] 10.222.0.15 () {56 vars in 1186 bytes} [Mon Aug 22 15:46:44 2022] GET /identity/v3/projects/c05dae9fd0dc44f9a2ab2d373bbc913f => generated 302 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 15:46:44.488444 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce6e08d3-3e13-4382-8ef9-165ef07bfe71 None admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 15:46:44.610852 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce6e08d3-3e13-4382-8ef9-165ef07bfe71 None admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 15:46:44.635586 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 15:46:44.647367 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0085dd79-576b-4a14-b75f-9f8746699a5d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 15:46:44.648008 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0085dd79-576b-4a14-b75f-9f8746699a5d None admin] REQUEST_METHOD: `PUT` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:46:44.648286 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0085dd79-576b-4a14-b75f-9f8746699a5d None admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:46:44.648789 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0085dd79-576b-4a14-b75f-9f8746699a5d None admin] PATH_INFO: `/v3/projects/c05dae9fd0dc44f9a2ab2d373bbc913f/users/bfcf841e2f9e42fbb981ad7584d890cb/roles/1ce6b924c9604048906c07dbfb126da9` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:46:44.650779 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0085dd79-576b-4a14-b75f-9f8746699a5d None admin] RBAC: Authorizing `identity:create_grant(project_id=c05dae9fd0dc44f9a2ab2d373bbc913f, user_id=bfcf841e2f9e42fbb981ad7584d890cb, role_id=1ce6b924c9604048906c07dbfb126da9)` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 15:46:44.652277 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0085dd79-576b-4a14-b75f-9f8746699a5d None admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 15:46:44.839994 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 120/234] 10.222.0.15 () {58 vars in 1439 bytes} [Mon Aug 22 15:46:44 2022] PUT /identity/v3/projects/c05dae9fd0dc44f9a2ab2d373bbc913f/users/bfcf841e2f9e42fbb981ad7584d890cb/roles/1ce6b924c9604048906c07dbfb126da9 => generated 0 bytes in 352 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 15:46:45.334622 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7a56b87-8177-44b3-9cd7-badf9cc63df4 None None] REQUEST_METHOD: `GET` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:46:45.335672 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7a56b87-8177-44b3-9cd7-badf9cc63df4 None None] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:46:45.336175 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7a56b87-8177-44b3-9cd7-badf9cc63df4 None None] PATH_INFO: `/` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:46:45.338032 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 115/235] 10.222.0.15 () {54 vars in 927 bytes} [Mon Aug 22 15:46:45 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 15:46:45.344524 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dae7f2ab-5bb1-46ae-a143-5a93d2de15bd None None] REQUEST_METHOD: `POST` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:46:45.344994 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dae7f2ab-5bb1-46ae-a143-5a93d2de15bd None None] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:46:45.345074 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dae7f2ab-5bb1-46ae-a143-5a93d2de15bd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:46:45.368810 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.auth.core [None req-dae7f2ab-5bb1-46ae-a143-5a93d2de15bd None None] MFA Rules not processed for user `6b2199ae0fb0480aa917ada391f5ec54`. Rule list: `[]` (Enabled: `True`). {{(pid=81768) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 15:46:45.370020 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-dae7f2ab-5bb1-46ae-a143-5a93d2de15bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 15:46:45.457648 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 121/236] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Aug 22 15:46:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 22 15:46:45.597900 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-638af8da-1392-418f-b265-a335beb8fa9c None None] REQUEST_METHOD: `GET` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:46:45.598108 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-638af8da-1392-418f-b265-a335beb8fa9c None None] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:46:45.598363 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-638af8da-1392-418f-b265-a335beb8fa9c None None] PATH_INFO: `/` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:46:45.600048 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 116/237] 10.222.0.15 () {54 vars in 927 bytes} [Mon Aug 22 15:46:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 15:46:45.605326 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff95da31-af58-4b87-81ae-e4f0e598911b None None] REQUEST_METHOD: `POST` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:46:45.605608 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff95da31-af58-4b87-81ae-e4f0e598911b None None] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:46:45.605914 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff95da31-af58-4b87-81ae-e4f0e598911b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:46:45.629934 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.auth.core [None req-ff95da31-af58-4b87-81ae-e4f0e598911b None None] MFA Rules not processed for user `6b2199ae0fb0480aa917ada391f5ec54`. Rule list: `[]` (Enabled: `True`). {{(pid=81768) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 15:46:45.631143 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-ff95da31-af58-4b87-81ae-e4f0e598911b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 15:46:45.721828 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 122/238] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Aug 22 15:46:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 22 15:46:45.853392 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-638af8da-1392-418f-b265-a335beb8fa9c None None] Authenticating user token {{(pid=81767) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 15:46:45.992989 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-638af8da-1392-418f-b265-a335beb8fa9c None None] Validating token access rules against request {{(pid=81767) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 15:46:46.001333 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff95da31-af58-4b87-81ae-e4f0e598911b None None] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 15:46:46.044575 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b428d8a-5558-48b6-ba20-952401e19882 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81767) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 15:46:46.045212 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b428d8a-5558-48b6-ba20-952401e19882 None admin] REQUEST_METHOD: `GET` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:46:46.045487 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b428d8a-5558-48b6-ba20-952401e19882 None admin] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:46:46.045753 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b428d8a-5558-48b6-ba20-952401e19882 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:46:46.054211 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG dbcounter [-] [81767] Writing DB stats keystone:SELECT=100 {{(pid=81767) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 15:46:46.065160 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b428d8a-5558-48b6-ba20-952401e19882 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 22 15:46:46.072523 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application [None req-2b428d8a-5558-48b6-ba20-952401e19882 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 22 15:46:46.072523 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 15:46:46.072523 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 15:46:46.072523 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 15:46:46.072523 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 15:46:46.072523 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 15:46:46.072523 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 15:46:46.072523 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 15:46:46.072523 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 15:46:46.072523 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 15:46:46.072523 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 15:46:46.072523 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 15:46:46.072523 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 22 15:46:46.072523 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 22 15:46:46.072523 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 22 15:46:46.072523 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 22 15:46:46.073407 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 22 15:46:46.073407 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 22 15:46:46.073407 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 22 15:46:46.073407 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 22 15:46:46.073407 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 15:46:46.073407 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 15:46:46.073407 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 15:46:46.073407 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 15:46:46.073407 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 15:46:46.073407 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 15:46:46.073407 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 15:46:46.073407 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application with Lock( Aug 22 15:46:46.073407 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 15:46:46.073407 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return self._enter() Aug 22 15:46:46.073407 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 15:46:46.073407 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 15:46:46.074171 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 15:46:46.074171 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return self.creator() Aug 22 15:46:46.074171 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 15:46:46.074171 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application created_value = creator( Aug 22 15:46:46.074171 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 22 15:46:46.074171 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 22 15:46:46.074171 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 22 15:46:46.074171 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 22 15:46:46.074171 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 22 15:46:46.074171 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 22 15:46:46.074171 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 22 15:46:46.074171 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application Aug 22 15:46:46.074171 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 117/239] 10.222.0.15 () {56 vars in 1099 bytes} [Mon Aug 22 15:46:45 2022] GET /identity/v3/roles/reader => generated 84 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 15:46:46.084891 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b428d8a-5558-48b6-ba20-952401e19882 None admin] Authenticating user token {{(pid=81767) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 15:46:46.092989 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 15:46:46.174220 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff95da31-af58-4b87-81ae-e4f0e598911b None None] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 15:46:46.241966 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67e7bd3e-c94c-4ca2-bea9-34b129d797f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 15:46:46.242976 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e7bd3e-c94c-4ca2-bea9-34b129d797f2 None admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:46:46.243481 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e7bd3e-c94c-4ca2-bea9-34b129d797f2 None admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:46:46.244025 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e7bd3e-c94c-4ca2-bea9-34b129d797f2 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:46:46.246825 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67e7bd3e-c94c-4ca2-bea9-34b129d797f2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 15:46:46.249749 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67e7bd3e-c94c-4ca2-bea9-34b129d797f2 None admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 15:46:46.251831 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 123/240] 10.222.0.15 () {56 vars in 1108 bytes} [Mon Aug 22 15:46:46 2022] GET /identity/v3/domains/default => generated 199 bytes in 251 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 15:46:46.258453 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67e7bd3e-c94c-4ca2-bea9-34b129d797f2 None admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 15:46:46.280924 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b428d8a-5558-48b6-ba20-952401e19882 None admin] Validating token access rules against request {{(pid=81767) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 15:46:46.348557 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9e33604-1174-48a0-8dc6-488149ca0670 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81767) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 15:46:46.350016 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9e33604-1174-48a0-8dc6-488149ca0670 None admin] REQUEST_METHOD: `GET` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:46:46.350492 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9e33604-1174-48a0-8dc6-488149ca0670 None admin] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:46:46.359880 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9e33604-1174-48a0-8dc6-488149ca0670 None admin] PATH_INFO: `/v3/roles` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:46:46.360964 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9e33604-1174-48a0-8dc6-488149ca0670 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81767) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 15:46:46.361465 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9e33604-1174-48a0-8dc6-488149ca0670 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 15:46:46.363140 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9e33604-1174-48a0-8dc6-488149ca0670 None admin] RBAC: Authorization granted {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 15:46:46.378725 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 118/241] 10.222.0.15 () {56 vars in 1101 bytes} [Mon Aug 22 15:46:46 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 294 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 22 15:46:46.385656 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9e33604-1174-48a0-8dc6-488149ca0670 None admin] Authenticating user token {{(pid=81767) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 15:46:46.444363 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67e7bd3e-c94c-4ca2-bea9-34b129d797f2 None admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 15:46:46.461761 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=100 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 15:46:46.467857 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG dbcounter [-] [81767] Writing DB stats keystone:SELECT=100 {{(pid=81767) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 15:46:46.519297 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9e33604-1174-48a0-8dc6-488149ca0670 None admin] Validating token access rules against request {{(pid=81767) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 15:46:46.523498 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ada20abf-67f5-4ea9-9f2e-cfb67ddd96e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 15:46:46.524113 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ada20abf-67f5-4ea9-9f2e-cfb67ddd96e3 None admin] REQUEST_METHOD: `POST` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:46:46.524448 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ada20abf-67f5-4ea9-9f2e-cfb67ddd96e3 None admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:46:46.524652 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ada20abf-67f5-4ea9-9f2e-cfb67ddd96e3 None admin] PATH_INFO: `/v3/users` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:46:46.525309 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ada20abf-67f5-4ea9-9f2e-cfb67ddd96e3 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 15:46:46.526807 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ada20abf-67f5-4ea9-9f2e-cfb67ddd96e3 None admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 15:46:46.557680 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaa8529b-f062-49bb-8b08-dfbec7f89022 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81767) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 15:46:46.558743 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaa8529b-f062-49bb-8b08-dfbec7f89022 None admin] REQUEST_METHOD: `GET` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:46:46.559987 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaa8529b-f062-49bb-8b08-dfbec7f89022 None admin] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:46:46.560535 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaa8529b-f062-49bb-8b08-dfbec7f89022 None admin] PATH_INFO: `/v3/users/f1f4430458e44ebcb8cfd11bf292c973` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:46:46.575854 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaa8529b-f062-49bb-8b08-dfbec7f89022 None admin] RBAC: Authorizing `identity:get_user(user_id=f1f4430458e44ebcb8cfd11bf292c973)` {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 15:46:46.577607 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaa8529b-f062-49bb-8b08-dfbec7f89022 None admin] RBAC: Authorization granted {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 15:46:46.579484 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 119/242] 10.222.0.15 () {56 vars in 1177 bytes} [Mon Aug 22 15:46:46 2022] GET /identity/v3/users/f1f4430458e44ebcb8cfd11bf292c973 => generated 297 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 15:46:46.580681 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0e85f9c-d8cc-49dc-9164-e9637792e145 None None] REQUEST_METHOD: `GET` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:46:46.580957 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0e85f9c-d8cc-49dc-9164-e9637792e145 None None] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:46:46.581222 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0e85f9c-d8cc-49dc-9164-e9637792e145 None None] PATH_INFO: `/` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:46:46.582632 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 120/243] 10.222.0.15 () {54 vars in 927 bytes} [Mon Aug 22 15:46:46 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 15:46:46.585023 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0e85f9c-d8cc-49dc-9164-e9637792e145 None None] Authenticating user token {{(pid=81767) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 15:46:46.695726 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 124/244] 10.222.0.15 () {60 vars in 1132 bytes} [Mon Aug 22 15:46:46 2022] POST /identity/v3/users => generated 293 bytes in 437 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 22 15:46:46.697448 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d35d3f-829a-4306-bdcd-d9b04c155d75 None None] REQUEST_METHOD: `POST` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:46:46.697995 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d35d3f-829a-4306-bdcd-d9b04c155d75 None None] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:46:46.698495 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d35d3f-829a-4306-bdcd-d9b04c155d75 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:46:46.712096 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0e85f9c-d8cc-49dc-9164-e9637792e145 None None] Validating token access rules against request {{(pid=81767) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 15:46:46.721997 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.auth.core [None req-31d35d3f-829a-4306-bdcd-d9b04c155d75 None None] MFA Rules not processed for user `6b2199ae0fb0480aa917ada391f5ec54`. Rule list: `[]` (Enabled: `True`). {{(pid=81768) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 15:46:46.723607 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-31d35d3f-829a-4306-bdcd-d9b04c155d75 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 15:46:46.741016 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG dbcounter [-] [81767] Writing DB stats keystone:SELECT=100 {{(pid=81767) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 15:46:46.813612 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 125/245] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Aug 22 15:46:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 22 15:46:46.821087 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c567f59-1847-4164-a22c-a3e2d655fe48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81767) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 15:46:46.821789 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c567f59-1847-4164-a22c-a3e2d655fe48 None admin] REQUEST_METHOD: `GET` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:46:46.822067 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c567f59-1847-4164-a22c-a3e2d655fe48 None admin] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:46:46.822338 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c567f59-1847-4164-a22c-a3e2d655fe48 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:46:46.832799 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc76641d-731d-4c63-848c-3d6a13767d8f None None] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:46:46.833099 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc76641d-731d-4c63-848c-3d6a13767d8f None None] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:46:46.833345 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc76641d-731d-4c63-848c-3d6a13767d8f None None] PATH_INFO: `/` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:46:46.834988 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 126/246] 10.222.0.15 () {54 vars in 927 bytes} [Mon Aug 22 15:46:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 22 15:46:46.849885 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ffa9cd6-8be4-46be-b170-b708d5cb18d7 None None] REQUEST_METHOD: `POST` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:46:46.850150 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ffa9cd6-8be4-46be-b170-b708d5cb18d7 None None] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:46:46.850413 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ffa9cd6-8be4-46be-b170-b708d5cb18d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:46:46.900048 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.auth.core [None req-2ffa9cd6-8be4-46be-b170-b708d5cb18d7 None None] MFA Rules not processed for user `6b2199ae0fb0480aa917ada391f5ec54`. Rule list: `[]` (Enabled: `True`). {{(pid=81768) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 15:46:46.901290 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-2ffa9cd6-8be4-46be-b170-b708d5cb18d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 15:46:46.923643 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c567f59-1847-4164-a22c-a3e2d655fe48 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 15:46:46.925478 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c567f59-1847-4164-a22c-a3e2d655fe48 None admin] RBAC: Authorization granted {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 15:46:46.989547 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 127/247] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Aug 22 15:46:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 22 15:46:47.050659 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 121/248] 10.222.0.15 () {58 vars in 1282 bytes} [Mon Aug 22 15:46:46 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 466 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 22 15:46:47.056458 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ffa9cd6-8be4-46be-b170-b708d5cb18d7 None None] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 15:46:47.091840 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 15:46:47.159876 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c567f59-1847-4164-a22c-a3e2d655fe48 None admin] Authenticating user token {{(pid=81767) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 15:46:47.178158 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG dbcounter [-] [81767] Writing DB stats keystone:SELECT=100 {{(pid=81767) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 15:46:47.193352 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ffa9cd6-8be4-46be-b170-b708d5cb18d7 None None] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 15:46:47.231349 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b522d4a-b7ca-4ddb-a700-3f78142bb08f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 15:46:47.232332 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b522d4a-b7ca-4ddb-a700-3f78142bb08f None admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:46:47.232837 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b522d4a-b7ca-4ddb-a700-3f78142bb08f None admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:46:47.233321 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b522d4a-b7ca-4ddb-a700-3f78142bb08f None admin] PATH_INFO: `/v3/projects/c05dae9fd0dc44f9a2ab2d373bbc913f` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:46:47.234820 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b522d4a-b7ca-4ddb-a700-3f78142bb08f None admin] RBAC: Authorizing `identity:get_project(project_id=c05dae9fd0dc44f9a2ab2d373bbc913f)` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 15:46:47.236572 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b522d4a-b7ca-4ddb-a700-3f78142bb08f None admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 15:46:47.238470 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 128/249] 10.222.0.15 () {56 vars in 1186 bytes} [Mon Aug 22 15:46:47 2022] GET /identity/v3/projects/c05dae9fd0dc44f9a2ab2d373bbc913f => generated 302 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 22 15:46:47.244509 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b522d4a-b7ca-4ddb-a700-3f78142bb08f None admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 15:46:47.302146 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c567f59-1847-4164-a22c-a3e2d655fe48 None admin] Validating token access rules against request {{(pid=81767) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 15:46:47.348365 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fd59afb-eac8-4797-b883-e7e48904fc41 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81767) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 15:46:47.349097 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fd59afb-eac8-4797-b883-e7e48904fc41 None admin] REQUEST_METHOD: `GET` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:46:47.349655 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fd59afb-eac8-4797-b883-e7e48904fc41 None admin] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:46:47.349655 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fd59afb-eac8-4797-b883-e7e48904fc41 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:46:47.370991 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fd59afb-eac8-4797-b883-e7e48904fc41 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 22 15:46:47.376099 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application [None req-5fd59afb-eac8-4797-b883-e7e48904fc41 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 22 15:46:47.376099 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 15:46:47.376099 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 15:46:47.376099 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 15:46:47.376099 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 15:46:47.376099 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 15:46:47.376099 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 15:46:47.376099 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 15:46:47.376099 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 15:46:47.376099 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 15:46:47.376099 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 15:46:47.376099 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 15:46:47.376099 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 22 15:46:47.376099 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 22 15:46:47.376099 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 22 15:46:47.376099 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 22 15:46:47.377123 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 22 15:46:47.377123 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 22 15:46:47.377123 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 22 15:46:47.377123 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 22 15:46:47.377123 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 15:46:47.377123 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 15:46:47.377123 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 15:46:47.377123 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 15:46:47.377123 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 15:46:47.377123 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 15:46:47.377123 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 15:46:47.377123 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application with Lock( Aug 22 15:46:47.377123 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 15:46:47.377123 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return self._enter() Aug 22 15:46:47.377123 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 15:46:47.377123 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 15:46:47.377899 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 15:46:47.377899 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return self.creator() Aug 22 15:46:47.377899 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 15:46:47.377899 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application created_value = creator( Aug 22 15:46:47.377899 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 22 15:46:47.377899 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 22 15:46:47.377899 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 22 15:46:47.377899 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 22 15:46:47.377899 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 22 15:46:47.377899 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 22 15:46:47.377899 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 22 15:46:47.377899 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application Aug 22 15:46:47.379393 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 122/250] 10.222.0.15 () {56 vars in 1099 bytes} [Mon Aug 22 15:46:47 2022] GET /identity/v3/roles/member => generated 84 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 22 15:46:47.379846 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=100 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 15:46:47.380291 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fd59afb-eac8-4797-b883-e7e48904fc41 None admin] Authenticating user token {{(pid=81767) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 15:46:47.390166 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b522d4a-b7ca-4ddb-a700-3f78142bb08f None admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 15:46:47.444887 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-292ee82f-0ad4-4ed4-a107-858c1e630bf3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 15:46:47.445942 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-292ee82f-0ad4-4ed4-a107-858c1e630bf3 None admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:46:47.446228 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-292ee82f-0ad4-4ed4-a107-858c1e630bf3 None admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:46:47.446503 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-292ee82f-0ad4-4ed4-a107-858c1e630bf3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:46:47.448011 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-292ee82f-0ad4-4ed4-a107-858c1e630bf3 None admin] RBAC: Adding query filter params (role.id=640209879fb24ac0ae3f65dd107697c3, scope.project.id=c05dae9fd0dc44f9a2ab2d373bbc913f, user.id=f1f4430458e44ebcb8cfd11bf292c973) {{(pid=81768) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 15:46:47.448330 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-292ee82f-0ad4-4ed4-a107-858c1e630bf3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 15:46:47.450452 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-292ee82f-0ad4-4ed4-a107-858c1e630bf3 None admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 15:46:47.459180 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 129/251] 10.222.0.15 () {56 vars in 1374 bytes} [Mon Aug 22 15:46:47 2022] GET /identity/v3/role_assignments?user.id=f1f4430458e44ebcb8cfd11bf292c973&scope.project.id=c05dae9fd0dc44f9a2ab2d373bbc913f&role.id=640209879fb24ac0ae3f65dd107697c3 => generated 259 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 15:46:47.460545 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cce75dc-dddf-4774-8afd-24c2a50eee1a None None] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:46:47.460833 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cce75dc-dddf-4774-8afd-24c2a50eee1a None None] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:46:47.461125 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cce75dc-dddf-4774-8afd-24c2a50eee1a None None] PATH_INFO: `/` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:46:47.462452 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 130/252] 10.222.0.15 () {54 vars in 927 bytes} [Mon Aug 22 15:46:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 15:46:47.467407 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cce75dc-dddf-4774-8afd-24c2a50eee1a None None] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 15:46:47.500464 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG dbcounter [-] [81767] Writing DB stats keystone:SELECT=100 {{(pid=81767) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 15:46:47.566334 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fd59afb-eac8-4797-b883-e7e48904fc41 None admin] Validating token access rules against request {{(pid=81767) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 15:46:47.606437 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cce75dc-dddf-4774-8afd-24c2a50eee1a None None] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 15:46:47.649625 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aabbcf7-9453-412c-96da-46aa824b65c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 15:46:47.650327 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aabbcf7-9453-412c-96da-46aa824b65c6 None admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:46:47.650713 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aabbcf7-9453-412c-96da-46aa824b65c6 None admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:46:47.650989 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aabbcf7-9453-412c-96da-46aa824b65c6 None admin] PATH_INFO: `/v3/roles` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:46:47.651799 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aabbcf7-9453-412c-96da-46aa824b65c6 None admin] RBAC: Adding query filter params (name=member) {{(pid=81768) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 15:46:47.652683 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aabbcf7-9453-412c-96da-46aa824b65c6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 15:46:47.654314 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aabbcf7-9453-412c-96da-46aa824b65c6 None admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 15:46:47.664929 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 131/253] 10.222.0.15 () {56 vars in 1101 bytes} [Mon Aug 22 15:46:47 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 15:46:47.666171 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2c3c104-505c-4dd9-9171-c4480cfe2ae5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81767) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 15:46:47.666302 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75f17386-26ef-45f7-a18a-3edf8deb8808 None None] REQUEST_METHOD: `POST` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:46:47.666650 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75f17386-26ef-45f7-a18a-3edf8deb8808 None None] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:46:47.666927 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75f17386-26ef-45f7-a18a-3edf8deb8808 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:46:47.667036 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c3c104-505c-4dd9-9171-c4480cfe2ae5 None admin] REQUEST_METHOD: `GET` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:46:47.667197 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c3c104-505c-4dd9-9171-c4480cfe2ae5 None admin] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:46:47.667548 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c3c104-505c-4dd9-9171-c4480cfe2ae5 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:46:47.688544 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.auth.core [None req-75f17386-26ef-45f7-a18a-3edf8deb8808 None None] MFA Rules not processed for user `6b2199ae0fb0480aa917ada391f5ec54`. Rule list: `[]` (Enabled: `True`). {{(pid=81768) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 15:46:47.689847 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-75f17386-26ef-45f7-a18a-3edf8deb8808 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 15:46:47.694737 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2c3c104-505c-4dd9-9171-c4480cfe2ae5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 22 15:46:47.700319 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application [None req-e2c3c104-505c-4dd9-9171-c4480cfe2ae5 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 22 15:46:47.700319 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 15:46:47.700319 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 15:46:47.700319 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 15:46:47.700319 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 15:46:47.700319 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 15:46:47.700319 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 15:46:47.700319 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 15:46:47.700319 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 15:46:47.700319 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 15:46:47.700319 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 15:46:47.700319 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 15:46:47.700319 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 22 15:46:47.700319 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 22 15:46:47.700319 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 22 15:46:47.700319 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 22 15:46:47.702589 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 22 15:46:47.702589 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 22 15:46:47.702589 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 22 15:46:47.702589 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 22 15:46:47.702589 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 15:46:47.702589 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 15:46:47.702589 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 15:46:47.702589 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 15:46:47.702589 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 15:46:47.702589 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 15:46:47.702589 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 15:46:47.702589 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application with Lock( Aug 22 15:46:47.702589 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 15:46:47.702589 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return self._enter() Aug 22 15:46:47.702589 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 15:46:47.702589 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 15:46:47.704608 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 15:46:47.704608 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return self.creator() Aug 22 15:46:47.704608 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 15:46:47.704608 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application created_value = creator( Aug 22 15:46:47.704608 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 22 15:46:47.704608 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 22 15:46:47.704608 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 22 15:46:47.704608 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 22 15:46:47.704608 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 22 15:46:47.704608 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 22 15:46:47.704608 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 22 15:46:47.704608 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application Aug 22 15:46:47.704608 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 123/254] 10.222.0.15 () {56 vars in 1096 bytes} [Mon Aug 22 15:46:47 2022] GET /identity/v3/roles/admin => generated 83 bytes in 322 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 15:46:47.704608 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2c3c104-505c-4dd9-9171-c4480cfe2ae5 None admin] Authenticating user token {{(pid=81767) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 15:46:47.723678 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=100 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 15:46:47.831426 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2c3c104-505c-4dd9-9171-c4480cfe2ae5 None admin] Validating token access rules against request {{(pid=81767) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 15:46:47.841766 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 132/255] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Aug 22 15:46:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 22 15:46:47.842716 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75f17386-26ef-45f7-a18a-3edf8deb8808 None None] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 15:46:47.870826 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-194b6277-13a2-4c24-b8b7-669243f731d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81767) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 15:46:47.872256 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-194b6277-13a2-4c24-b8b7-669243f731d5 None admin] REQUEST_METHOD: `GET` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:46:47.872531 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-194b6277-13a2-4c24-b8b7-669243f731d5 None admin] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:46:47.872961 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-194b6277-13a2-4c24-b8b7-669243f731d5 None admin] PATH_INFO: `/v3/users/038404f0743144d7bbc039278710e57e` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:46:47.874645 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-194b6277-13a2-4c24-b8b7-669243f731d5 None admin] RBAC: Authorizing `identity:get_user(user_id=038404f0743144d7bbc039278710e57e)` {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 15:46:47.876280 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-194b6277-13a2-4c24-b8b7-669243f731d5 None admin] RBAC: Authorization granted {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 15:46:47.878819 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 124/256] 10.222.0.15 () {56 vars in 1177 bytes} [Mon Aug 22 15:46:47 2022] GET /identity/v3/users/038404f0743144d7bbc039278710e57e => generated 297 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 15:46:47.885551 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-194b6277-13a2-4c24-b8b7-669243f731d5 None admin] Authenticating user token {{(pid=81767) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 15:46:47.895627 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG dbcounter [-] [81767] Writing DB stats keystone:SELECT=100 {{(pid=81767) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 15:46:48.062209 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-194b6277-13a2-4c24-b8b7-669243f731d5 None admin] Validating token access rules against request {{(pid=81767) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 15:46:48.132221 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6446855b-4826-4770-b735-ddede1551537 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81767) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 15:46:48.132783 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6446855b-4826-4770-b735-ddede1551537 None admin] REQUEST_METHOD: `GET` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:46:48.133060 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6446855b-4826-4770-b735-ddede1551537 None admin] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:46:48.133388 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6446855b-4826-4770-b735-ddede1551537 None admin] PATH_INFO: `/v3/projects/c05dae9fd0dc44f9a2ab2d373bbc913f` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:46:48.134607 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6446855b-4826-4770-b735-ddede1551537 None admin] RBAC: Authorizing `identity:get_project(project_id=c05dae9fd0dc44f9a2ab2d373bbc913f)` {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 15:46:48.136072 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6446855b-4826-4770-b735-ddede1551537 None admin] RBAC: Authorization granted {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 15:46:48.139211 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 125/257] 10.222.0.15 () {56 vars in 1186 bytes} [Mon Aug 22 15:46:47 2022] GET /identity/v3/projects/c05dae9fd0dc44f9a2ab2d373bbc913f => generated 302 bytes in 254 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 15:46:48.141130 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75f17386-26ef-45f7-a18a-3edf8deb8808 None None] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 15:46:48.156587 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6446855b-4826-4770-b735-ddede1551537 None admin] Authenticating user token {{(pid=81767) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 15:46:48.248094 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3213a786-84bc-457e-8d72-fe8df67633e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 15:46:48.248758 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3213a786-84bc-457e-8d72-fe8df67633e1 None admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:46:48.249042 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3213a786-84bc-457e-8d72-fe8df67633e1 None admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:46:48.249319 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3213a786-84bc-457e-8d72-fe8df67633e1 None admin] PATH_INFO: `/v3/roles` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:46:48.250010 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3213a786-84bc-457e-8d72-fe8df67633e1 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81768) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 15:46:48.250319 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3213a786-84bc-457e-8d72-fe8df67633e1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 15:46:48.251779 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3213a786-84bc-457e-8d72-fe8df67633e1 None admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 15:46:48.265540 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 133/258] 10.222.0.15 () {56 vars in 1099 bytes} [Mon Aug 22 15:46:47 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 423 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 15:46:48.265992 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG dbcounter [-] [81767] Writing DB stats keystone:SELECT=100 {{(pid=81767) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 15:46:48.277299 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3213a786-84bc-457e-8d72-fe8df67633e1 None admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 15:46:48.315672 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=100 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 15:46:48.353228 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6446855b-4826-4770-b735-ddede1551537 None admin] Validating token access rules against request {{(pid=81767) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 15:46:48.422165 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb6f0867-6f89-4ba6-bc7c-74816652652c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81767) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 15:46:48.422756 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb6f0867-6f89-4ba6-bc7c-74816652652c None admin] REQUEST_METHOD: `PUT` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:46:48.423036 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb6f0867-6f89-4ba6-bc7c-74816652652c None admin] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:46:48.423335 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb6f0867-6f89-4ba6-bc7c-74816652652c None admin] PATH_INFO: `/v3/projects/c05dae9fd0dc44f9a2ab2d373bbc913f/users/038404f0743144d7bbc039278710e57e/roles/e6fc1ee3fb974125ba9b4fe55b12b45e` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:46:48.426734 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb6f0867-6f89-4ba6-bc7c-74816652652c None admin] RBAC: Authorizing `identity:create_grant(project_id=c05dae9fd0dc44f9a2ab2d373bbc913f, user_id=038404f0743144d7bbc039278710e57e, role_id=e6fc1ee3fb974125ba9b4fe55b12b45e)` {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 15:46:48.428267 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb6f0867-6f89-4ba6-bc7c-74816652652c None admin] RBAC: Authorization granted {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 15:46:48.463465 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3213a786-84bc-457e-8d72-fe8df67633e1 None admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 15:46:48.502162 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db3ce19c-d1f6-4d3b-a1ea-222d8262da8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 15:46:48.502807 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db3ce19c-d1f6-4d3b-a1ea-222d8262da8e None admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:46:48.503087 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db3ce19c-d1f6-4d3b-a1ea-222d8262da8e None admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:46:48.503385 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db3ce19c-d1f6-4d3b-a1ea-222d8262da8e None admin] PATH_INFO: `/v3/users/bfcf841e2f9e42fbb981ad7584d890cb` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:46:48.504564 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db3ce19c-d1f6-4d3b-a1ea-222d8262da8e None admin] RBAC: Authorizing `identity:get_user(user_id=bfcf841e2f9e42fbb981ad7584d890cb)` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 15:46:48.506030 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db3ce19c-d1f6-4d3b-a1ea-222d8262da8e None admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 15:46:48.508307 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 134/259] 10.222.0.15 () {56 vars in 1177 bytes} [Mon Aug 22 15:46:48 2022] GET /identity/v3/users/bfcf841e2f9e42fbb981ad7584d890cb => generated 283 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 15:46:48.509245 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db3ce19c-d1f6-4d3b-a1ea-222d8262da8e None admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 15:46:48.531488 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 126/260] 10.222.0.15 () {58 vars in 1439 bytes} [Mon Aug 22 15:46:48 2022] PUT /identity/v3/projects/c05dae9fd0dc44f9a2ab2d373bbc913f/users/038404f0743144d7bbc039278710e57e/roles/e6fc1ee3fb974125ba9b4fe55b12b45e => generated 0 bytes in 370 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 22 15:46:48.532555 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb6f0867-6f89-4ba6-bc7c-74816652652c None admin] Authenticating user token {{(pid=81767) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 15:46:48.682151 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb6f0867-6f89-4ba6-bc7c-74816652652c None admin] Validating token access rules against request {{(pid=81767) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 15:46:48.723210 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-009412fc-8782-4fed-a3ce-7236624ba5d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81767) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 15:46:48.724390 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-009412fc-8782-4fed-a3ce-7236624ba5d7 None admin] REQUEST_METHOD: `GET` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:46:48.724868 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-009412fc-8782-4fed-a3ce-7236624ba5d7 None admin] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:46:48.725350 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-009412fc-8782-4fed-a3ce-7236624ba5d7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:46:48.731547 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG dbcounter [-] [81767] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81767) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 15:46:48.788153 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=100 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 15:46:48.829682 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db3ce19c-d1f6-4d3b-a1ea-222d8262da8e None admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 15:46:48.858520 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-009412fc-8782-4fed-a3ce-7236624ba5d7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 15:46:48.860753 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-009412fc-8782-4fed-a3ce-7236624ba5d7 None admin] RBAC: Authorization granted {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 15:46:48.916025 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c24a895-b4b5-4ca2-9b5f-e4f45dde1e9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 15:46:48.916918 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c24a895-b4b5-4ca2-9b5f-e4f45dde1e9e None admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:46:48.916918 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c24a895-b4b5-4ca2-9b5f-e4f45dde1e9e None admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:46:48.917209 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c24a895-b4b5-4ca2-9b5f-e4f45dde1e9e None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:46:48.952110 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c24a895-b4b5-4ca2-9b5f-e4f45dde1e9e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 22 15:46:48.957464 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application [None req-1c24a895-b4b5-4ca2-9b5f-e4f45dde1e9e None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 22 15:46:48.957464 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 15:46:48.957464 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 15:46:48.957464 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 15:46:48.957464 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 15:46:48.957464 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 15:46:48.957464 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 15:46:48.957464 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 15:46:48.957464 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 15:46:48.957464 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 15:46:48.957464 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 15:46:48.957464 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 15:46:48.957464 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 22 15:46:48.957464 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 22 15:46:48.957464 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 22 15:46:48.957464 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 22 15:46:48.958328 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 22 15:46:48.958328 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 22 15:46:48.958328 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 22 15:46:48.958328 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 22 15:46:48.958328 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 15:46:48.958328 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 15:46:48.958328 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 15:46:48.958328 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 15:46:48.958328 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 15:46:48.958328 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 15:46:48.958328 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 15:46:48.958328 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application with Lock( Aug 22 15:46:48.958328 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 15:46:48.958328 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self._enter() Aug 22 15:46:48.958328 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 15:46:48.958328 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 15:46:48.959106 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 15:46:48.959106 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self.creator() Aug 22 15:46:48.959106 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 15:46:48.959106 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application created_value = creator( Aug 22 15:46:48.959106 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 22 15:46:48.959106 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 22 15:46:48.959106 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 22 15:46:48.959106 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 22 15:46:48.959106 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 22 15:46:48.959106 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 22 15:46:48.959106 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 22 15:46:48.959106 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application Aug 22 15:46:48.959106 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 135/261] 10.222.0.15 () {56 vars in 1114 bytes} [Mon Aug 22 15:46:48 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 450 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 15:46:48.968327 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c24a895-b4b5-4ca2-9b5f-e4f45dde1e9e None admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 15:46:48.984248 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 127/262] 10.222.0.15 () {58 vars in 1282 bytes} [Mon Aug 22 15:46:48 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 452 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 22 15:46:48.990135 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-009412fc-8782-4fed-a3ce-7236624ba5d7 None admin] Authenticating user token {{(pid=81767) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 15:46:49.051696 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG dbcounter [-] [81767] Writing DB stats keystone:SELECT=100 {{(pid=81767) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 15:46:49.089853 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c24a895-b4b5-4ca2-9b5f-e4f45dde1e9e None admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 15:46:49.094316 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-009412fc-8782-4fed-a3ce-7236624ba5d7 None admin] Validating token access rules against request {{(pid=81767) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 15:46:49.127875 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b66207b-f8be-43b9-a9de-90bf1fa52105 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 15:46:49.128491 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b66207b-f8be-43b9-a9de-90bf1fa52105 None admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:46:49.128773 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b66207b-f8be-43b9-a9de-90bf1fa52105 None admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:46:49.129043 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b66207b-f8be-43b9-a9de-90bf1fa52105 None admin] PATH_INFO: `/v3/roles` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:46:49.129838 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b66207b-f8be-43b9-a9de-90bf1fa52105 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81768) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 15:46:49.130194 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b66207b-f8be-43b9-a9de-90bf1fa52105 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 15:46:49.131903 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b66207b-f8be-43b9-a9de-90bf1fa52105 None admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 15:46:49.133198 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c861bff9-d20b-403c-a634-f04564e6cba0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81767) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 15:46:49.134083 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c861bff9-d20b-403c-a634-f04564e6cba0 None admin] REQUEST_METHOD: `GET` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:46:49.134566 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c861bff9-d20b-403c-a634-f04564e6cba0 None admin] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:46:49.135028 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c861bff9-d20b-403c-a634-f04564e6cba0 None admin] PATH_INFO: `/v3/projects/c05dae9fd0dc44f9a2ab2d373bbc913f` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:46:49.140521 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c861bff9-d20b-403c-a634-f04564e6cba0 None admin] RBAC: Authorizing `identity:get_project(project_id=c05dae9fd0dc44f9a2ab2d373bbc913f)` {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 15:46:49.141196 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 136/263] 10.222.0.15 () {56 vars in 1111 bytes} [Mon Aug 22 15:46:48 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 15:46:49.142371 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c861bff9-d20b-403c-a634-f04564e6cba0 None admin] RBAC: Authorization granted {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 15:46:49.146636 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b66207b-f8be-43b9-a9de-90bf1fa52105 None admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 15:46:49.148881 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 128/264] 10.222.0.15 () {56 vars in 1186 bytes} [Mon Aug 22 15:46:48 2022] GET /identity/v3/projects/c05dae9fd0dc44f9a2ab2d373bbc913f => generated 302 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 15:46:49.149866 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=100 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 15:46:49.154779 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c861bff9-d20b-403c-a634-f04564e6cba0 None admin] Authenticating user token {{(pid=81767) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 15:46:49.260964 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b66207b-f8be-43b9-a9de-90bf1fa52105 None admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 15:46:49.310601 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ca6f93f-f261-4a8a-8bac-4c9efa5b78e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 15:46:49.311241 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ca6f93f-f261-4a8a-8bac-4c9efa5b78e1 None admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:46:49.311547 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ca6f93f-f261-4a8a-8bac-4c9efa5b78e1 None admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:46:49.311813 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ca6f93f-f261-4a8a-8bac-4c9efa5b78e1 None admin] PATH_INFO: `/v3/users/bfcf841e2f9e42fbb981ad7584d890cb` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:46:49.314201 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ca6f93f-f261-4a8a-8bac-4c9efa5b78e1 None admin] RBAC: Authorizing `identity:get_user(user_id=bfcf841e2f9e42fbb981ad7584d890cb)` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 15:46:49.315751 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ca6f93f-f261-4a8a-8bac-4c9efa5b78e1 None admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 15:46:49.318572 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 137/265] 10.222.0.15 () {56 vars in 1177 bytes} [Mon Aug 22 15:46:49 2022] GET /identity/v3/users/bfcf841e2f9e42fbb981ad7584d890cb => generated 283 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 15:46:49.325522 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ca6f93f-f261-4a8a-8bac-4c9efa5b78e1 None admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 15:46:49.331090 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG dbcounter [-] [81767] Writing DB stats keystone:SELECT=100 {{(pid=81767) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 15:46:49.341967 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c861bff9-d20b-403c-a634-f04564e6cba0 None admin] Validating token access rules against request {{(pid=81767) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 15:46:49.428342 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f70ab1c-7954-4bde-9118-da5f71f806a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81767) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 15:46:49.428986 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f70ab1c-7954-4bde-9118-da5f71f806a4 None admin] REQUEST_METHOD: `GET` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:46:49.429202 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f70ab1c-7954-4bde-9118-da5f71f806a4 None admin] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:46:49.429632 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f70ab1c-7954-4bde-9118-da5f71f806a4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:46:49.430498 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f70ab1c-7954-4bde-9118-da5f71f806a4 None admin] RBAC: Adding query filter params (role.id=9d9848cfd22e4b089db443c350e56670, scope.project.id=c05dae9fd0dc44f9a2ab2d373bbc913f, user.id=bfcf841e2f9e42fbb981ad7584d890cb) {{(pid=81767) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 15:46:49.432038 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f70ab1c-7954-4bde-9118-da5f71f806a4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 15:46:49.435579 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f70ab1c-7954-4bde-9118-da5f71f806a4 None admin] RBAC: Authorization granted {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 15:46:49.447765 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 129/266] 10.222.0.15 () {56 vars in 1374 bytes} [Mon Aug 22 15:46:49 2022] GET /identity/v3/role_assignments?user.id=bfcf841e2f9e42fbb981ad7584d890cb&scope.project.id=c05dae9fd0dc44f9a2ab2d373bbc913f&role.id=9d9848cfd22e4b089db443c350e56670 => generated 608 bytes in 293 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 22 15:46:49.489031 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=100 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 15:46:49.520108 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ca6f93f-f261-4a8a-8bac-4c9efa5b78e1 None admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 15:46:49.572287 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d4b9a55-576e-4829-a018-83d526d48d83 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 15:46:49.573156 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d4b9a55-576e-4829-a018-83d526d48d83 None admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:46:49.573573 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d4b9a55-576e-4829-a018-83d526d48d83 None admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:46:49.573980 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d4b9a55-576e-4829-a018-83d526d48d83 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:46:49.658877 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d4b9a55-576e-4829-a018-83d526d48d83 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 15:46:49.660388 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d4b9a55-576e-4829-a018-83d526d48d83 None admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 15:46:49.738496 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=100 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 15:46:49.774603 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 138/267] 10.222.0.15 () {58 vars in 1282 bytes} [Mon Aug 22 15:46:49 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 451 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 22 15:46:49.780407 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f70ab1c-7954-4bde-9118-da5f71f806a4 None admin] Authenticating user token {{(pid=81767) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 15:46:49.793717 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-010b38df-6d57-4b5e-901a-e4f046c049c4 None None] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:46:49.794021 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-010b38df-6d57-4b5e-901a-e4f046c049c4 None None] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:46:49.794264 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-010b38df-6d57-4b5e-901a-e4f046c049c4 None None] PATH_INFO: `/` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:46:49.796454 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 139/268] 10.222.0.15 () {54 vars in 927 bytes} [Mon Aug 22 15:46:49 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 15:46:49.802783 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a482f70-d27c-4b44-a74c-920962f57554 None None] REQUEST_METHOD: `POST` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:46:49.803061 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a482f70-d27c-4b44-a74c-920962f57554 None None] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:46:49.803364 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a482f70-d27c-4b44-a74c-920962f57554 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:46:49.827880 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.auth.core [None req-1a482f70-d27c-4b44-a74c-920962f57554 None None] MFA Rules not processed for user `6b2199ae0fb0480aa917ada391f5ec54`. Rule list: `[]` (Enabled: `True`). {{(pid=81768) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 15:46:49.829414 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-1a482f70-d27c-4b44-a74c-920962f57554 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 15:46:49.904035 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 140/269] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Aug 22 15:46:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 22 15:46:49.957857 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f70ab1c-7954-4bde-9118-da5f71f806a4 None admin] Validating token access rules against request {{(pid=81767) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 15:46:50.010967 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d2b4679-060f-45f6-bb49-1e47848e6ee0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81767) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 15:46:50.011622 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d2b4679-060f-45f6-bb49-1e47848e6ee0 None admin] REQUEST_METHOD: `GET` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:46:50.012531 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d2b4679-060f-45f6-bb49-1e47848e6ee0 None admin] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:46:50.012803 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d2b4679-060f-45f6-bb49-1e47848e6ee0 None admin] PATH_INFO: `/v3/projects/c05dae9fd0dc44f9a2ab2d373bbc913f` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:46:50.013945 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d2b4679-060f-45f6-bb49-1e47848e6ee0 None admin] RBAC: Authorizing `identity:get_project(project_id=c05dae9fd0dc44f9a2ab2d373bbc913f)` {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 15:46:50.015406 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d2b4679-060f-45f6-bb49-1e47848e6ee0 None admin] RBAC: Authorization granted {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 15:46:50.017356 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 130/270] 10.222.0.15 () {56 vars in 1186 bytes} [Mon Aug 22 15:46:49 2022] GET /identity/v3/projects/c05dae9fd0dc44f9a2ab2d373bbc913f => generated 302 bytes in 237 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 22 15:46:50.023183 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a482f70-d27c-4b44-a74c-920962f57554 None None] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 15:46:50.119143 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94cf27d2-bd5e-45fc-9cc0-076e0d5b4d34 None None] REQUEST_METHOD: `GET` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:46:50.119804 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94cf27d2-bd5e-45fc-9cc0-076e0d5b4d34 None None] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:46:50.120266 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94cf27d2-bd5e-45fc-9cc0-076e0d5b4d34 None None] PATH_INFO: `/` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:46:50.122372 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 131/271] 10.222.0.15 () {54 vars in 927 bytes} [Mon Aug 22 15:46:50 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 22 15:46:50.133675 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd3bb119-418b-4757-9464-3b97a2ac71ed None None] REQUEST_METHOD: `POST` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:46:50.133957 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd3bb119-418b-4757-9464-3b97a2ac71ed None None] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:46:50.134234 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd3bb119-418b-4757-9464-3b97a2ac71ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:46:50.138002 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a482f70-d27c-4b44-a74c-920962f57554 None None] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 15:46:50.139598 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=100 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 15:46:50.165369 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.auth.core [None req-bd3bb119-418b-4757-9464-3b97a2ac71ed None None] MFA Rules not processed for user `6b2199ae0fb0480aa917ada391f5ec54`. Rule list: `[]` (Enabled: `True`). {{(pid=81767) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 15:46:50.166786 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.fernet_utils [None req-bd3bb119-418b-4757-9464-3b97a2ac71ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81767) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 15:46:50.178136 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd3da8cb-daa9-42aa-b808-20baada8afc0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 15:46:50.178743 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd3da8cb-daa9-42aa-b808-20baada8afc0 None admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:46:50.179019 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd3da8cb-daa9-42aa-b808-20baada8afc0 None admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:46:50.179324 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd3da8cb-daa9-42aa-b808-20baada8afc0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:46:50.180047 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd3da8cb-daa9-42aa-b808-20baada8afc0 None admin] RBAC: Adding query filter params (role.id=1ce6b924c9604048906c07dbfb126da9, scope.project.id=c05dae9fd0dc44f9a2ab2d373bbc913f, user.id=bfcf841e2f9e42fbb981ad7584d890cb) {{(pid=81768) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 15:46:50.180383 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd3da8cb-daa9-42aa-b808-20baada8afc0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 15:46:50.182129 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd3da8cb-daa9-42aa-b808-20baada8afc0 None admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 15:46:50.195689 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 141/272] 10.222.0.15 () {56 vars in 1374 bytes} [Mon Aug 22 15:46:50 2022] GET /identity/v3/role_assignments?user.id=bfcf841e2f9e42fbb981ad7584d890cb&scope.project.id=c05dae9fd0dc44f9a2ab2d373bbc913f&role.id=1ce6b924c9604048906c07dbfb126da9 => generated 608 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 15:46:50.195886 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG dbcounter [-] [81767] Writing DB stats keystone:SELECT=100 {{(pid=81767) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 15:46:50.196952 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e113d56-2128-439c-8df7-1f471d413bcd None None] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:46:50.197232 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e113d56-2128-439c-8df7-1f471d413bcd None None] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:46:50.197554 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e113d56-2128-439c-8df7-1f471d413bcd None None] PATH_INFO: `/` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:46:50.198998 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 142/273] 10.222.0.15 () {54 vars in 927 bytes} [Mon Aug 22 15:46:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 15:46:50.206293 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85952312-70a9-4467-8ba9-29e6e91554e1 None None] REQUEST_METHOD: `POST` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:46:50.206567 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85952312-70a9-4467-8ba9-29e6e91554e1 None None] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:46:50.206886 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85952312-70a9-4467-8ba9-29e6e91554e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:46:50.239724 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.auth.core [None req-85952312-70a9-4467-8ba9-29e6e91554e1 None None] MFA Rules not processed for user `6b2199ae0fb0480aa917ada391f5ec54`. Rule list: `[]` (Enabled: `True`). {{(pid=81768) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 15:46:50.242825 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-85952312-70a9-4467-8ba9-29e6e91554e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 15:46:50.267416 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 132/274] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Aug 22 15:46:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 22 15:46:50.305459 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd3bb119-418b-4757-9464-3b97a2ac71ed None None] Authenticating user token {{(pid=81767) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 15:46:50.392855 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 143/275] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Aug 22 15:46:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 22 15:46:50.449233 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd3bb119-418b-4757-9464-3b97a2ac71ed None None] Validating token access rules against request {{(pid=81767) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 15:46:50.484212 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-908ae5e5-9b65-4094-b4c6-0adc6ca2c780 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81767) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 15:46:50.484837 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-908ae5e5-9b65-4094-b4c6-0adc6ca2c780 None admin] REQUEST_METHOD: `GET` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:46:50.485112 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-908ae5e5-9b65-4094-b4c6-0adc6ca2c780 None admin] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:46:50.485381 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-908ae5e5-9b65-4094-b4c6-0adc6ca2c780 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:46:50.486486 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-908ae5e5-9b65-4094-b4c6-0adc6ca2c780 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 15:46:50.488047 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-908ae5e5-9b65-4094-b4c6-0adc6ca2c780 None admin] RBAC: Authorization granted {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 15:46:50.489615 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 133/276] 10.222.0.15 () {56 vars in 1108 bytes} [Mon Aug 22 15:46:50 2022] GET /identity/v3/domains/default => generated 199 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 15:46:50.495150 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85952312-70a9-4467-8ba9-29e6e91554e1 None None] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 15:46:50.584938 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-908ae5e5-9b65-4094-b4c6-0adc6ca2c780 None admin] Authenticating user token {{(pid=81767) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 15:46:50.598051 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=100 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 15:46:50.605078 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85952312-70a9-4467-8ba9-29e6e91554e1 None None] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 15:46:50.627672 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG dbcounter [-] [81767] Writing DB stats keystone:SELECT=100 {{(pid=81767) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 15:46:50.646964 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78d49ea3-da99-4ab8-910f-8191816be445 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 15:46:50.647533 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78d49ea3-da99-4ab8-910f-8191816be445 None admin] REQUEST_METHOD: `POST` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:46:50.648073 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78d49ea3-da99-4ab8-910f-8191816be445 None admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:46:50.648532 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78d49ea3-da99-4ab8-910f-8191816be445 None admin] PATH_INFO: `/v3/users` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:46:50.649391 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78d49ea3-da99-4ab8-910f-8191816be445 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 15:46:50.651216 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78d49ea3-da99-4ab8-910f-8191816be445 None admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 15:46:50.715856 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-908ae5e5-9b65-4094-b4c6-0adc6ca2c780 None admin] Validating token access rules against request {{(pid=81767) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 15:46:50.750117 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e08780a3-c750-4e9c-836c-d1c38d337e02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81767) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 15:46:50.750740 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e08780a3-c750-4e9c-836c-d1c38d337e02 None admin] REQUEST_METHOD: `GET` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:46:50.751011 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e08780a3-c750-4e9c-836c-d1c38d337e02 None admin] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:46:50.751301 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e08780a3-c750-4e9c-836c-d1c38d337e02 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:46:50.761115 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e08780a3-c750-4e9c-836c-d1c38d337e02 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 22 15:46:50.766356 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application [None req-e08780a3-c750-4e9c-836c-d1c38d337e02 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 22 15:46:50.766356 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 15:46:50.766356 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 15:46:50.766356 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 15:46:50.766356 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 15:46:50.766356 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 15:46:50.766356 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 15:46:50.766356 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 15:46:50.766356 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 15:46:50.766356 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 15:46:50.766356 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 15:46:50.766356 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 15:46:50.766356 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 22 15:46:50.766356 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 22 15:46:50.766356 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 22 15:46:50.766356 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 22 15:46:50.768714 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 22 15:46:50.768714 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 22 15:46:50.768714 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 22 15:46:50.768714 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 22 15:46:50.768714 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 15:46:50.768714 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 15:46:50.768714 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 15:46:50.768714 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 15:46:50.768714 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 15:46:50.768714 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 15:46:50.768714 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 15:46:50.768714 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application with Lock( Aug 22 15:46:50.768714 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 15:46:50.768714 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return self._enter() Aug 22 15:46:50.768714 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 15:46:50.768714 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 15:46:50.769505 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 15:46:50.769505 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return self.creator() Aug 22 15:46:50.769505 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 15:46:50.769505 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application created_value = creator( Aug 22 15:46:50.769505 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 22 15:46:50.769505 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 22 15:46:50.769505 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 22 15:46:50.769505 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 22 15:46:50.769505 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 22 15:46:50.769505 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 22 15:46:50.769505 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 22 15:46:50.769505 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application Aug 22 15:46:50.769505 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 134/277] 10.222.0.15 () {56 vars in 1099 bytes} [Mon Aug 22 15:46:50 2022] GET /identity/v3/roles/reader => generated 84 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 22 15:46:50.769505 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e08780a3-c750-4e9c-836c-d1c38d337e02 None admin] Authenticating user token {{(pid=81767) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 15:46:50.795046 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 144/278] 10.222.0.15 () {60 vars in 1132 bytes} [Mon Aug 22 15:46:50 2022] POST /identity/v3/users => generated 293 bytes in 300 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 22 15:46:50.795046 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78d49ea3-da99-4ab8-910f-8191816be445 None admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 15:46:50.904302 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG dbcounter [-] [81767] Writing DB stats keystone:SELECT=100 {{(pid=81767) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 15:46:50.923670 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e08780a3-c750-4e9c-836c-d1c38d337e02 None admin] Validating token access rules against request {{(pid=81767) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 15:46:50.924732 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78d49ea3-da99-4ab8-910f-8191816be445 None admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 15:46:50.973903 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb0508a1-6483-454f-abee-91e5c532d022 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81767) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 15:46:50.974554 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb0508a1-6483-454f-abee-91e5c532d022 None admin] REQUEST_METHOD: `GET` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:46:50.974827 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb0508a1-6483-454f-abee-91e5c532d022 None admin] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:46:50.975125 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb0508a1-6483-454f-abee-91e5c532d022 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:46:50.977070 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb477f59-4647-46de-a1dc-167ab58dc4bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 15:46:50.977813 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb477f59-4647-46de-a1dc-167ab58dc4bc None admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:46:50.977983 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb477f59-4647-46de-a1dc-167ab58dc4bc None admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:46:50.978224 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb477f59-4647-46de-a1dc-167ab58dc4bc None admin] PATH_INFO: `/v3/roles` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:46:50.979024 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb477f59-4647-46de-a1dc-167ab58dc4bc None admin] RBAC: Adding query filter params (name=reader) {{(pid=81768) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 15:46:50.979404 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb477f59-4647-46de-a1dc-167ab58dc4bc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 15:46:50.981390 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb477f59-4647-46de-a1dc-167ab58dc4bc None admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 15:46:50.989127 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb0508a1-6483-454f-abee-91e5c532d022 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 22 15:46:50.991325 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 145/279] 10.222.0.15 () {56 vars in 1101 bytes} [Mon Aug 22 15:46:50 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 15:46:50.998388 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application [None req-cb0508a1-6483-454f-abee-91e5c532d022 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 22 15:46:50.998388 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 15:46:50.998388 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 15:46:50.998388 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 15:46:50.998388 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 15:46:50.998388 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 15:46:50.998388 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 15:46:50.998388 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 15:46:50.998388 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 15:46:50.998388 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 15:46:50.998388 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 15:46:50.998388 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 15:46:50.998388 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 22 15:46:50.998388 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 22 15:46:50.998388 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 22 15:46:50.998388 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 22 15:46:51.000085 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 22 15:46:51.000085 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 22 15:46:51.000085 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 22 15:46:51.000085 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 22 15:46:51.000085 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 15:46:51.000085 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 15:46:51.000085 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 15:46:51.000085 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 15:46:51.000085 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 15:46:51.000085 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 15:46:51.000085 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 15:46:51.000085 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application with Lock( Aug 22 15:46:51.000085 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 15:46:51.000085 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return self._enter() Aug 22 15:46:51.000085 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 15:46:51.000085 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 15:46:51.001033 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 15:46:51.001033 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return self.creator() Aug 22 15:46:51.001033 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 15:46:51.001033 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application created_value = creator( Aug 22 15:46:51.001033 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 22 15:46:51.001033 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 22 15:46:51.001033 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 22 15:46:51.001033 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 22 15:46:51.001033 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 22 15:46:51.001033 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 22 15:46:51.001033 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 22 15:46:51.001033 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application Aug 22 15:46:51.001779 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb477f59-4647-46de-a1dc-167ab58dc4bc None admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 15:46:51.001859 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 135/280] 10.222.0.15 () {56 vars in 1099 bytes} [Mon Aug 22 15:46:50 2022] GET /identity/v3/roles/member => generated 84 bytes in 232 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 15:46:51.006665 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb0508a1-6483-454f-abee-91e5c532d022 None admin] Authenticating user token {{(pid=81767) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 15:46:51.021547 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 15:46:51.134944 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb477f59-4647-46de-a1dc-167ab58dc4bc None admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 15:46:51.154634 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb0508a1-6483-454f-abee-91e5c532d022 None admin] Validating token access rules against request {{(pid=81767) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 15:46:51.178626 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bad4dcc-f6cc-4556-9e44-ef0cba30080f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 15:46:51.179496 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bad4dcc-f6cc-4556-9e44-ef0cba30080f None admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:46:51.179870 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bad4dcc-f6cc-4556-9e44-ef0cba30080f None admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:46:51.180253 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bad4dcc-f6cc-4556-9e44-ef0cba30080f None admin] PATH_INFO: `/v3/users/f1f4430458e44ebcb8cfd11bf292c973` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:46:51.182997 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bad4dcc-f6cc-4556-9e44-ef0cba30080f None admin] RBAC: Authorizing `identity:get_user(user_id=f1f4430458e44ebcb8cfd11bf292c973)` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 15:46:51.184607 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bad4dcc-f6cc-4556-9e44-ef0cba30080f None admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 15:46:51.190540 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 146/281] 10.222.0.15 () {56 vars in 1177 bytes} [Mon Aug 22 15:46:50 2022] GET /identity/v3/users/f1f4430458e44ebcb8cfd11bf292c973 => generated 297 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 15:46:51.192066 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bad4dcc-f6cc-4556-9e44-ef0cba30080f None admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 15:46:51.202120 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e59bfb2d-eb45-4b90-b160-2bee8ab0ee55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81767) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 15:46:51.203295 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e59bfb2d-eb45-4b90-b160-2bee8ab0ee55 None admin] REQUEST_METHOD: `GET` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:46:51.203654 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e59bfb2d-eb45-4b90-b160-2bee8ab0ee55 None admin] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:46:51.204003 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e59bfb2d-eb45-4b90-b160-2bee8ab0ee55 None admin] PATH_INFO: `/v3/roles` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:46:51.205209 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e59bfb2d-eb45-4b90-b160-2bee8ab0ee55 None admin] RBAC: Adding query filter params (name=member) {{(pid=81767) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 15:46:51.205583 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e59bfb2d-eb45-4b90-b160-2bee8ab0ee55 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 15:46:51.207762 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e59bfb2d-eb45-4b90-b160-2bee8ab0ee55 None admin] RBAC: Authorization granted {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 15:46:51.218800 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 136/282] 10.222.0.15 () {56 vars in 1101 bytes} [Mon Aug 22 15:46:51 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 22 15:46:51.224736 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e59bfb2d-eb45-4b90-b160-2bee8ab0ee55 None admin] Authenticating user token {{(pid=81767) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 15:46:51.240778 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG dbcounter [-] [81767] Writing DB stats keystone:SELECT=100 {{(pid=81767) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 15:46:51.294003 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=100 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 15:46:51.320067 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bad4dcc-f6cc-4556-9e44-ef0cba30080f None admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 15:46:51.347860 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e59bfb2d-eb45-4b90-b160-2bee8ab0ee55 None admin] Validating token access rules against request {{(pid=81767) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 15:46:51.376569 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71c8606d-98be-4d5e-a195-8d7fd98d6cdd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 15:46:51.378122 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c8606d-98be-4d5e-a195-8d7fd98d6cdd None admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:46:51.378889 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c8606d-98be-4d5e-a195-8d7fd98d6cdd None admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:46:51.379397 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c8606d-98be-4d5e-a195-8d7fd98d6cdd None admin] PATH_INFO: `/v3/projects/c05dae9fd0dc44f9a2ab2d373bbc913f` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:46:51.381593 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71c8606d-98be-4d5e-a195-8d7fd98d6cdd None admin] RBAC: Authorizing `identity:get_project(project_id=c05dae9fd0dc44f9a2ab2d373bbc913f)` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 15:46:51.383744 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71c8606d-98be-4d5e-a195-8d7fd98d6cdd None admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 15:46:51.386284 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 147/283] 10.222.0.15 () {56 vars in 1186 bytes} [Mon Aug 22 15:46:51 2022] GET /identity/v3/projects/c05dae9fd0dc44f9a2ab2d373bbc913f => generated 302 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 22 15:46:51.392739 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb50054-7c98-4a4c-8395-33a043bb2241 None None] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:46:51.392871 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4594efc9-41ac-436b-a92b-93a10986974a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81767) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 15:46:51.392871 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4594efc9-41ac-436b-a92b-93a10986974a None admin] REQUEST_METHOD: `GET` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:46:51.392871 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4594efc9-41ac-436b-a92b-93a10986974a None admin] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:46:51.392871 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4594efc9-41ac-436b-a92b-93a10986974a None admin] PATH_INFO: `/v3/users/9efe902c9ad4404ebcbedea832133050` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:46:51.393781 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb50054-7c98-4a4c-8395-33a043bb2241 None None] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:46:51.395142 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb50054-7c98-4a4c-8395-33a043bb2241 None None] PATH_INFO: `/` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:46:51.396755 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 148/284] 10.222.0.15 () {54 vars in 927 bytes} [Mon Aug 22 15:46:51 2022] GET /identity => generated 270 bytes in 10 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 15:46:51.397660 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bb50054-7c98-4a4c-8395-33a043bb2241 None None] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 15:46:51.411520 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4594efc9-41ac-436b-a92b-93a10986974a None admin] RBAC: Authorizing `identity:get_user(user_id=9efe902c9ad4404ebcbedea832133050)` {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 15:46:51.412964 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4594efc9-41ac-436b-a92b-93a10986974a None admin] RBAC: Authorization granted {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 15:46:51.415100 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 137/285] 10.222.0.15 () {56 vars in 1177 bytes} [Mon Aug 22 15:46:51 2022] GET /identity/v3/users/9efe902c9ad4404ebcbedea832133050 => generated 293 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 15:46:51.416376 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55f22e23-aa39-4433-a7f7-dd8848a5521c None None] REQUEST_METHOD: `POST` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:46:51.416723 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55f22e23-aa39-4433-a7f7-dd8848a5521c None None] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:46:51.417062 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55f22e23-aa39-4433-a7f7-dd8848a5521c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:46:51.440089 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.auth.core [None req-55f22e23-aa39-4433-a7f7-dd8848a5521c None None] MFA Rules not processed for user `6b2199ae0fb0480aa917ada391f5ec54`. Rule list: `[]` (Enabled: `True`). {{(pid=81767) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 15:46:51.441455 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.fernet_utils [None req-55f22e23-aa39-4433-a7f7-dd8848a5521c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81767) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 15:46:51.529740 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bb50054-7c98-4a4c-8395-33a043bb2241 None None] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 15:46:51.530324 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG dbcounter [-] [81767] Writing DB stats keystone:SELECT=100 {{(pid=81767) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 15:46:51.535989 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 138/286] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Aug 22 15:46:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 22 15:46:51.544338 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55f22e23-aa39-4433-a7f7-dd8848a5521c None None] Authenticating user token {{(pid=81767) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 15:46:51.566965 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f869dd9c-a10a-4710-9682-5001968dc02c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 15:46:51.567672 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f869dd9c-a10a-4710-9682-5001968dc02c None admin] REQUEST_METHOD: `PUT` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:46:51.567948 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f869dd9c-a10a-4710-9682-5001968dc02c None admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:46:51.568212 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f869dd9c-a10a-4710-9682-5001968dc02c None admin] PATH_INFO: `/v3/projects/c05dae9fd0dc44f9a2ab2d373bbc913f/users/f1f4430458e44ebcb8cfd11bf292c973/roles/640209879fb24ac0ae3f65dd107697c3` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:46:51.570186 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f869dd9c-a10a-4710-9682-5001968dc02c None admin] RBAC: Authorizing `identity:create_grant(project_id=c05dae9fd0dc44f9a2ab2d373bbc913f, user_id=f1f4430458e44ebcb8cfd11bf292c973, role_id=640209879fb24ac0ae3f65dd107697c3)` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 15:46:51.571710 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f869dd9c-a10a-4710-9682-5001968dc02c None admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 15:46:51.645870 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 149/287] 10.222.0.15 () {58 vars in 1439 bytes} [Mon Aug 22 15:46:51 2022] PUT /identity/v3/projects/c05dae9fd0dc44f9a2ab2d373bbc913f/users/f1f4430458e44ebcb8cfd11bf292c973/roles/640209879fb24ac0ae3f65dd107697c3 => generated 0 bytes in 248 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 15:46:51.679834 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55f22e23-aa39-4433-a7f7-dd8848a5521c None None] Validating token access rules against request {{(pid=81767) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 15:46:51.734602 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4376727-9849-426b-9e9f-df986d394a73 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81767) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 15:46:51.735289 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4376727-9849-426b-9e9f-df986d394a73 None admin] REQUEST_METHOD: `GET` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:46:51.735583 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4376727-9849-426b-9e9f-df986d394a73 None admin] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:46:51.735965 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4376727-9849-426b-9e9f-df986d394a73 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:46:51.736776 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4376727-9849-426b-9e9f-df986d394a73 None admin] RBAC: Adding query filter params (role.id=e6fc1ee3fb974125ba9b4fe55b12b45e, user.id=9efe902c9ad4404ebcbedea832133050, scope.system=all) {{(pid=81767) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 15:46:51.737075 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4376727-9849-426b-9e9f-df986d394a73 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 15:46:51.738542 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4376727-9849-426b-9e9f-df986d394a73 None admin] RBAC: Authorization granted {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 15:46:51.746831 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 139/288] 10.222.0.15 () {56 vars in 1308 bytes} [Mon Aug 22 15:46:51 2022] GET /identity/v3/role_assignments?user.id=9efe902c9ad4404ebcbedea832133050&scope.system=all&role.id=e6fc1ee3fb974125ba9b4fe55b12b45e => generated 226 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 15:46:51.845179 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f869dd9c-a10a-4710-9682-5001968dc02c None admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 15:46:51.858565 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 15:46:51.961013 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f869dd9c-a10a-4710-9682-5001968dc02c None admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 15:46:52.000998 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf6e04b5-4795-4521-89ee-8b6425f67e64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 15:46:52.001638 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf6e04b5-4795-4521-89ee-8b6425f67e64 None admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:46:52.001936 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf6e04b5-4795-4521-89ee-8b6425f67e64 None admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:46:52.002260 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf6e04b5-4795-4521-89ee-8b6425f67e64 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:46:52.017408 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf6e04b5-4795-4521-89ee-8b6425f67e64 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 22 15:46:52.022850 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application [None req-cf6e04b5-4795-4521-89ee-8b6425f67e64 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 22 15:46:52.022850 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 15:46:52.022850 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 15:46:52.022850 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 15:46:52.022850 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 15:46:52.022850 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 15:46:52.022850 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 15:46:52.022850 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 15:46:52.022850 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 15:46:52.022850 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 15:46:52.022850 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 15:46:52.022850 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 15:46:52.022850 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 22 15:46:52.022850 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 22 15:46:52.022850 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 22 15:46:52.022850 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 22 15:46:52.023806 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 22 15:46:52.023806 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 22 15:46:52.023806 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 22 15:46:52.023806 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 22 15:46:52.023806 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 15:46:52.023806 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 15:46:52.023806 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 15:46:52.023806 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 15:46:52.023806 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 15:46:52.023806 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 15:46:52.023806 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 15:46:52.023806 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application with Lock( Aug 22 15:46:52.023806 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 15:46:52.023806 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self._enter() Aug 22 15:46:52.023806 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 15:46:52.023806 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 15:46:52.025035 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 15:46:52.025035 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self.creator() Aug 22 15:46:52.025035 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 15:46:52.025035 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application created_value = creator( Aug 22 15:46:52.025035 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 22 15:46:52.025035 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 22 15:46:52.025035 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 22 15:46:52.025035 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 22 15:46:52.025035 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 22 15:46:52.025035 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 22 15:46:52.025035 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 22 15:46:52.025035 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application Aug 22 15:46:52.025035 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 150/289] 10.222.0.15 () {56 vars in 1099 bytes} [Mon Aug 22 15:46:51 2022] GET /identity/v3/roles/member => generated 84 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 15:46:52.032117 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4376727-9849-426b-9e9f-df986d394a73 None admin] Authenticating user token {{(pid=81767) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 15:46:52.108880 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG dbcounter [-] [81767] Writing DB stats keystone:SELECT=100 {{(pid=81767) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 15:46:52.157565 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4376727-9849-426b-9e9f-df986d394a73 None admin] Validating token access rules against request {{(pid=81767) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 15:46:52.213078 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85da3c84-f153-4ba9-af6a-e76a098361a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81767) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 15:46:52.213612 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85da3c84-f153-4ba9-af6a-e76a098361a5 None admin] REQUEST_METHOD: `GET` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:46:52.213886 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85da3c84-f153-4ba9-af6a-e76a098361a5 None admin] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:46:52.214153 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85da3c84-f153-4ba9-af6a-e76a098361a5 None admin] PATH_INFO: `/v3/roles` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:46:52.214841 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85da3c84-f153-4ba9-af6a-e76a098361a5 None admin] RBAC: Adding query filter params (name=member) {{(pid=81767) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 15:46:52.215140 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85da3c84-f153-4ba9-af6a-e76a098361a5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 15:46:52.216604 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85da3c84-f153-4ba9-af6a-e76a098361a5 None admin] RBAC: Authorization granted {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 15:46:52.232680 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 140/290] 10.222.0.15 () {56 vars in 1101 bytes} [Mon Aug 22 15:46:52 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 22 15:46:52.238803 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf6e04b5-4795-4521-89ee-8b6425f67e64 None admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 15:46:52.324680 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=100 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 15:46:52.367183 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf6e04b5-4795-4521-89ee-8b6425f67e64 None admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 15:46:52.419154 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c1bafd2-8da2-497a-a393-082b618cc932 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 15:46:52.419821 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c1bafd2-8da2-497a-a393-082b618cc932 None admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:46:52.420093 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c1bafd2-8da2-497a-a393-082b618cc932 None admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:46:52.420359 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c1bafd2-8da2-497a-a393-082b618cc932 None admin] PATH_INFO: `/v3/users/038404f0743144d7bbc039278710e57e` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:46:52.421565 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c1bafd2-8da2-497a-a393-082b618cc932 None admin] RBAC: Authorizing `identity:get_user(user_id=038404f0743144d7bbc039278710e57e)` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 15:46:52.423017 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c1bafd2-8da2-497a-a393-082b618cc932 None admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 15:46:52.425032 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 151/291] 10.222.0.15 () {56 vars in 1177 bytes} [Mon Aug 22 15:46:52 2022] GET /identity/v3/users/038404f0743144d7bbc039278710e57e => generated 297 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 15:46:52.430613 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85da3c84-f153-4ba9-af6a-e76a098361a5 None admin] Authenticating user token {{(pid=81767) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 15:46:52.568861 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85da3c84-f153-4ba9-af6a-e76a098361a5 None admin] Validating token access rules against request {{(pid=81767) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 15:46:52.597831 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG dbcounter [-] [81767] Writing DB stats keystone:SELECT=100 {{(pid=81767) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 15:46:52.614428 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11f228e7-8721-4e35-bb46-9caacfa2e46f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81767) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 15:46:52.615052 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11f228e7-8721-4e35-bb46-9caacfa2e46f None admin] REQUEST_METHOD: `GET` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:46:52.615347 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11f228e7-8721-4e35-bb46-9caacfa2e46f None admin] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:46:52.615621 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11f228e7-8721-4e35-bb46-9caacfa2e46f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:46:52.706111 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11f228e7-8721-4e35-bb46-9caacfa2e46f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 15:46:52.707654 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11f228e7-8721-4e35-bb46-9caacfa2e46f None admin] RBAC: Authorization granted {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 15:46:52.847539 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 141/292] 10.222.0.15 () {58 vars in 1282 bytes} [Mon Aug 22 15:46:52 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 417 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 22 15:46:52.854473 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c1bafd2-8da2-497a-a393-082b618cc932 None admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 15:46:53.061593 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c1bafd2-8da2-497a-a393-082b618cc932 None admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 15:46:53.106919 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4458f7bf-b6a4-42ad-bf7e-67646d0aeafb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 15:46:53.107964 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4458f7bf-b6a4-42ad-bf7e-67646d0aeafb None admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:46:53.108441 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4458f7bf-b6a4-42ad-bf7e-67646d0aeafb None admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:46:53.108905 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4458f7bf-b6a4-42ad-bf7e-67646d0aeafb None admin] PATH_INFO: `/v3/projects/c05dae9fd0dc44f9a2ab2d373bbc913f` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:46:53.110272 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4458f7bf-b6a4-42ad-bf7e-67646d0aeafb None admin] RBAC: Authorizing `identity:get_project(project_id=c05dae9fd0dc44f9a2ab2d373bbc913f)` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 15:46:53.112011 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4458f7bf-b6a4-42ad-bf7e-67646d0aeafb None admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 15:46:53.113848 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 152/293] 10.222.0.15 () {56 vars in 1186 bytes} [Mon Aug 22 15:46:52 2022] GET /identity/v3/projects/c05dae9fd0dc44f9a2ab2d373bbc913f => generated 302 bytes in 260 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 15:46:53.119951 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11f228e7-8721-4e35-bb46-9caacfa2e46f None admin] Authenticating user token {{(pid=81767) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 15:46:53.159607 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG dbcounter [-] [81767] Writing DB stats keystone:SELECT=100 {{(pid=81767) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 15:46:53.252699 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b61f070b-5673-4f2e-a407-a5a3553e4435 None None] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:46:53.253041 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b61f070b-5673-4f2e-a407-a5a3553e4435 None None] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:46:53.253240 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b61f070b-5673-4f2e-a407-a5a3553e4435 None None] PATH_INFO: `/` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:46:53.254696 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 153/294] 10.222.0.15 () {54 vars in 927 bytes} [Mon Aug 22 15:46:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 22 15:46:53.260155 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6754be6f-b638-47ca-937f-bdce7fc4a256 None None] REQUEST_METHOD: `POST` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:46:53.260434 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6754be6f-b638-47ca-937f-bdce7fc4a256 None None] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:46:53.260706 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6754be6f-b638-47ca-937f-bdce7fc4a256 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:46:53.268612 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11f228e7-8721-4e35-bb46-9caacfa2e46f None admin] Validating token access rules against request {{(pid=81767) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 15:46:53.273622 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=100 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 15:46:53.290244 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.auth.core [None req-6754be6f-b638-47ca-937f-bdce7fc4a256 None None] MFA Rules not processed for user `6b2199ae0fb0480aa917ada391f5ec54`. Rule list: `[]` (Enabled: `True`). {{(pid=81768) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 15:46:53.291658 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-6754be6f-b638-47ca-937f-bdce7fc4a256 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 15:46:53.338694 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35d69c0d-c483-41b3-b056-655f12ca409b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81767) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 15:46:53.339408 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35d69c0d-c483-41b3-b056-655f12ca409b None admin] REQUEST_METHOD: `GET` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:46:53.339686 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35d69c0d-c483-41b3-b056-655f12ca409b None admin] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:46:53.340542 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35d69c0d-c483-41b3-b056-655f12ca409b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:46:53.345121 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35d69c0d-c483-41b3-b056-655f12ca409b None admin] RBAC: Adding query filter params (role.id=e6fc1ee3fb974125ba9b4fe55b12b45e, scope.project.id=c05dae9fd0dc44f9a2ab2d373bbc913f, user.id=038404f0743144d7bbc039278710e57e) {{(pid=81767) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 15:46:53.345121 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35d69c0d-c483-41b3-b056-655f12ca409b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 15:46:53.345121 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35d69c0d-c483-41b3-b056-655f12ca409b None admin] RBAC: Authorization granted {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 15:46:53.357920 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 142/295] 10.222.0.15 () {56 vars in 1374 bytes} [Mon Aug 22 15:46:53 2022] GET /identity/v3/role_assignments?user.id=038404f0743144d7bbc039278710e57e&scope.project.id=c05dae9fd0dc44f9a2ab2d373bbc913f&role.id=e6fc1ee3fb974125ba9b4fe55b12b45e => generated 608 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 15:46:53.437795 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 154/296] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Aug 22 15:46:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 22 15:46:53.471040 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9000f799-f675-445c-be93-3fc7871cc51b None None] REQUEST_METHOD: `GET` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:46:53.471433 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9000f799-f675-445c-be93-3fc7871cc51b None None] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:46:53.471638 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9000f799-f675-445c-be93-3fc7871cc51b None None] PATH_INFO: `/` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:46:53.473157 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 143/297] 10.222.0.15 () {54 vars in 927 bytes} [Mon Aug 22 15:46:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 15:46:53.478560 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d46e056f-6153-41c0-9ce3-849348de44af None None] REQUEST_METHOD: `POST` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:46:53.478845 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d46e056f-6153-41c0-9ce3-849348de44af None None] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:46:53.479260 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d46e056f-6153-41c0-9ce3-849348de44af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:46:53.509319 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.auth.core [None req-d46e056f-6153-41c0-9ce3-849348de44af None None] MFA Rules not processed for user `6b2199ae0fb0480aa917ada391f5ec54`. Rule list: `[]` (Enabled: `True`). {{(pid=81768) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 15:46:53.510517 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-d46e056f-6153-41c0-9ce3-849348de44af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 15:46:53.596350 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 155/298] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Aug 22 15:46:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 22 15:46:53.742472 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9000f799-f675-445c-be93-3fc7871cc51b None None] Authenticating user token {{(pid=81767) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 15:46:53.848887 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG dbcounter [-] [81767] Writing DB stats keystone:SELECT=100 {{(pid=81767) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 15:46:53.864683 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9000f799-f675-445c-be93-3fc7871cc51b None None] Validating token access rules against request {{(pid=81767) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 15:46:53.886367 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d46e056f-6153-41c0-9ce3-849348de44af None None] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 15:46:53.907124 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fc21f3a-1fb9-4bc2-883a-ecb78a919ff0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81767) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 15:46:53.908010 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fc21f3a-1fb9-4bc2-883a-ecb78a919ff0 None admin] REQUEST_METHOD: `GET` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:46:53.908477 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fc21f3a-1fb9-4bc2-883a-ecb78a919ff0 None admin] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:46:53.909707 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fc21f3a-1fb9-4bc2-883a-ecb78a919ff0 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:46:53.926337 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fc21f3a-1fb9-4bc2-883a-ecb78a919ff0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 22 15:46:53.933823 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application [None req-1fc21f3a-1fb9-4bc2-883a-ecb78a919ff0 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 22 15:46:53.933823 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 15:46:53.933823 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 15:46:53.933823 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 15:46:53.933823 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 15:46:53.933823 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 15:46:53.933823 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 15:46:53.933823 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 15:46:53.933823 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 15:46:53.933823 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 15:46:53.933823 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 15:46:53.933823 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 15:46:53.933823 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 22 15:46:53.933823 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 22 15:46:53.933823 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 22 15:46:53.933823 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 22 15:46:53.934730 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 22 15:46:53.934730 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 22 15:46:53.934730 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 22 15:46:53.934730 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 22 15:46:53.934730 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 15:46:53.934730 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 15:46:53.934730 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 15:46:53.934730 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 15:46:53.934730 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 15:46:53.934730 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 15:46:53.934730 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 15:46:53.934730 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application with Lock( Aug 22 15:46:53.934730 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 15:46:53.934730 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return self._enter() Aug 22 15:46:53.934730 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 15:46:53.934730 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 15:46:53.935912 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 15:46:53.935912 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return self.creator() Aug 22 15:46:53.935912 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 15:46:53.935912 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application created_value = creator( Aug 22 15:46:53.935912 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 22 15:46:53.935912 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 22 15:46:53.935912 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 22 15:46:53.935912 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 22 15:46:53.935912 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 22 15:46:53.935912 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 22 15:46:53.935912 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 22 15:46:53.935912 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application Aug 22 15:46:53.938327 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 144/299] 10.222.0.15 () {56 vars in 1099 bytes} [Mon Aug 22 15:46:53 2022] GET /identity/v3/roles/reader => generated 84 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 15:46:53.946538 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fc21f3a-1fb9-4bc2-883a-ecb78a919ff0 None admin] Authenticating user token {{(pid=81767) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 15:46:53.977314 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=100 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 15:46:54.019453 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d46e056f-6153-41c0-9ce3-849348de44af None None] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 15:46:54.060462 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca0218b8-7b99-49ea-9688-197b38369042 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 15:46:54.061097 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca0218b8-7b99-49ea-9688-197b38369042 None admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:46:54.061375 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca0218b8-7b99-49ea-9688-197b38369042 None admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:46:54.061646 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca0218b8-7b99-49ea-9688-197b38369042 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:46:54.062754 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca0218b8-7b99-49ea-9688-197b38369042 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 15:46:54.064410 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca0218b8-7b99-49ea-9688-197b38369042 None admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 15:46:54.071320 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 156/300] 10.222.0.15 () {56 vars in 1108 bytes} [Mon Aug 22 15:46:53 2022] GET /identity/v3/domains/default => generated 199 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 22 15:46:54.073824 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca0218b8-7b99-49ea-9688-197b38369042 None admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 15:46:54.089370 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fc21f3a-1fb9-4bc2-883a-ecb78a919ff0 None admin] Validating token access rules against request {{(pid=81767) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 15:46:54.128562 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6677b31e-0919-483b-93dc-22468014163f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81767) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 15:46:54.129169 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6677b31e-0919-483b-93dc-22468014163f None admin] REQUEST_METHOD: `GET` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:46:54.129444 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6677b31e-0919-483b-93dc-22468014163f None admin] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:46:54.129716 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6677b31e-0919-483b-93dc-22468014163f None admin] PATH_INFO: `/v3/roles` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:46:54.130381 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6677b31e-0919-483b-93dc-22468014163f None admin] RBAC: Adding query filter params (name=reader) {{(pid=81767) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 15:46:54.130685 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6677b31e-0919-483b-93dc-22468014163f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 15:46:54.132073 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6677b31e-0919-483b-93dc-22468014163f None admin] RBAC: Authorization granted {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 15:46:54.141262 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 145/301] 10.222.0.15 () {56 vars in 1101 bytes} [Mon Aug 22 15:46:53 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 22 15:46:54.142216 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f5bcf97-27d6-4089-bf27-42c1e71e732c None None] REQUEST_METHOD: `GET` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:46:54.142496 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f5bcf97-27d6-4089-bf27-42c1e71e732c None None] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:46:54.142765 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f5bcf97-27d6-4089-bf27-42c1e71e732c None None] PATH_INFO: `/` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:46:54.147451 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 146/302] 10.222.0.15 () {54 vars in 927 bytes} [Mon Aug 22 15:46:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 15:46:54.147451 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f5bcf97-27d6-4089-bf27-42c1e71e732c None None] Authenticating user token {{(pid=81767) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 15:46:54.161604 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG dbcounter [-] [81767] Writing DB stats keystone:SELECT=100 {{(pid=81767) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 15:46:54.207828 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca0218b8-7b99-49ea-9688-197b38369042 None admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 15:46:54.242871 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=100 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 15:46:54.248122 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c103a0f3-8cc5-4b2e-86f1-b65457c9971d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 15:46:54.248730 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c103a0f3-8cc5-4b2e-86f1-b65457c9971d None admin] REQUEST_METHOD: `POST` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:46:54.249005 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c103a0f3-8cc5-4b2e-86f1-b65457c9971d None admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:46:54.249273 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c103a0f3-8cc5-4b2e-86f1-b65457c9971d None admin] PATH_INFO: `/v3/groups` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:46:54.249921 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c103a0f3-8cc5-4b2e-86f1-b65457c9971d None admin] RBAC: Authorizing `identity:create_group()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 15:46:54.251398 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c103a0f3-8cc5-4b2e-86f1-b65457c9971d None admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 15:46:54.264906 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f5bcf97-27d6-4089-bf27-42c1e71e732c None None] Validating token access rules against request {{(pid=81767) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 15:46:54.298157 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0880cd73-a341-4e00-8d41-c8fe414c8bf1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81767) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 15:46:54.298779 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0880cd73-a341-4e00-8d41-c8fe414c8bf1 None admin] REQUEST_METHOD: `GET` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:46:54.299051 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0880cd73-a341-4e00-8d41-c8fe414c8bf1 None admin] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:46:54.299366 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0880cd73-a341-4e00-8d41-c8fe414c8bf1 None admin] PATH_INFO: `/v3/users/68b4a39eda4d43cb8b69acf6c3bbdc9d` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:46:54.313296 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0880cd73-a341-4e00-8d41-c8fe414c8bf1 None admin] RBAC: Authorizing `identity:get_user(user_id=68b4a39eda4d43cb8b69acf6c3bbdc9d)` {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 15:46:54.317389 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0880cd73-a341-4e00-8d41-c8fe414c8bf1 None admin] RBAC: Authorization granted {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 15:46:54.317389 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 147/303] 10.222.0.15 () {56 vars in 1177 bytes} [Mon Aug 22 15:46:54 2022] GET /identity/v3/users/68b4a39eda4d43cb8b69acf6c3bbdc9d => generated 293 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 15:46:54.317545 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5bc1954-7e9e-465c-93c2-8b2f539f4fdd None None] REQUEST_METHOD: `POST` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:46:54.317745 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5bc1954-7e9e-465c-93c2-8b2f539f4fdd None None] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:46:54.318030 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5bc1954-7e9e-465c-93c2-8b2f539f4fdd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:46:54.335835 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.auth.core [None req-d5bc1954-7e9e-465c-93c2-8b2f539f4fdd None None] MFA Rules not processed for user `6b2199ae0fb0480aa917ada391f5ec54`. Rule list: `[]` (Enabled: `True`). {{(pid=81767) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 15:46:54.340457 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.fernet_utils [None req-d5bc1954-7e9e-465c-93c2-8b2f539f4fdd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81767) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 15:46:54.342089 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 157/304] 10.222.0.15 () {60 vars in 1134 bytes} [Mon Aug 22 15:46:54 2022] POST /identity/v3/groups => generated 229 bytes in 269 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 22 15:46:54.343377 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03f7c3df-4dcb-44e7-a8c5-8d7ed62329cc None None] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:46:54.343667 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03f7c3df-4dcb-44e7-a8c5-8d7ed62329cc None None] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:46:54.343944 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03f7c3df-4dcb-44e7-a8c5-8d7ed62329cc None None] PATH_INFO: `/` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:46:54.345323 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 158/305] 10.222.0.15 () {54 vars in 927 bytes} [Mon Aug 22 15:46:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 15:46:54.345918 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03f7c3df-4dcb-44e7-a8c5-8d7ed62329cc None None] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 15:46:54.426706 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG dbcounter [-] [81767] Writing DB stats keystone:SELECT=100 {{(pid=81767) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 15:46:54.436234 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 148/306] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Aug 22 15:46:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 22 15:46:54.437904 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-911379a8-98b3-40f0-a1c2-b758a5fc757c None None] REQUEST_METHOD: `POST` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:46:54.438249 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-911379a8-98b3-40f0-a1c2-b758a5fc757c None None] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:46:54.443032 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-911379a8-98b3-40f0-a1c2-b758a5fc757c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:46:54.466556 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.auth.core [None req-911379a8-98b3-40f0-a1c2-b758a5fc757c None None] MFA Rules not processed for user `6b2199ae0fb0480aa917ada391f5ec54`. Rule list: `[]` (Enabled: `True`). {{(pid=81767) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 15:46:54.467773 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.fernet_utils [None req-911379a8-98b3-40f0-a1c2-b758a5fc757c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81767) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 15:46:54.494125 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03f7c3df-4dcb-44e7-a8c5-8d7ed62329cc None None] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 15:46:54.532824 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-622e9bb9-874c-45d0-9b98-a563036fe47f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 15:46:54.533518 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-622e9bb9-874c-45d0-9b98-a563036fe47f None admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:46:54.533799 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-622e9bb9-874c-45d0-9b98-a563036fe47f None admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:46:54.534081 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-622e9bb9-874c-45d0-9b98-a563036fe47f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:46:54.534804 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-622e9bb9-874c-45d0-9b98-a563036fe47f None admin] RBAC: Adding query filter params (role.id=640209879fb24ac0ae3f65dd107697c3, user.id=68b4a39eda4d43cb8b69acf6c3bbdc9d, scope.system=all) {{(pid=81768) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 15:46:54.535144 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-622e9bb9-874c-45d0-9b98-a563036fe47f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 15:46:54.536634 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-622e9bb9-874c-45d0-9b98-a563036fe47f None admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 15:46:54.544537 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 149/307] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Aug 22 15:46:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 22 15:46:54.548122 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 159/308] 10.222.0.15 () {56 vars in 1308 bytes} [Mon Aug 22 15:46:54 2022] GET /identity/v3/role_assignments?user.id=68b4a39eda4d43cb8b69acf6c3bbdc9d&scope.system=all&role.id=640209879fb24ac0ae3f65dd107697c3 => generated 226 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 15:46:54.710256 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-911379a8-98b3-40f0-a1c2-b758a5fc757c None None] Authenticating user token {{(pid=81767) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 15:46:54.814540 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-911379a8-98b3-40f0-a1c2-b758a5fc757c None None] Validating token access rules against request {{(pid=81767) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 15:46:54.830056 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-622e9bb9-874c-45d0-9b98-a563036fe47f None admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 15:46:54.843698 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG dbcounter [-] [81767] Writing DB stats keystone:SELECT=100 {{(pid=81767) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 15:46:54.852774 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-705ff38d-04a5-43f9-afa1-ad30bfcdb499 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81767) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 15:46:54.853742 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-705ff38d-04a5-43f9-afa1-ad30bfcdb499 None admin] REQUEST_METHOD: `GET` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:46:54.856910 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-705ff38d-04a5-43f9-afa1-ad30bfcdb499 None admin] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:46:54.856910 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-705ff38d-04a5-43f9-afa1-ad30bfcdb499 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:46:54.869469 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-705ff38d-04a5-43f9-afa1-ad30bfcdb499 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 22 15:46:54.879354 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application [None req-705ff38d-04a5-43f9-afa1-ad30bfcdb499 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 22 15:46:54.879354 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 15:46:54.879354 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 15:46:54.879354 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 15:46:54.879354 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 15:46:54.879354 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 15:46:54.879354 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 15:46:54.879354 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 15:46:54.879354 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 15:46:54.879354 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 15:46:54.879354 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 15:46:54.879354 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 15:46:54.879354 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 22 15:46:54.879354 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 22 15:46:54.879354 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 22 15:46:54.879354 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 22 15:46:54.880288 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 22 15:46:54.880288 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 22 15:46:54.880288 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 22 15:46:54.880288 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 22 15:46:54.880288 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 15:46:54.880288 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 15:46:54.880288 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 15:46:54.880288 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 15:46:54.880288 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 15:46:54.880288 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 15:46:54.880288 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 15:46:54.880288 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application with Lock( Aug 22 15:46:54.880288 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 15:46:54.880288 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return self._enter() Aug 22 15:46:54.880288 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 15:46:54.880288 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 15:46:54.881050 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 15:46:54.881050 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return self.creator() Aug 22 15:46:54.881050 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 15:46:54.881050 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application created_value = creator( Aug 22 15:46:54.881050 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 22 15:46:54.881050 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 22 15:46:54.881050 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 22 15:46:54.881050 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 22 15:46:54.881050 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 22 15:46:54.881050 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 22 15:46:54.881050 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 22 15:46:54.881050 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application Aug 22 15:46:54.881050 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 150/309] 10.222.0.15 () {56 vars in 1099 bytes} [Mon Aug 22 15:46:54 2022] GET /identity/v3/roles/member => generated 84 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 15:46:54.884675 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-705ff38d-04a5-43f9-afa1-ad30bfcdb499 None admin] Authenticating user token {{(pid=81767) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 15:46:54.911143 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 15:46:54.961627 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-622e9bb9-874c-45d0-9b98-a563036fe47f None admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 15:46:55.007326 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76c3761b-859e-4cee-ab2c-b05eff341427 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 15:46:55.008431 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76c3761b-859e-4cee-ab2c-b05eff341427 None admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:46:55.008768 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76c3761b-859e-4cee-ab2c-b05eff341427 None admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:46:55.009051 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76c3761b-859e-4cee-ab2c-b05eff341427 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:46:55.022059 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76c3761b-859e-4cee-ab2c-b05eff341427 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 22 15:46:55.022362 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-705ff38d-04a5-43f9-afa1-ad30bfcdb499 None admin] Validating token access rules against request {{(pid=81767) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 15:46:55.028616 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application [None req-76c3761b-859e-4cee-ab2c-b05eff341427 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 22 15:46:55.028616 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 15:46:55.028616 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 15:46:55.028616 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 15:46:55.028616 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 15:46:55.028616 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 15:46:55.028616 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 15:46:55.028616 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 15:46:55.028616 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 15:46:55.028616 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 15:46:55.028616 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 15:46:55.028616 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 15:46:55.028616 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 22 15:46:55.028616 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 22 15:46:55.028616 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 22 15:46:55.028616 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 22 15:46:55.029533 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 22 15:46:55.029533 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 22 15:46:55.029533 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 22 15:46:55.029533 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 22 15:46:55.029533 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 15:46:55.029533 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 15:46:55.029533 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 15:46:55.029533 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 15:46:55.029533 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 15:46:55.029533 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 15:46:55.029533 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 15:46:55.029533 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application with Lock( Aug 22 15:46:55.029533 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 15:46:55.029533 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self._enter() Aug 22 15:46:55.029533 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 15:46:55.029533 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 15:46:55.030326 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 15:46:55.030326 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self.creator() Aug 22 15:46:55.030326 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 15:46:55.030326 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application created_value = creator( Aug 22 15:46:55.030326 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 22 15:46:55.030326 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 22 15:46:55.030326 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 22 15:46:55.030326 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 22 15:46:55.030326 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 22 15:46:55.030326 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 22 15:46:55.030326 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 22 15:46:55.030326 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application Aug 22 15:46:55.032702 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 160/310] 10.222.0.15 () {56 vars in 1099 bytes} [Mon Aug 22 15:46:54 2022] GET /identity/v3/roles/reader => generated 84 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 15:46:55.039239 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76c3761b-859e-4cee-ab2c-b05eff341427 None admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 15:46:55.064234 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d993c42-0b71-4fd1-9bcb-94001a95d1ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81767) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 15:46:55.065117 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d993c42-0b71-4fd1-9bcb-94001a95d1ae None admin] REQUEST_METHOD: `GET` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:46:55.065402 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d993c42-0b71-4fd1-9bcb-94001a95d1ae None admin] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:46:55.065672 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d993c42-0b71-4fd1-9bcb-94001a95d1ae None admin] PATH_INFO: `/v3/roles` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:46:55.066346 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d993c42-0b71-4fd1-9bcb-94001a95d1ae None admin] RBAC: Adding query filter params (name=member) {{(pid=81767) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 15:46:55.067056 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d993c42-0b71-4fd1-9bcb-94001a95d1ae None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 15:46:55.068491 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d993c42-0b71-4fd1-9bcb-94001a95d1ae None admin] RBAC: Authorization granted {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 15:46:55.083419 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 151/311] 10.222.0.15 () {56 vars in 1101 bytes} [Mon Aug 22 15:46:54 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 15:46:55.087269 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d993c42-0b71-4fd1-9bcb-94001a95d1ae None admin] Authenticating user token {{(pid=81767) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 15:46:55.141423 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG dbcounter [-] [81767] Writing DB stats keystone:SELECT=100 {{(pid=81767) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 15:46:55.172739 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76c3761b-859e-4cee-ab2c-b05eff341427 None admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 15:46:55.193337 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=100 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 15:46:55.196207 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d993c42-0b71-4fd1-9bcb-94001a95d1ae None admin] Validating token access rules against request {{(pid=81767) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 15:46:55.214786 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f06cf13-5600-4137-8a4d-3c6ddf266a90 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 15:46:55.215863 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f06cf13-5600-4137-8a4d-3c6ddf266a90 None admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:46:55.216357 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f06cf13-5600-4137-8a4d-3c6ddf266a90 None admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:46:55.216821 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f06cf13-5600-4137-8a4d-3c6ddf266a90 None admin] PATH_INFO: `/v3/roles` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:46:55.217842 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f06cf13-5600-4137-8a4d-3c6ddf266a90 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81768) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 15:46:55.218338 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f06cf13-5600-4137-8a4d-3c6ddf266a90 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 15:46:55.219948 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f06cf13-5600-4137-8a4d-3c6ddf266a90 None admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 15:46:55.230315 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 161/312] 10.222.0.15 () {56 vars in 1101 bytes} [Mon Aug 22 15:46:55 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 15:46:55.235324 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-636ffde5-2778-48af-8b95-205ccaacb2fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81767) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 15:46:55.235324 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-636ffde5-2778-48af-8b95-205ccaacb2fc None admin] REQUEST_METHOD: `GET` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:46:55.235324 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-636ffde5-2778-48af-8b95-205ccaacb2fc None admin] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:46:55.235324 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-636ffde5-2778-48af-8b95-205ccaacb2fc None admin] PATH_INFO: `/v3/users/9efe902c9ad4404ebcbedea832133050` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:46:55.235665 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-636ffde5-2778-48af-8b95-205ccaacb2fc None admin] RBAC: Authorizing `identity:get_user(user_id=9efe902c9ad4404ebcbedea832133050)` {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 15:46:55.235837 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-636ffde5-2778-48af-8b95-205ccaacb2fc None admin] RBAC: Authorization granted {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 15:46:55.236802 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f06cf13-5600-4137-8a4d-3c6ddf266a90 None admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 15:46:55.237657 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 152/313] 10.222.0.15 () {56 vars in 1177 bytes} [Mon Aug 22 15:46:55 2022] GET /identity/v3/users/9efe902c9ad4404ebcbedea832133050 => generated 293 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 15:46:55.244433 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-636ffde5-2778-48af-8b95-205ccaacb2fc None admin] Authenticating user token {{(pid=81767) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 15:46:55.353840 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-636ffde5-2778-48af-8b95-205ccaacb2fc None admin] Validating token access rules against request {{(pid=81767) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 15:46:55.367300 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f06cf13-5600-4137-8a4d-3c6ddf266a90 None admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 15:46:55.373592 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG dbcounter [-] [81767] Writing DB stats keystone:SELECT=100 {{(pid=81767) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 15:46:55.392455 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-645ec960-6b9c-45e3-be4c-542caa04bc7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81767) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 15:46:55.393117 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-645ec960-6b9c-45e3-be4c-542caa04bc7c None admin] REQUEST_METHOD: `PUT` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:46:55.393418 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-645ec960-6b9c-45e3-be4c-542caa04bc7c None admin] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:46:55.393666 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-645ec960-6b9c-45e3-be4c-542caa04bc7c None admin] PATH_INFO: `/v3/system/users/9efe902c9ad4404ebcbedea832133050/roles/e6fc1ee3fb974125ba9b4fe55b12b45e` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:46:55.395098 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-645ec960-6b9c-45e3-be4c-542caa04bc7c None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=9efe902c9ad4404ebcbedea832133050, role_id=e6fc1ee3fb974125ba9b4fe55b12b45e)` {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 15:46:55.396505 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-645ec960-6b9c-45e3-be4c-542caa04bc7c None admin] RBAC: Authorization granted {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 15:46:55.401586 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-506d433d-7fd3-4de8-b81e-88f7bdfd268f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 15:46:55.402327 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-506d433d-7fd3-4de8-b81e-88f7bdfd268f None admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:46:55.402676 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-506d433d-7fd3-4de8-b81e-88f7bdfd268f None admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:46:55.403033 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-506d433d-7fd3-4de8-b81e-88f7bdfd268f None admin] PATH_INFO: `/v3/users/f1f4430458e44ebcb8cfd11bf292c973` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:46:55.404321 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-506d433d-7fd3-4de8-b81e-88f7bdfd268f None admin] RBAC: Authorizing `identity:get_user(user_id=f1f4430458e44ebcb8cfd11bf292c973)` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 15:46:55.405848 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-506d433d-7fd3-4de8-b81e-88f7bdfd268f None admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 15:46:55.407623 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 162/314] 10.222.0.15 () {56 vars in 1177 bytes} [Mon Aug 22 15:46:55 2022] GET /identity/v3/users/f1f4430458e44ebcb8cfd11bf292c973 => generated 297 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 15:46:55.413080 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-506d433d-7fd3-4de8-b81e-88f7bdfd268f None admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 15:46:55.467672 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=100 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 15:46:55.520846 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-506d433d-7fd3-4de8-b81e-88f7bdfd268f None admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 15:46:55.531329 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 153/315] 10.222.0.15 () {58 vars in 1334 bytes} [Mon Aug 22 15:46:55 2022] PUT /identity/v3/system/users/9efe902c9ad4404ebcbedea832133050/roles/e6fc1ee3fb974125ba9b4fe55b12b45e => generated 0 bytes in 288 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 22 15:46:55.572165 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b25f4f26-4ebb-46c0-81fc-6fd412399108 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 15:46:55.572962 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b25f4f26-4ebb-46c0-81fc-6fd412399108 None admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:46:55.573370 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b25f4f26-4ebb-46c0-81fc-6fd412399108 None admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:46:55.573646 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b25f4f26-4ebb-46c0-81fc-6fd412399108 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:46:55.657978 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b25f4f26-4ebb-46c0-81fc-6fd412399108 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 15:46:55.659514 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b25f4f26-4ebb-46c0-81fc-6fd412399108 None admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 15:46:55.713961 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=100 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 15:46:55.772001 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 163/316] 10.222.0.15 () {58 vars in 1282 bytes} [Mon Aug 22 15:46:55 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 359 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 22 15:46:55.778231 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-645ec960-6b9c-45e3-be4c-542caa04bc7c None admin] Authenticating user token {{(pid=81767) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 15:46:55.886756 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-645ec960-6b9c-45e3-be4c-542caa04bc7c None admin] Validating token access rules against request {{(pid=81767) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 15:46:55.938835 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ce01638-24e8-492e-9ea8-eddc8cc3006b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81767) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 15:46:55.940363 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ce01638-24e8-492e-9ea8-eddc8cc3006b None admin] REQUEST_METHOD: `GET` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:46:55.941123 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ce01638-24e8-492e-9ea8-eddc8cc3006b None admin] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:46:55.941853 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ce01638-24e8-492e-9ea8-eddc8cc3006b None admin] PATH_INFO: `/v3/projects/c05dae9fd0dc44f9a2ab2d373bbc913f` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:46:55.944037 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ce01638-24e8-492e-9ea8-eddc8cc3006b None admin] RBAC: Authorizing `identity:get_project(project_id=c05dae9fd0dc44f9a2ab2d373bbc913f)` {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 15:46:55.946995 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ce01638-24e8-492e-9ea8-eddc8cc3006b None admin] RBAC: Authorization granted {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 15:46:55.950646 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 154/317] 10.222.0.15 () {56 vars in 1186 bytes} [Mon Aug 22 15:46:55 2022] GET /identity/v3/projects/c05dae9fd0dc44f9a2ab2d373bbc913f => generated 302 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 15:46:55.959818 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b25f4f26-4ebb-46c0-81fc-6fd412399108 None admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 15:46:56.077259 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b25f4f26-4ebb-46c0-81fc-6fd412399108 None admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 15:46:56.113705 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-162e8162-b999-48ea-8bad-ca476cef7789 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 15:46:56.114655 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-162e8162-b999-48ea-8bad-ca476cef7789 None admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:46:56.115135 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-162e8162-b999-48ea-8bad-ca476cef7789 None admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:46:56.115663 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-162e8162-b999-48ea-8bad-ca476cef7789 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:46:56.116589 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-162e8162-b999-48ea-8bad-ca476cef7789 None admin] RBAC: Adding query filter params (role.id=640209879fb24ac0ae3f65dd107697c3, scope.project.id=c05dae9fd0dc44f9a2ab2d373bbc913f, user.id=f1f4430458e44ebcb8cfd11bf292c973) {{(pid=81768) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 15:46:56.117092 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-162e8162-b999-48ea-8bad-ca476cef7789 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 15:46:56.118956 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-162e8162-b999-48ea-8bad-ca476cef7789 None admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 15:46:56.126105 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 164/318] 10.222.0.15 () {56 vars in 1374 bytes} [Mon Aug 22 15:46:55 2022] GET /identity/v3/role_assignments?user.id=f1f4430458e44ebcb8cfd11bf292c973&scope.project.id=c05dae9fd0dc44f9a2ab2d373bbc913f&role.id=640209879fb24ac0ae3f65dd107697c3 => generated 608 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 15:46:56.380366 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd074601-79e0-42f1-b1e1-417a00b7611e None None] REQUEST_METHOD: `GET` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:46:56.380939 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd074601-79e0-42f1-b1e1-417a00b7611e None None] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:46:56.381261 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd074601-79e0-42f1-b1e1-417a00b7611e None None] PATH_INFO: `/` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:46:56.382827 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 155/319] 10.222.0.15 () {54 vars in 927 bytes} [Mon Aug 22 15:46:56 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 15:46:56.391372 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c6ad8af-b8fc-4925-b08f-70184a88598b None None] REQUEST_METHOD: `POST` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:46:56.391783 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c6ad8af-b8fc-4925-b08f-70184a88598b None None] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:46:56.392222 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c6ad8af-b8fc-4925-b08f-70184a88598b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:46:56.412656 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.auth.core [None req-8c6ad8af-b8fc-4925-b08f-70184a88598b None None] MFA Rules not processed for user `6b2199ae0fb0480aa917ada391f5ec54`. Rule list: `[]` (Enabled: `True`). {{(pid=81768) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 15:46:56.413826 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-8c6ad8af-b8fc-4925-b08f-70184a88598b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 15:46:56.417422 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=100 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 15:46:56.483494 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 165/320] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Aug 22 15:46:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 22 15:46:56.608093 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a95a6b8-9b34-4e4f-822c-7441716c0bcf None None] REQUEST_METHOD: `GET` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:46:56.608382 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a95a6b8-9b34-4e4f-822c-7441716c0bcf None None] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:46:56.608543 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a95a6b8-9b34-4e4f-822c-7441716c0bcf None None] PATH_INFO: `/` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:46:56.610017 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 156/321] 10.222.0.15 () {54 vars in 927 bytes} [Mon Aug 22 15:46:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 15:46:56.615295 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f760b7d8-9eb8-4246-a07a-61bb9d72ad8c None None] REQUEST_METHOD: `POST` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:46:56.615578 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f760b7d8-9eb8-4246-a07a-61bb9d72ad8c None None] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:46:56.615852 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f760b7d8-9eb8-4246-a07a-61bb9d72ad8c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:46:56.634493 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.auth.core [None req-f760b7d8-9eb8-4246-a07a-61bb9d72ad8c None None] MFA Rules not processed for user `6b2199ae0fb0480aa917ada391f5ec54`. Rule list: `[]` (Enabled: `True`). {{(pid=81768) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 15:46:56.635696 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-f760b7d8-9eb8-4246-a07a-61bb9d72ad8c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 15:46:56.710560 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 166/322] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Aug 22 15:46:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 22 15:46:56.752875 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a95a6b8-9b34-4e4f-822c-7441716c0bcf None None] Authenticating user token {{(pid=81767) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 15:46:56.805823 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG dbcounter [-] [81767] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81767) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 15:46:56.860408 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a95a6b8-9b34-4e4f-822c-7441716c0bcf None None] Validating token access rules against request {{(pid=81767) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 15:46:56.898853 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c097a8a-fd32-45f2-b789-d2d213913347 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81767) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 15:46:56.899544 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c097a8a-fd32-45f2-b789-d2d213913347 None admin] REQUEST_METHOD: `GET` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:46:56.899789 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c097a8a-fd32-45f2-b789-d2d213913347 None admin] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:46:56.900053 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c097a8a-fd32-45f2-b789-d2d213913347 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:46:56.901146 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c097a8a-fd32-45f2-b789-d2d213913347 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 15:46:56.902738 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c097a8a-fd32-45f2-b789-d2d213913347 None admin] RBAC: Authorization granted {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 15:46:56.908812 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 157/323] 10.222.0.15 () {56 vars in 1108 bytes} [Mon Aug 22 15:46:56 2022] GET /identity/v3/domains/default => generated 199 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 15:46:56.912949 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f760b7d8-9eb8-4246-a07a-61bb9d72ad8c None None] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 15:46:56.988010 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=100 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 15:46:56.998745 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c097a8a-fd32-45f2-b789-d2d213913347 None admin] Authenticating user token {{(pid=81767) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 15:46:57.021880 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f760b7d8-9eb8-4246-a07a-61bb9d72ad8c None None] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 15:46:57.060143 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08a73337-daad-454c-9870-87323e32a078 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 15:46:57.061022 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08a73337-daad-454c-9870-87323e32a078 None admin] REQUEST_METHOD: `POST` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:46:57.061301 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08a73337-daad-454c-9870-87323e32a078 None admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:46:57.061719 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08a73337-daad-454c-9870-87323e32a078 None admin] PATH_INFO: `/v3/groups` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:46:57.062670 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08a73337-daad-454c-9870-87323e32a078 None admin] RBAC: Authorizing `identity:create_group()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 15:46:57.064153 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08a73337-daad-454c-9870-87323e32a078 None admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 15:46:57.102948 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c097a8a-fd32-45f2-b789-d2d213913347 None admin] Validating token access rules against request {{(pid=81767) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 15:46:57.120323 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG dbcounter [-] [81767] Writing DB stats keystone:SELECT=100 {{(pid=81767) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 15:46:57.137499 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f930fc9e-7558-4bad-b5d2-d537c63136db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81767) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 15:46:57.138182 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f930fc9e-7558-4bad-b5d2-d537c63136db None admin] REQUEST_METHOD: `GET` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:46:57.138458 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f930fc9e-7558-4bad-b5d2-d537c63136db None admin] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:46:57.138749 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f930fc9e-7558-4bad-b5d2-d537c63136db None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:46:57.149215 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f930fc9e-7558-4bad-b5d2-d537c63136db None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 22 15:46:57.154485 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application [None req-f930fc9e-7558-4bad-b5d2-d537c63136db None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 22 15:46:57.154485 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 15:46:57.154485 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 15:46:57.154485 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 15:46:57.154485 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 15:46:57.154485 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 15:46:57.154485 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 15:46:57.154485 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 15:46:57.154485 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 15:46:57.154485 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 15:46:57.154485 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 15:46:57.154485 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 15:46:57.154485 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 22 15:46:57.154485 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 22 15:46:57.154485 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 22 15:46:57.154485 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 22 15:46:57.155607 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 22 15:46:57.155607 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 22 15:46:57.155607 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 22 15:46:57.155607 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 22 15:46:57.155607 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 15:46:57.155607 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 15:46:57.155607 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 15:46:57.155607 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 15:46:57.155607 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 15:46:57.155607 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 15:46:57.155607 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 15:46:57.155607 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application with Lock( Aug 22 15:46:57.155607 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 15:46:57.155607 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return self._enter() Aug 22 15:46:57.155607 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 15:46:57.155607 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 15:46:57.172739 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 15:46:57.172739 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return self.creator() Aug 22 15:46:57.172739 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 15:46:57.172739 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application created_value = creator( Aug 22 15:46:57.172739 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 22 15:46:57.172739 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 22 15:46:57.172739 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 22 15:46:57.172739 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 22 15:46:57.172739 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 22 15:46:57.172739 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 22 15:46:57.172739 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 22 15:46:57.172739 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application Aug 22 15:46:57.172739 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 158/324] 10.222.0.15 () {56 vars in 1099 bytes} [Mon Aug 22 15:46:56 2022] GET /identity/v3/roles/reader => generated 84 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 15:46:57.172739 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f930fc9e-7558-4bad-b5d2-d537c63136db None admin] Authenticating user token {{(pid=81767) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 15:46:57.197672 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 167/325] 10.222.0.15 () {60 vars in 1134 bytes} [Mon Aug 22 15:46:56 2022] POST /identity/v3/groups => generated 226 bytes in 286 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 22 15:46:57.282912 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f930fc9e-7558-4bad-b5d2-d537c63136db None admin] Validating token access rules against request {{(pid=81767) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 15:46:57.319864 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d8d6326-1e2b-40d5-9b81-4a694d901758 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81767) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 15:46:57.320481 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d8d6326-1e2b-40d5-9b81-4a694d901758 None admin] REQUEST_METHOD: `GET` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:46:57.320759 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d8d6326-1e2b-40d5-9b81-4a694d901758 None admin] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:46:57.321060 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d8d6326-1e2b-40d5-9b81-4a694d901758 None admin] PATH_INFO: `/v3/roles` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:46:57.321745 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d8d6326-1e2b-40d5-9b81-4a694d901758 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81767) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 15:46:57.322050 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d8d6326-1e2b-40d5-9b81-4a694d901758 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 15:46:57.323532 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d8d6326-1e2b-40d5-9b81-4a694d901758 None admin] RBAC: Authorization granted {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 15:46:57.337447 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 159/326] 10.222.0.15 () {56 vars in 1101 bytes} [Mon Aug 22 15:46:57 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 22 15:46:57.339230 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08a73337-daad-454c-9870-87323e32a078 None admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 15:46:57.461892 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08a73337-daad-454c-9870-87323e32a078 None admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 15:46:57.494350 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 15:46:57.500105 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddccafda-8ecf-46f8-af07-888e6cb98360 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 15:46:57.500738 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddccafda-8ecf-46f8-af07-888e6cb98360 None admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:46:57.501012 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddccafda-8ecf-46f8-af07-888e6cb98360 None admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:46:57.501295 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddccafda-8ecf-46f8-af07-888e6cb98360 None admin] PATH_INFO: `/v3/users/68b4a39eda4d43cb8b69acf6c3bbdc9d` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:46:57.502458 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddccafda-8ecf-46f8-af07-888e6cb98360 None admin] RBAC: Authorizing `identity:get_user(user_id=68b4a39eda4d43cb8b69acf6c3bbdc9d)` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 15:46:57.503949 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddccafda-8ecf-46f8-af07-888e6cb98360 None admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 15:46:57.510128 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 168/327] 10.222.0.15 () {56 vars in 1177 bytes} [Mon Aug 22 15:46:57 2022] GET /identity/v3/users/68b4a39eda4d43cb8b69acf6c3bbdc9d => generated 293 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 15:46:57.511364 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d8d6326-1e2b-40d5-9b81-4a694d901758 None admin] Authenticating user token {{(pid=81767) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 15:46:57.564841 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG dbcounter [-] [81767] Writing DB stats keystone:SELECT=100 {{(pid=81767) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 15:46:57.583409 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69fe19c2-caad-4729-94cd-53a7e97d9478 None None] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:46:57.583706 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69fe19c2-caad-4729-94cd-53a7e97d9478 None None] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:46:57.583953 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69fe19c2-caad-4729-94cd-53a7e97d9478 None None] PATH_INFO: `/` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:46:57.585403 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 169/328] 10.222.0.15 () {54 vars in 927 bytes} [Mon Aug 22 15:46:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 15:46:57.593882 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6707e4bf-6cba-4330-9a61-104f6fd96dce None None] REQUEST_METHOD: `POST` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:46:57.594556 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6707e4bf-6cba-4330-9a61-104f6fd96dce None None] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:46:57.594916 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6707e4bf-6cba-4330-9a61-104f6fd96dce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:46:57.618289 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.auth.core [None req-6707e4bf-6cba-4330-9a61-104f6fd96dce None None] MFA Rules not processed for user `6b2199ae0fb0480aa917ada391f5ec54`. Rule list: `[]` (Enabled: `True`). {{(pid=81768) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 15:46:57.619889 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-6707e4bf-6cba-4330-9a61-104f6fd96dce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 15:46:57.636008 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d8d6326-1e2b-40d5-9b81-4a694d901758 None admin] Validating token access rules against request {{(pid=81767) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 15:46:57.680432 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0e23f1a-872b-4977-b517-0ed30928cd21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81767) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 15:46:57.681102 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e23f1a-872b-4977-b517-0ed30928cd21 None admin] REQUEST_METHOD: `PUT` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:46:57.681673 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e23f1a-872b-4977-b517-0ed30928cd21 None admin] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:46:57.681950 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e23f1a-872b-4977-b517-0ed30928cd21 None admin] PATH_INFO: `/v3/system/users/68b4a39eda4d43cb8b69acf6c3bbdc9d/roles/640209879fb24ac0ae3f65dd107697c3` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:46:57.684929 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0e23f1a-872b-4977-b517-0ed30928cd21 None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=68b4a39eda4d43cb8b69acf6c3bbdc9d, role_id=640209879fb24ac0ae3f65dd107697c3)` {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 15:46:57.686285 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0e23f1a-872b-4977-b517-0ed30928cd21 None admin] RBAC: Authorization granted {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 15:46:57.708430 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 170/329] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Aug 22 15:46:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 22 15:46:57.982068 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6707e4bf-6cba-4330-9a61-104f6fd96dce None None] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 15:46:58.088044 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6707e4bf-6cba-4330-9a61-104f6fd96dce None None] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 15:46:58.115977 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=100 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 15:46:58.124760 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be7e9442-e1e1-44e9-a16f-74735aedcef1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 15:46:58.125393 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be7e9442-e1e1-44e9-a16f-74735aedcef1 None admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:46:58.125666 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be7e9442-e1e1-44e9-a16f-74735aedcef1 None admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:46:58.125932 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be7e9442-e1e1-44e9-a16f-74735aedcef1 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:46:58.136215 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be7e9442-e1e1-44e9-a16f-74735aedcef1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 22 15:46:58.141632 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application [None req-be7e9442-e1e1-44e9-a16f-74735aedcef1 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 22 15:46:58.141632 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 15:46:58.141632 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 15:46:58.141632 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 15:46:58.141632 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 15:46:58.141632 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 15:46:58.141632 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 15:46:58.141632 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 15:46:58.141632 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 15:46:58.141632 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 15:46:58.141632 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 15:46:58.141632 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 15:46:58.141632 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 22 15:46:58.141632 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 22 15:46:58.141632 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 22 15:46:58.141632 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 22 15:46:58.142529 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 22 15:46:58.142529 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 22 15:46:58.142529 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 22 15:46:58.142529 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 22 15:46:58.142529 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 15:46:58.142529 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 15:46:58.142529 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 15:46:58.142529 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 15:46:58.142529 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 15:46:58.142529 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 15:46:58.142529 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 15:46:58.142529 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application with Lock( Aug 22 15:46:58.142529 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 15:46:58.142529 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self._enter() Aug 22 15:46:58.142529 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 15:46:58.142529 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 15:46:58.143392 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 15:46:58.143392 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self.creator() Aug 22 15:46:58.143392 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 15:46:58.143392 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application created_value = creator( Aug 22 15:46:58.143392 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 22 15:46:58.143392 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 22 15:46:58.143392 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 22 15:46:58.143392 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 22 15:46:58.143392 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 22 15:46:58.143392 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 22 15:46:58.143392 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 22 15:46:58.143392 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application Aug 22 15:46:58.143392 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 171/330] 10.222.0.15 () {56 vars in 1099 bytes} [Mon Aug 22 15:46:57 2022] GET /identity/v3/roles/member => generated 84 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 22 15:46:58.148527 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be7e9442-e1e1-44e9-a16f-74735aedcef1 None admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 15:46:58.204550 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 160/331] 10.222.0.15 () {58 vars in 1334 bytes} [Mon Aug 22 15:46:57 2022] PUT /identity/v3/system/users/68b4a39eda4d43cb8b69acf6c3bbdc9d/roles/640209879fb24ac0ae3f65dd107697c3 => generated 0 bytes in 693 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 22 15:46:58.267828 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be7e9442-e1e1-44e9-a16f-74735aedcef1 None admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 15:46:58.305609 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4435742-7c3c-4be8-82b2-da8807d58258 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 15:46:58.306509 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4435742-7c3c-4be8-82b2-da8807d58258 None admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:46:58.306786 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4435742-7c3c-4be8-82b2-da8807d58258 None admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:46:58.307132 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4435742-7c3c-4be8-82b2-da8807d58258 None admin] PATH_INFO: `/v3/roles` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:46:58.307813 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4435742-7c3c-4be8-82b2-da8807d58258 None admin] RBAC: Adding query filter params (name=member) {{(pid=81768) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 15:46:58.308113 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4435742-7c3c-4be8-82b2-da8807d58258 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 15:46:58.309573 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4435742-7c3c-4be8-82b2-da8807d58258 None admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 15:46:58.319140 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 172/332] 10.222.0.15 () {56 vars in 1101 bytes} [Mon Aug 22 15:46:58 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 22 15:46:58.326770 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0e23f1a-872b-4977-b517-0ed30928cd21 None admin] Authenticating user token {{(pid=81767) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 15:46:58.471684 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0e23f1a-872b-4977-b517-0ed30928cd21 None admin] Validating token access rules against request {{(pid=81767) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 15:46:58.474373 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG dbcounter [-] [81767] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81767) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 15:46:58.510974 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b159517a-f738-4114-8bea-c92810d903a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81767) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 15:46:58.512167 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b159517a-f738-4114-8bea-c92810d903a1 None admin] REQUEST_METHOD: `GET` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:46:58.512446 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b159517a-f738-4114-8bea-c92810d903a1 None admin] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:46:58.512718 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b159517a-f738-4114-8bea-c92810d903a1 None admin] PATH_INFO: `/v3/users/9efe902c9ad4404ebcbedea832133050` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:46:58.513955 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b159517a-f738-4114-8bea-c92810d903a1 None admin] RBAC: Authorizing `identity:get_user(user_id=9efe902c9ad4404ebcbedea832133050)` {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 15:46:58.515446 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b159517a-f738-4114-8bea-c92810d903a1 None admin] RBAC: Authorization granted {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 15:46:58.517048 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 161/333] 10.222.0.15 () {56 vars in 1177 bytes} [Mon Aug 22 15:46:58 2022] GET /identity/v3/users/9efe902c9ad4404ebcbedea832133050 => generated 293 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 15:46:58.525086 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4435742-7c3c-4be8-82b2-da8807d58258 None admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 15:46:58.600466 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=100 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 15:46:58.658346 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4435742-7c3c-4be8-82b2-da8807d58258 None admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 15:46:58.707099 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2520aaf1-d0d3-44dd-a2bf-5b847dad073e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 15:46:58.707734 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2520aaf1-d0d3-44dd-a2bf-5b847dad073e None admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:46:58.708013 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2520aaf1-d0d3-44dd-a2bf-5b847dad073e None admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:46:58.708289 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2520aaf1-d0d3-44dd-a2bf-5b847dad073e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:46:58.709020 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2520aaf1-d0d3-44dd-a2bf-5b847dad073e None admin] RBAC: Adding query filter params (role.id=e6fc1ee3fb974125ba9b4fe55b12b45e, user.id=9efe902c9ad4404ebcbedea832133050, scope.system=all) {{(pid=81768) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 15:46:58.709322 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2520aaf1-d0d3-44dd-a2bf-5b847dad073e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 15:46:58.710764 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2520aaf1-d0d3-44dd-a2bf-5b847dad073e None admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 15:46:58.726713 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 173/334] 10.222.0.15 () {56 vars in 1308 bytes} [Mon Aug 22 15:46:58 2022] GET /identity/v3/role_assignments?user.id=9efe902c9ad4404ebcbedea832133050&scope.system=all&role.id=e6fc1ee3fb974125ba9b4fe55b12b45e => generated 510 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 15:46:59.776423 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb8cf43c-e43b-4fa4-8563-db38ddcc6778 None None] REQUEST_METHOD: `GET` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:46:59.776940 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb8cf43c-e43b-4fa4-8563-db38ddcc6778 None None] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:46:59.776940 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb8cf43c-e43b-4fa4-8563-db38ddcc6778 None None] PATH_INFO: `/` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:46:59.778951 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 162/335] 10.222.0.15 () {54 vars in 927 bytes} [Mon Aug 22 15:46:59 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 15:46:59.785092 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-839a1f55-d3b5-4918-8286-30d22b2e3366 None None] REQUEST_METHOD: `POST` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:46:59.785599 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-839a1f55-d3b5-4918-8286-30d22b2e3366 None None] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:46:59.785680 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-839a1f55-d3b5-4918-8286-30d22b2e3366 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:46:59.805373 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.auth.core [None req-839a1f55-d3b5-4918-8286-30d22b2e3366 None None] MFA Rules not processed for user `6b2199ae0fb0480aa917ada391f5ec54`. Rule list: `[]` (Enabled: `True`). {{(pid=81768) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 15:46:59.807238 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-839a1f55-d3b5-4918-8286-30d22b2e3366 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 15:46:59.880963 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5b30a22-4ea3-4c6e-9269-61b03a436a57 None None] REQUEST_METHOD: `GET` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:46:59.881300 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5b30a22-4ea3-4c6e-9269-61b03a436a57 None None] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:46:59.881512 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5b30a22-4ea3-4c6e-9269-61b03a436a57 None None] PATH_INFO: `/` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:46:59.883248 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 163/336] 10.222.0.15 () {54 vars in 927 bytes} [Mon Aug 22 15:46:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 22 15:46:59.894832 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab56047-2941-4d89-b133-2033c6d78a24 None None] REQUEST_METHOD: `POST` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:46:59.895154 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab56047-2941-4d89-b133-2033c6d78a24 None None] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:46:59.899067 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab56047-2941-4d89-b133-2033c6d78a24 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:46:59.899900 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 174/337] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Aug 22 15:46:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 22 15:46:59.924579 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.auth.core [None req-7ab56047-2941-4d89-b133-2033c6d78a24 None None] MFA Rules not processed for user `6b2199ae0fb0480aa917ada391f5ec54`. Rule list: `[]` (Enabled: `True`). {{(pid=81767) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 15:46:59.925790 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.fernet_utils [None req-7ab56047-2941-4d89-b133-2033c6d78a24 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81767) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 15:47:00.014301 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 164/338] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Aug 22 15:46:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 22 15:47:00.202709 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-839a1f55-d3b5-4918-8286-30d22b2e3366 None None] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 15:47:00.245275 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=100 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 15:47:00.322740 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-839a1f55-d3b5-4918-8286-30d22b2e3366 None None] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 15:47:00.365320 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e274cd9c-edd6-4697-af62-60e4e6eb95c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 15:47:00.365960 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e274cd9c-edd6-4697-af62-60e4e6eb95c0 None admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:47:00.366235 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e274cd9c-edd6-4697-af62-60e4e6eb95c0 None admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:47:00.366503 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e274cd9c-edd6-4697-af62-60e4e6eb95c0 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:47:00.389670 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e274cd9c-edd6-4697-af62-60e4e6eb95c0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 22 15:47:00.398074 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application [None req-e274cd9c-edd6-4697-af62-60e4e6eb95c0 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 22 15:47:00.398074 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 15:47:00.398074 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 15:47:00.398074 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 15:47:00.398074 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 15:47:00.398074 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 15:47:00.398074 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 15:47:00.398074 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 15:47:00.398074 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 15:47:00.398074 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 15:47:00.398074 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 15:47:00.398074 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 15:47:00.398074 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 22 15:47:00.398074 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 22 15:47:00.398074 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 22 15:47:00.398074 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 22 15:47:00.398955 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 22 15:47:00.398955 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 22 15:47:00.398955 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 22 15:47:00.398955 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 22 15:47:00.398955 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 15:47:00.398955 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 15:47:00.398955 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 15:47:00.398955 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 15:47:00.398955 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 15:47:00.398955 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 15:47:00.398955 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 15:47:00.398955 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application with Lock( Aug 22 15:47:00.398955 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 15:47:00.398955 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self._enter() Aug 22 15:47:00.398955 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 15:47:00.398955 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 15:47:00.399789 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 15:47:00.399789 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self.creator() Aug 22 15:47:00.399789 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 15:47:00.399789 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application created_value = creator( Aug 22 15:47:00.399789 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 22 15:47:00.399789 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 22 15:47:00.399789 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 22 15:47:00.399789 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 22 15:47:00.399789 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 22 15:47:00.399789 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 22 15:47:00.399789 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 22 15:47:00.399789 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application Aug 22 15:47:00.403382 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 175/339] 10.222.0.15 () {56 vars in 1099 bytes} [Mon Aug 22 15:47:00 2022] GET /identity/v3/roles/member => generated 84 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 15:47:00.407335 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ab56047-2941-4d89-b133-2033c6d78a24 None None] Authenticating user token {{(pid=81767) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 15:47:00.568235 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ab56047-2941-4d89-b133-2033c6d78a24 None None] Validating token access rules against request {{(pid=81767) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 15:47:00.571657 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG dbcounter [-] [81767] Writing DB stats keystone:SELECT=100 {{(pid=81767) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 15:47:00.620631 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3201993b-036f-4b61-89cb-af11d21a6b3b None None] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:47:00.621509 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3201993b-036f-4b61-89cb-af11d21a6b3b None None] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:47:00.621745 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3201993b-036f-4b61-89cb-af11d21a6b3b None None] PATH_INFO: `/` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:47:00.625072 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cf51f81-2561-4471-bd65-1cf585fb5945 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81767) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 15:47:00.625711 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cf51f81-2561-4471-bd65-1cf585fb5945 None admin] REQUEST_METHOD: `GET` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:47:00.625825 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 176/340] 10.222.0.15 () {54 vars in 927 bytes} [Mon Aug 22 15:47:00 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 15:47:00.625987 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cf51f81-2561-4471-bd65-1cf585fb5945 None admin] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:47:00.626237 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cf51f81-2561-4471-bd65-1cf585fb5945 None admin] PATH_INFO: `/v3/roles` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:47:00.626939 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cf51f81-2561-4471-bd65-1cf585fb5945 None admin] RBAC: Adding query filter params (name=member) {{(pid=81767) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 15:47:00.627242 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cf51f81-2561-4471-bd65-1cf585fb5945 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 15:47:00.628658 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cf51f81-2561-4471-bd65-1cf585fb5945 None admin] RBAC: Authorization granted {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 15:47:00.632016 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f3b3ca3-4739-4aa4-a8f3-99825f1b66fe None None] REQUEST_METHOD: `POST` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:47:00.632352 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f3b3ca3-4739-4aa4-a8f3-99825f1b66fe None None] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:47:00.632567 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f3b3ca3-4739-4aa4-a8f3-99825f1b66fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:47:00.638334 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 165/341] 10.222.0.15 () {56 vars in 1101 bytes} [Mon Aug 22 15:47:00 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 15:47:00.649173 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cf51f81-2561-4471-bd65-1cf585fb5945 None admin] Authenticating user token {{(pid=81767) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 15:47:00.656612 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.auth.core [None req-1f3b3ca3-4739-4aa4-a8f3-99825f1b66fe None None] MFA Rules not processed for user `6b2199ae0fb0480aa917ada391f5ec54`. Rule list: `[]` (Enabled: `True`). {{(pid=81768) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 15:47:00.658118 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-1f3b3ca3-4739-4aa4-a8f3-99825f1b66fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 15:47:00.751677 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 177/342] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Aug 22 15:47:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 22 15:47:00.779564 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cf51f81-2561-4471-bd65-1cf585fb5945 None admin] Validating token access rules against request {{(pid=81767) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 15:47:00.798277 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f3b3ca3-4739-4aa4-a8f3-99825f1b66fe None None] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 15:47:00.827738 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15d6aff3-82cd-4eb2-930f-61def04b1d77 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81767) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 15:47:00.828331 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15d6aff3-82cd-4eb2-930f-61def04b1d77 None admin] REQUEST_METHOD: `GET` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:47:00.828611 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15d6aff3-82cd-4eb2-930f-61def04b1d77 None admin] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:47:00.828885 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15d6aff3-82cd-4eb2-930f-61def04b1d77 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:47:00.870781 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG dbcounter [-] [81767] Writing DB stats keystone:SELECT=100 {{(pid=81767) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 15:47:00.880755 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=100 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 15:47:00.921157 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15d6aff3-82cd-4eb2-930f-61def04b1d77 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 15:47:00.923165 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15d6aff3-82cd-4eb2-930f-61def04b1d77 None admin] RBAC: Authorization granted {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 15:47:01.047249 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 166/343] 10.222.0.15 () {58 vars in 1282 bytes} [Mon Aug 22 15:47:00 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 399 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 22 15:47:01.053658 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15d6aff3-82cd-4eb2-930f-61def04b1d77 None admin] Authenticating user token {{(pid=81767) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 15:47:01.065200 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f3b3ca3-4739-4aa4-a8f3-99825f1b66fe None None] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 15:47:01.129309 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG dbcounter [-] [81767] Writing DB stats keystone:SELECT=100 {{(pid=81767) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 15:47:01.151997 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1dbb4c1-d914-401e-871c-3c50b4cc6907 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 15:47:01.153209 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1dbb4c1-d914-401e-871c-3c50b4cc6907 None admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:47:01.153490 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1dbb4c1-d914-401e-871c-3c50b4cc6907 None admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:47:01.153762 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1dbb4c1-d914-401e-871c-3c50b4cc6907 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:47:01.171761 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1dbb4c1-d914-401e-871c-3c50b4cc6907 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 22 15:47:01.173698 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15d6aff3-82cd-4eb2-930f-61def04b1d77 None admin] Validating token access rules against request {{(pid=81767) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 15:47:01.178543 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application [None req-f1dbb4c1-d914-401e-871c-3c50b4cc6907 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 22 15:47:01.178543 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 15:47:01.178543 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 15:47:01.178543 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 15:47:01.178543 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 15:47:01.178543 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 15:47:01.178543 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 15:47:01.178543 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 15:47:01.178543 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 15:47:01.178543 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 15:47:01.178543 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 15:47:01.178543 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 15:47:01.178543 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 22 15:47:01.178543 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 22 15:47:01.178543 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 22 15:47:01.178543 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 22 15:47:01.179517 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 22 15:47:01.179517 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 22 15:47:01.179517 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 22 15:47:01.179517 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 22 15:47:01.179517 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 15:47:01.179517 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 15:47:01.179517 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 15:47:01.179517 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 15:47:01.179517 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 15:47:01.179517 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 15:47:01.179517 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 15:47:01.179517 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application with Lock( Aug 22 15:47:01.179517 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 15:47:01.179517 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self._enter() Aug 22 15:47:01.179517 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 15:47:01.179517 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 15:47:01.180340 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 15:47:01.180340 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self.creator() Aug 22 15:47:01.180340 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 15:47:01.180340 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application created_value = creator( Aug 22 15:47:01.180340 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 22 15:47:01.180340 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 22 15:47:01.180340 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 22 15:47:01.180340 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 22 15:47:01.180340 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 22 15:47:01.180340 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 22 15:47:01.180340 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 22 15:47:01.180340 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application Aug 22 15:47:01.180340 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 178/344] 10.222.0.15 () {56 vars in 1114 bytes} [Mon Aug 22 15:47:00 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 382 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 22 15:47:01.188095 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1dbb4c1-d914-401e-871c-3c50b4cc6907 None admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 15:47:01.232701 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6659cf85-51a4-43e4-9673-d276ceb9f497 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81767) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 15:47:01.233726 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6659cf85-51a4-43e4-9673-d276ceb9f497 None admin] REQUEST_METHOD: `GET` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:47:01.234219 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6659cf85-51a4-43e4-9673-d276ceb9f497 None admin] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:47:01.234819 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6659cf85-51a4-43e4-9673-d276ceb9f497 None admin] PATH_INFO: `/v3/projects/43171f5fd3454380b88625276d7375da` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:47:01.236385 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6659cf85-51a4-43e4-9673-d276ceb9f497 None admin] RBAC: Authorizing `identity:get_project(project_id=43171f5fd3454380b88625276d7375da)` {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 15:47:01.238282 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6659cf85-51a4-43e4-9673-d276ceb9f497 None admin] RBAC: Authorization granted {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 15:47:01.240556 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 167/345] 10.222.0.15 () {56 vars in 1186 bytes} [Mon Aug 22 15:47:01 2022] GET /identity/v3/projects/43171f5fd3454380b88625276d7375da => generated 298 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 15:47:01.246898 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6659cf85-51a4-43e4-9673-d276ceb9f497 None admin] Authenticating user token {{(pid=81767) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 15:47:01.329826 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=100 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 15:47:01.338193 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1dbb4c1-d914-401e-871c-3c50b4cc6907 None admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 15:47:01.393055 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6659cf85-51a4-43e4-9673-d276ceb9f497 None admin] Validating token access rules against request {{(pid=81767) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 15:47:01.393231 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77df8048-9461-4ff4-824e-a0501e69a11e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 15:47:01.393715 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77df8048-9461-4ff4-824e-a0501e69a11e None admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:47:01.394089 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77df8048-9461-4ff4-824e-a0501e69a11e None admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:47:01.394506 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77df8048-9461-4ff4-824e-a0501e69a11e None admin] PATH_INFO: `/v3/roles` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:47:01.395808 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77df8048-9461-4ff4-824e-a0501e69a11e None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81768) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 15:47:01.396207 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77df8048-9461-4ff4-824e-a0501e69a11e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 15:47:01.398830 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77df8048-9461-4ff4-824e-a0501e69a11e None admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 15:47:01.412775 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 179/346] 10.222.0.15 () {56 vars in 1111 bytes} [Mon Aug 22 15:47:01 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 15:47:01.413644 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77df8048-9461-4ff4-824e-a0501e69a11e None admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 15:47:01.435823 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG dbcounter [-] [81767] Writing DB stats keystone:SELECT=100 {{(pid=81767) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 15:47:01.447454 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0efd9c4c-ab53-42c3-bab3-1e4ec83fb635 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81767) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 15:47:01.448557 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0efd9c4c-ab53-42c3-bab3-1e4ec83fb635 None admin] REQUEST_METHOD: `GET` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:47:01.449593 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0efd9c4c-ab53-42c3-bab3-1e4ec83fb635 None admin] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:47:01.450133 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0efd9c4c-ab53-42c3-bab3-1e4ec83fb635 None admin] PATH_INFO: `/v3/groups/552e4a454dcf434ba77a8567804e8230` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:47:01.457450 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0efd9c4c-ab53-42c3-bab3-1e4ec83fb635 None admin] RBAC: Authorizing `identity:get_group(group_id=552e4a454dcf434ba77a8567804e8230)` {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 15:47:01.459259 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0efd9c4c-ab53-42c3-bab3-1e4ec83fb635 None admin] RBAC: Authorization granted {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 15:47:01.461393 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 168/347] 10.222.0.15 () {56 vars in 1180 bytes} [Mon Aug 22 15:47:01 2022] GET /identity/v3/groups/552e4a454dcf434ba77a8567804e8230 => generated 226 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 22 15:47:01.462361 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0efd9c4c-ab53-42c3-bab3-1e4ec83fb635 None admin] Authenticating user token {{(pid=81767) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 15:47:01.546710 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77df8048-9461-4ff4-824e-a0501e69a11e None admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 15:47:01.584234 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0efd9c4c-ab53-42c3-bab3-1e4ec83fb635 None admin] Validating token access rules against request {{(pid=81767) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 15:47:01.588541 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a951613-162e-4fff-9167-641afca99c4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 15:47:01.589202 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a951613-162e-4fff-9167-641afca99c4a None admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:47:01.589484 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a951613-162e-4fff-9167-641afca99c4a None admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:47:01.590325 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a951613-162e-4fff-9167-641afca99c4a None admin] PATH_INFO: `/v3/roles/member` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:47:01.602654 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a951613-162e-4fff-9167-641afca99c4a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 22 15:47:01.608403 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application [None req-3a951613-162e-4fff-9167-641afca99c4a None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 22 15:47:01.608403 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 15:47:01.608403 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 15:47:01.608403 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 15:47:01.608403 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 15:47:01.608403 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 15:47:01.608403 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 15:47:01.608403 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 15:47:01.608403 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 15:47:01.608403 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 15:47:01.608403 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 15:47:01.608403 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 15:47:01.608403 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 22 15:47:01.608403 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 22 15:47:01.608403 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 22 15:47:01.608403 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 22 15:47:01.609492 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 22 15:47:01.609492 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 22 15:47:01.609492 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 22 15:47:01.609492 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 22 15:47:01.609492 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 15:47:01.609492 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 15:47:01.609492 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 15:47:01.609492 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 15:47:01.609492 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 15:47:01.609492 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 15:47:01.609492 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 15:47:01.609492 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application with Lock( Aug 22 15:47:01.609492 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 15:47:01.609492 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self._enter() Aug 22 15:47:01.609492 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 15:47:01.609492 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 15:47:01.610618 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 15:47:01.610618 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self.creator() Aug 22 15:47:01.610618 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 15:47:01.610618 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application created_value = creator( Aug 22 15:47:01.610618 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 22 15:47:01.610618 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 22 15:47:01.610618 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 22 15:47:01.610618 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 22 15:47:01.610618 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 22 15:47:01.610618 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 22 15:47:01.610618 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 22 15:47:01.610618 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application Aug 22 15:47:01.610618 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 180/348] 10.222.0.15 () {56 vars in 1099 bytes} [Mon Aug 22 15:47:01 2022] GET /identity/v3/roles/member => generated 84 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 15:47:01.611444 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a951613-162e-4fff-9167-641afca99c4a None admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 15:47:01.628703 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=100 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 15:47:01.633869 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b033e216-0893-4776-91d2-e407c4e905ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81767) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 15:47:01.636340 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b033e216-0893-4776-91d2-e407c4e905ff None admin] REQUEST_METHOD: `GET` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:47:01.641504 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b033e216-0893-4776-91d2-e407c4e905ff None admin] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:47:01.641504 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b033e216-0893-4776-91d2-e407c4e905ff None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:47:01.730384 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG dbcounter [-] [81767] Writing DB stats keystone:SELECT=100 {{(pid=81767) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 15:47:01.744800 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b033e216-0893-4776-91d2-e407c4e905ff None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 15:47:01.746808 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b033e216-0893-4776-91d2-e407c4e905ff None admin] RBAC: Authorization granted {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 15:47:01.764599 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a951613-162e-4fff-9167-641afca99c4a None admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 15:47:01.826583 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27ed24be-7e0e-421e-9f80-311d4f356a78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 15:47:01.828033 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27ed24be-7e0e-421e-9f80-311d4f356a78 None admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:47:01.828530 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27ed24be-7e0e-421e-9f80-311d4f356a78 None admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:47:01.828993 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27ed24be-7e0e-421e-9f80-311d4f356a78 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:47:01.829936 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27ed24be-7e0e-421e-9f80-311d4f356a78 None admin] RBAC: Adding query filter params (group.id=552e4a454dcf434ba77a8567804e8230, role.id=e6fc1ee3fb974125ba9b4fe55b12b45e, scope.project.id=43171f5fd3454380b88625276d7375da) {{(pid=81768) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 15:47:01.830438 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27ed24be-7e0e-421e-9f80-311d4f356a78 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 15:47:01.832120 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27ed24be-7e0e-421e-9f80-311d4f356a78 None admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 15:47:01.838819 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 181/349] 10.222.0.15 () {56 vars in 1376 bytes} [Mon Aug 22 15:47:01 2022] GET /identity/v3/role_assignments?group.id=552e4a454dcf434ba77a8567804e8230&scope.project.id=43171f5fd3454380b88625276d7375da&role.id=e6fc1ee3fb974125ba9b4fe55b12b45e => generated 260 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 15:47:01.840086 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27ed24be-7e0e-421e-9f80-311d4f356a78 None admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 15:47:01.894854 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 169/350] 10.222.0.15 () {58 vars in 1282 bytes} [Mon Aug 22 15:47:01 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 433 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 22 15:47:01.897198 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-209849d9-0f37-44e2-802e-49950c2dd03a None None] REQUEST_METHOD: `GET` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:47:01.897490 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-209849d9-0f37-44e2-802e-49950c2dd03a None None] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:47:01.897760 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-209849d9-0f37-44e2-802e-49950c2dd03a None None] PATH_INFO: `/` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:47:01.899601 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 170/351] 10.222.0.15 () {54 vars in 927 bytes} [Mon Aug 22 15:47:01 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 15:47:01.905726 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6599aea4-2c94-4b51-9b8a-886f30f7794c None None] REQUEST_METHOD: `POST` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:47:01.906162 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6599aea4-2c94-4b51-9b8a-886f30f7794c None None] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:47:01.906294 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6599aea4-2c94-4b51-9b8a-886f30f7794c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:47:01.946682 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.auth.core [None req-6599aea4-2c94-4b51-9b8a-886f30f7794c None None] MFA Rules not processed for user `6b2199ae0fb0480aa917ada391f5ec54`. Rule list: `[]` (Enabled: `True`). {{(pid=81767) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 15:47:01.947927 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.fernet_utils [None req-6599aea4-2c94-4b51-9b8a-886f30f7794c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81767) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 15:47:02.010410 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=100 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 15:47:02.051949 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27ed24be-7e0e-421e-9f80-311d4f356a78 None admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 15:47:02.072338 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 171/352] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Aug 22 15:47:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 22 15:47:02.073924 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6599aea4-2c94-4b51-9b8a-886f30f7794c None None] Authenticating user token {{(pid=81767) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 15:47:02.092886 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0184a39-6184-4c68-b1a1-a3806bafc584 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 15:47:02.093504 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0184a39-6184-4c68-b1a1-a3806bafc584 None admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:47:02.093785 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0184a39-6184-4c68-b1a1-a3806bafc584 None admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:47:02.094060 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0184a39-6184-4c68-b1a1-a3806bafc584 None admin] PATH_INFO: `/v3/roles` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:47:02.094729 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0184a39-6184-4c68-b1a1-a3806bafc584 None admin] RBAC: Adding query filter params (name=member) {{(pid=81768) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 15:47:02.095044 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0184a39-6184-4c68-b1a1-a3806bafc584 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 15:47:02.096793 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0184a39-6184-4c68-b1a1-a3806bafc584 None admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 15:47:02.110739 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 182/353] 10.222.0.15 () {56 vars in 1101 bytes} [Mon Aug 22 15:47:01 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 267 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 15:47:02.110739 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb3d7da8-b1ae-4b3e-9cb8-c24314b95aa9 None None] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:47:02.110739 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb3d7da8-b1ae-4b3e-9cb8-c24314b95aa9 None None] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:47:02.110739 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb3d7da8-b1ae-4b3e-9cb8-c24314b95aa9 None None] PATH_INFO: `/` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:47:02.111127 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG dbcounter [-] [81767] Writing DB stats keystone:SELECT=100 {{(pid=81767) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 15:47:02.111199 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 183/354] 10.222.0.15 () {54 vars in 927 bytes} [Mon Aug 22 15:47:02 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 15:47:02.112149 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-681f5d69-0eae-45c3-923f-4f6aed1b94b0 None None] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:47:02.112429 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-681f5d69-0eae-45c3-923f-4f6aed1b94b0 None None] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:47:02.112698 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-681f5d69-0eae-45c3-923f-4f6aed1b94b0 None None] PATH_INFO: `/` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:47:02.114224 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 184/355] 10.222.0.15 () {54 vars in 927 bytes} [Mon Aug 22 15:47:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 15:47:02.114936 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-681f5d69-0eae-45c3-923f-4f6aed1b94b0 None None] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 15:47:02.213597 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6599aea4-2c94-4b51-9b8a-886f30f7794c None None] Validating token access rules against request {{(pid=81767) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 15:47:02.231202 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-681f5d69-0eae-45c3-923f-4f6aed1b94b0 None None] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 15:47:02.254450 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cad899bc-e71b-4f7b-b77b-d482039d0ba6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81767) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 15:47:02.255509 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cad899bc-e71b-4f7b-b77b-d482039d0ba6 None admin] REQUEST_METHOD: `GET` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:47:02.256002 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cad899bc-e71b-4f7b-b77b-d482039d0ba6 None admin] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:47:02.256470 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cad899bc-e71b-4f7b-b77b-d482039d0ba6 None admin] PATH_INFO: `/v3/projects/43171f5fd3454380b88625276d7375da` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:47:02.257992 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cad899bc-e71b-4f7b-b77b-d482039d0ba6 None admin] RBAC: Authorizing `identity:get_project(project_id=43171f5fd3454380b88625276d7375da)` {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 15:47:02.259913 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cad899bc-e71b-4f7b-b77b-d482039d0ba6 None admin] RBAC: Authorization granted {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 15:47:02.262522 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 172/356] 10.222.0.15 () {56 vars in 1186 bytes} [Mon Aug 22 15:47:02 2022] GET /identity/v3/projects/43171f5fd3454380b88625276d7375da => generated 298 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 15:47:02.264044 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07b71764-190f-4297-a385-a8e619abb901 None None] REQUEST_METHOD: `POST` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:47:02.264746 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07b71764-190f-4297-a385-a8e619abb901 None None] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:47:02.265280 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07b71764-190f-4297-a385-a8e619abb901 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:47:02.270128 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-858262a7-7a83-45fb-90b8-6611ed9ab33c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 15:47:02.270729 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-858262a7-7a83-45fb-90b8-6611ed9ab33c None admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:47:02.271008 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-858262a7-7a83-45fb-90b8-6611ed9ab33c None admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:47:02.271312 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-858262a7-7a83-45fb-90b8-6611ed9ab33c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:47:02.275722 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=100 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 15:47:02.289307 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.auth.core [None req-07b71764-190f-4297-a385-a8e619abb901 None None] MFA Rules not processed for user `6b2199ae0fb0480aa917ada391f5ec54`. Rule list: `[]` (Enabled: `True`). {{(pid=81767) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 15:47:02.291300 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.fernet_utils [None req-07b71764-190f-4297-a385-a8e619abb901 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81767) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 15:47:02.350162 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-858262a7-7a83-45fb-90b8-6611ed9ab33c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 15:47:02.351878 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-858262a7-7a83-45fb-90b8-6611ed9ab33c None admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 15:47:02.368268 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 173/357] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Aug 22 15:47:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 22 15:47:02.369648 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c4f2156-dfdf-465b-a1f8-f6e83fdfe70a None None] REQUEST_METHOD: `POST` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:47:02.369998 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c4f2156-dfdf-465b-a1f8-f6e83fdfe70a None None] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:47:02.370275 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c4f2156-dfdf-465b-a1f8-f6e83fdfe70a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:47:02.390476 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.auth.core [None req-8c4f2156-dfdf-465b-a1f8-f6e83fdfe70a None None] MFA Rules not processed for user `6b2199ae0fb0480aa917ada391f5ec54`. Rule list: `[]` (Enabled: `True`). {{(pid=81767) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 15:47:02.391665 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.fernet_utils [None req-8c4f2156-dfdf-465b-a1f8-f6e83fdfe70a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81767) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 15:47:02.415393 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG dbcounter [-] [81767] Writing DB stats keystone:SELECT=100 {{(pid=81767) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 15:47:02.469364 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 174/358] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Aug 22 15:47:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 22 15:47:02.470228 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c4f2156-dfdf-465b-a1f8-f6e83fdfe70a None None] Authenticating user token {{(pid=81767) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 15:47:02.472137 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 185/359] 10.222.0.15 () {58 vars in 1282 bytes} [Mon Aug 22 15:47:02 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 357 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 22 15:47:02.473547 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-858262a7-7a83-45fb-90b8-6611ed9ab33c None admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 15:47:02.512892 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=100 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 15:47:02.604628 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-858262a7-7a83-45fb-90b8-6611ed9ab33c None admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 15:47:02.604836 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c4f2156-dfdf-465b-a1f8-f6e83fdfe70a None None] Validating token access rules against request {{(pid=81767) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 15:47:02.644863 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12614e21-0c60-4fd7-9ced-fff753309834 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81767) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 15:47:02.645831 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12614e21-0c60-4fd7-9ced-fff753309834 None admin] REQUEST_METHOD: `GET` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:47:02.645972 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12614e21-0c60-4fd7-9ced-fff753309834 None admin] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:47:02.646199 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12614e21-0c60-4fd7-9ced-fff753309834 None admin] PATH_INFO: `/v3/groups/552e4a454dcf434ba77a8567804e8230` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:47:02.651307 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12614e21-0c60-4fd7-9ced-fff753309834 None admin] RBAC: Authorizing `identity:get_group(group_id=552e4a454dcf434ba77a8567804e8230)` {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 15:47:02.651307 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12614e21-0c60-4fd7-9ced-fff753309834 None admin] RBAC: Authorization granted {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 15:47:02.651307 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 175/360] 10.222.0.15 () {56 vars in 1180 bytes} [Mon Aug 22 15:47:02 2022] GET /identity/v3/groups/552e4a454dcf434ba77a8567804e8230 => generated 226 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 15:47:02.651692 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12614e21-0c60-4fd7-9ced-fff753309834 None admin] Authenticating user token {{(pid=81767) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 15:47:02.663071 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bd0c5c7-991a-4057-ace8-eda1c6c1d3ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 15:47:02.664070 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bd0c5c7-991a-4057-ace8-eda1c6c1d3ac None admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:47:02.664509 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bd0c5c7-991a-4057-ace8-eda1c6c1d3ac None admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:47:02.664932 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bd0c5c7-991a-4057-ace8-eda1c6c1d3ac None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:47:02.679642 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG dbcounter [-] [81767] Writing DB stats keystone:SELECT=100 {{(pid=81767) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 15:47:02.683081 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bd0c5c7-991a-4057-ace8-eda1c6c1d3ac None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 22 15:47:02.688177 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application [None req-3bd0c5c7-991a-4057-ace8-eda1c6c1d3ac None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 22 15:47:02.688177 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 15:47:02.688177 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 15:47:02.688177 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 15:47:02.688177 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 15:47:02.688177 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 15:47:02.688177 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 15:47:02.688177 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 15:47:02.688177 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 15:47:02.688177 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 15:47:02.688177 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 15:47:02.688177 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 15:47:02.688177 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 22 15:47:02.688177 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 22 15:47:02.688177 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 22 15:47:02.688177 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 22 15:47:02.689354 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 22 15:47:02.689354 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 22 15:47:02.689354 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 22 15:47:02.689354 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 22 15:47:02.689354 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 15:47:02.689354 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 15:47:02.689354 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 15:47:02.689354 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 15:47:02.689354 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 15:47:02.689354 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 15:47:02.689354 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 15:47:02.689354 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application with Lock( Aug 22 15:47:02.689354 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 15:47:02.689354 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self._enter() Aug 22 15:47:02.689354 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 15:47:02.689354 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 15:47:02.690304 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 15:47:02.690304 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self.creator() Aug 22 15:47:02.690304 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 15:47:02.690304 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application created_value = creator( Aug 22 15:47:02.690304 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 22 15:47:02.690304 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 22 15:47:02.690304 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 22 15:47:02.690304 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 22 15:47:02.690304 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 22 15:47:02.690304 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 22 15:47:02.690304 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 22 15:47:02.690304 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application Aug 22 15:47:02.698131 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 186/361] 10.222.0.15 () {56 vars in 1099 bytes} [Mon Aug 22 15:47:02 2022] GET /identity/v3/roles/reader => generated 84 bytes in 225 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 15:47:02.699199 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bd0c5c7-991a-4057-ace8-eda1c6c1d3ac None admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 15:47:02.772466 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12614e21-0c60-4fd7-9ced-fff753309834 None admin] Validating token access rules against request {{(pid=81767) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 15:47:02.814319 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a7c1760-9e60-49ad-90bf-ae7424e33c21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81767) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 15:47:02.815004 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a7c1760-9e60-49ad-90bf-ae7424e33c21 None admin] REQUEST_METHOD: `GET` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:47:02.815299 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a7c1760-9e60-49ad-90bf-ae7424e33c21 None admin] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:47:02.815582 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a7c1760-9e60-49ad-90bf-ae7424e33c21 None admin] PATH_INFO: `/v3/projects/c05dae9fd0dc44f9a2ab2d373bbc913f` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:47:02.816905 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a7c1760-9e60-49ad-90bf-ae7424e33c21 None admin] RBAC: Authorizing `identity:get_project(project_id=c05dae9fd0dc44f9a2ab2d373bbc913f)` {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 15:47:02.818464 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a7c1760-9e60-49ad-90bf-ae7424e33c21 None admin] RBAC: Authorization granted {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 15:47:02.820350 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 176/362] 10.222.0.15 () {56 vars in 1186 bytes} [Mon Aug 22 15:47:02 2022] GET /identity/v3/projects/c05dae9fd0dc44f9a2ab2d373bbc913f => generated 302 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 15:47:02.821393 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a7c1760-9e60-49ad-90bf-ae7424e33c21 None admin] Authenticating user token {{(pid=81767) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 15:47:02.832999 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=100 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 15:47:02.839716 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bd0c5c7-991a-4057-ace8-eda1c6c1d3ac None admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 15:47:02.882242 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1216f91e-92ed-46ea-afb1-f30b55048c95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 15:47:02.882853 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1216f91e-92ed-46ea-afb1-f30b55048c95 None admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:47:02.883133 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1216f91e-92ed-46ea-afb1-f30b55048c95 None admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:47:02.883435 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1216f91e-92ed-46ea-afb1-f30b55048c95 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:47:02.884236 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1216f91e-92ed-46ea-afb1-f30b55048c95 None admin] RBAC: Adding query filter params (group.id=552e4a454dcf434ba77a8567804e8230, role.id=1ce6b924c9604048906c07dbfb126da9, scope.project.id=43171f5fd3454380b88625276d7375da) {{(pid=81768) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 15:47:02.884538 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1216f91e-92ed-46ea-afb1-f30b55048c95 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 15:47:02.886009 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1216f91e-92ed-46ea-afb1-f30b55048c95 None admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 15:47:02.893133 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 187/363] 10.222.0.15 () {56 vars in 1376 bytes} [Mon Aug 22 15:47:02 2022] GET /identity/v3/role_assignments?group.id=552e4a454dcf434ba77a8567804e8230&scope.project.id=43171f5fd3454380b88625276d7375da&role.id=1ce6b924c9604048906c07dbfb126da9 => generated 260 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 15:47:02.893800 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1216f91e-92ed-46ea-afb1-f30b55048c95 None admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 15:47:02.958161 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG dbcounter [-] [81767] Writing DB stats keystone:SELECT=100 {{(pid=81767) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 15:47:02.971368 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a7c1760-9e60-49ad-90bf-ae7424e33c21 None admin] Validating token access rules against request {{(pid=81767) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 15:47:03.024690 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af17acc4-a44f-4c97-b5ab-a2c38da2f424 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81767) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 15:47:03.025229 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af17acc4-a44f-4c97-b5ab-a2c38da2f424 None admin] REQUEST_METHOD: `GET` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:47:03.025505 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af17acc4-a44f-4c97-b5ab-a2c38da2f424 None admin] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:47:03.025774 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af17acc4-a44f-4c97-b5ab-a2c38da2f424 None admin] PATH_INFO: `/v3/roles` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:47:03.026450 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af17acc4-a44f-4c97-b5ab-a2c38da2f424 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81767) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 15:47:03.027138 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af17acc4-a44f-4c97-b5ab-a2c38da2f424 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 15:47:03.028387 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af17acc4-a44f-4c97-b5ab-a2c38da2f424 None admin] RBAC: Authorization granted {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 15:47:03.040219 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 177/364] 10.222.0.15 () {56 vars in 1101 bytes} [Mon Aug 22 15:47:02 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 15:47:03.041085 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af17acc4-a44f-4c97-b5ab-a2c38da2f424 None admin] Authenticating user token {{(pid=81767) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 15:47:03.062650 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1216f91e-92ed-46ea-afb1-f30b55048c95 None admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 15:47:03.106941 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-095137c5-c7c2-4f25-9069-31718700495b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 15:47:03.107750 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-095137c5-c7c2-4f25-9069-31718700495b None admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:47:03.108052 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-095137c5-c7c2-4f25-9069-31718700495b None admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:47:03.108396 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-095137c5-c7c2-4f25-9069-31718700495b None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:47:03.120165 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-095137c5-c7c2-4f25-9069-31718700495b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 22 15:47:03.125420 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application [None req-095137c5-c7c2-4f25-9069-31718700495b None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 22 15:47:03.125420 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 15:47:03.125420 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 15:47:03.125420 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 15:47:03.125420 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 15:47:03.125420 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 15:47:03.125420 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 15:47:03.125420 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 15:47:03.125420 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 15:47:03.125420 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 15:47:03.125420 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 15:47:03.125420 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 15:47:03.125420 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 22 15:47:03.125420 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 22 15:47:03.125420 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 22 15:47:03.125420 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 22 15:47:03.126327 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 22 15:47:03.126327 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 22 15:47:03.126327 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 22 15:47:03.126327 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 22 15:47:03.126327 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 15:47:03.126327 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 15:47:03.126327 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 15:47:03.126327 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 15:47:03.126327 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 15:47:03.126327 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 15:47:03.126327 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 15:47:03.126327 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application with Lock( Aug 22 15:47:03.126327 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 15:47:03.126327 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self._enter() Aug 22 15:47:03.126327 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 15:47:03.126327 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 15:47:03.127100 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 15:47:03.127100 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self.creator() Aug 22 15:47:03.127100 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 15:47:03.127100 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application created_value = creator( Aug 22 15:47:03.127100 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 22 15:47:03.127100 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 22 15:47:03.127100 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 22 15:47:03.127100 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 22 15:47:03.127100 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 22 15:47:03.127100 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 22 15:47:03.127100 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 22 15:47:03.127100 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application Aug 22 15:47:03.127100 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 188/365] 10.222.0.15 () {56 vars in 1114 bytes} [Mon Aug 22 15:47:02 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 233 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 15:47:03.127790 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-095137c5-c7c2-4f25-9069-31718700495b None admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 15:47:03.153745 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=100 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 15:47:03.166986 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af17acc4-a44f-4c97-b5ab-a2c38da2f424 None admin] Validating token access rules against request {{(pid=81767) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 15:47:03.221121 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d768c16-cebb-4393-859b-271fe739805f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81767) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 15:47:03.221878 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d768c16-cebb-4393-859b-271fe739805f None admin] REQUEST_METHOD: `GET` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:47:03.222160 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d768c16-cebb-4393-859b-271fe739805f None admin] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:47:03.222452 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d768c16-cebb-4393-859b-271fe739805f None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:47:03.236164 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d768c16-cebb-4393-859b-271fe739805f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 22 15:47:03.241475 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application [None req-8d768c16-cebb-4393-859b-271fe739805f None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 22 15:47:03.241475 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 15:47:03.241475 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 15:47:03.241475 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 15:47:03.241475 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 15:47:03.241475 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 15:47:03.241475 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 15:47:03.241475 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 15:47:03.241475 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 15:47:03.241475 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 15:47:03.241475 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 15:47:03.241475 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 15:47:03.241475 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 22 15:47:03.241475 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 22 15:47:03.241475 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 22 15:47:03.241475 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 22 15:47:03.242379 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 22 15:47:03.242379 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 22 15:47:03.242379 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 22 15:47:03.242379 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 22 15:47:03.242379 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 15:47:03.242379 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 15:47:03.242379 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 15:47:03.242379 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 15:47:03.242379 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 15:47:03.242379 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 15:47:03.242379 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 15:47:03.242379 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application with Lock( Aug 22 15:47:03.242379 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 15:47:03.242379 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return self._enter() Aug 22 15:47:03.242379 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 15:47:03.242379 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 15:47:03.243221 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 15:47:03.243221 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return self.creator() Aug 22 15:47:03.243221 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 15:47:03.243221 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application created_value = creator( Aug 22 15:47:03.243221 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 22 15:47:03.243221 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 22 15:47:03.243221 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 22 15:47:03.243221 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 22 15:47:03.243221 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 22 15:47:03.243221 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 22 15:47:03.243221 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 22 15:47:03.243221 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application Aug 22 15:47:03.243221 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 178/366] 10.222.0.15 () {56 vars in 1096 bytes} [Mon Aug 22 15:47:03 2022] GET /identity/v3/roles/admin => generated 83 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 15:47:03.244084 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d768c16-cebb-4393-859b-271fe739805f None admin] Authenticating user token {{(pid=81767) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 15:47:03.253502 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-095137c5-c7c2-4f25-9069-31718700495b None admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 15:47:03.261380 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG dbcounter [-] [81767] Writing DB stats keystone:SELECT=100 {{(pid=81767) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 15:47:03.294705 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a2fb0f0-4bdf-4dcc-ac3f-07f81b5f1e71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 15:47:03.295369 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a2fb0f0-4bdf-4dcc-ac3f-07f81b5f1e71 None admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:47:03.295705 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a2fb0f0-4bdf-4dcc-ac3f-07f81b5f1e71 None admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:47:03.295975 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a2fb0f0-4bdf-4dcc-ac3f-07f81b5f1e71 None admin] PATH_INFO: `/v3/groups/552e4a454dcf434ba77a8567804e8230` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:47:03.297128 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a2fb0f0-4bdf-4dcc-ac3f-07f81b5f1e71 None admin] RBAC: Authorizing `identity:get_group(group_id=552e4a454dcf434ba77a8567804e8230)` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 15:47:03.298683 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a2fb0f0-4bdf-4dcc-ac3f-07f81b5f1e71 None admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 15:47:03.302206 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 189/367] 10.222.0.15 () {56 vars in 1180 bytes} [Mon Aug 22 15:47:03 2022] GET /identity/v3/groups/552e4a454dcf434ba77a8567804e8230 => generated 226 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 15:47:03.303114 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a2fb0f0-4bdf-4dcc-ac3f-07f81b5f1e71 None admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 15:47:03.385766 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d768c16-cebb-4393-859b-271fe739805f None admin] Validating token access rules against request {{(pid=81767) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 15:47:03.418590 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=100 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 15:47:03.423151 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3597446d-c5fb-4925-b112-4ab035f5ef06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81767) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 15:47:03.423883 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3597446d-c5fb-4925-b112-4ab035f5ef06 None admin] REQUEST_METHOD: `GET` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:47:03.424163 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3597446d-c5fb-4925-b112-4ab035f5ef06 None admin] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:47:03.424471 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3597446d-c5fb-4925-b112-4ab035f5ef06 None admin] PATH_INFO: `/v3/users/68b4a39eda4d43cb8b69acf6c3bbdc9d` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:47:03.425635 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3597446d-c5fb-4925-b112-4ab035f5ef06 None admin] RBAC: Authorizing `identity:get_user(user_id=68b4a39eda4d43cb8b69acf6c3bbdc9d)` {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 15:47:03.427061 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3597446d-c5fb-4925-b112-4ab035f5ef06 None admin] RBAC: Authorization granted {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 15:47:03.428892 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 179/368] 10.222.0.15 () {56 vars in 1177 bytes} [Mon Aug 22 15:47:03 2022] GET /identity/v3/users/68b4a39eda4d43cb8b69acf6c3bbdc9d => generated 293 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 15:47:03.430708 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3597446d-c5fb-4925-b112-4ab035f5ef06 None admin] Authenticating user token {{(pid=81767) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 15:47:03.438776 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a2fb0f0-4bdf-4dcc-ac3f-07f81b5f1e71 None admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 15:47:03.473552 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a27fa4fc-2a45-4b8f-9ea7-7f12fab0f57c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 15:47:03.474524 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a27fa4fc-2a45-4b8f-9ea7-7f12fab0f57c None admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:47:03.475000 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a27fa4fc-2a45-4b8f-9ea7-7f12fab0f57c None admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:47:03.475537 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a27fa4fc-2a45-4b8f-9ea7-7f12fab0f57c None admin] PATH_INFO: `/v3/roles` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:47:03.476401 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a27fa4fc-2a45-4b8f-9ea7-7f12fab0f57c None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81768) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 15:47:03.476909 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a27fa4fc-2a45-4b8f-9ea7-7f12fab0f57c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 15:47:03.478534 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a27fa4fc-2a45-4b8f-9ea7-7f12fab0f57c None admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 15:47:03.489108 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 190/369] 10.222.0.15 () {56 vars in 1111 bytes} [Mon Aug 22 15:47:03 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 15:47:03.490240 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a27fa4fc-2a45-4b8f-9ea7-7f12fab0f57c None admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 15:47:03.524109 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG dbcounter [-] [81767] Writing DB stats keystone:SELECT=100 {{(pid=81767) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 15:47:03.545855 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3597446d-c5fb-4925-b112-4ab035f5ef06 None admin] Validating token access rules against request {{(pid=81767) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 15:47:03.599372 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b110a588-d05c-48b4-a18e-c3247682dd47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81767) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 15:47:03.599993 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b110a588-d05c-48b4-a18e-c3247682dd47 None admin] REQUEST_METHOD: `GET` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:47:03.600270 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b110a588-d05c-48b4-a18e-c3247682dd47 None admin] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:47:03.600555 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b110a588-d05c-48b4-a18e-c3247682dd47 None admin] PATH_INFO: `/v3/roles` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:47:03.601338 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b110a588-d05c-48b4-a18e-c3247682dd47 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81767) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 15:47:03.603354 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b110a588-d05c-48b4-a18e-c3247682dd47 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 15:47:03.604676 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b110a588-d05c-48b4-a18e-c3247682dd47 None admin] RBAC: Authorization granted {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 15:47:03.614854 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 180/370] 10.222.0.15 () {56 vars in 1099 bytes} [Mon Aug 22 15:47:03 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 15:47:03.615828 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b110a588-d05c-48b4-a18e-c3247682dd47 None admin] Authenticating user token {{(pid=81767) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 15:47:03.617209 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a27fa4fc-2a45-4b8f-9ea7-7f12fab0f57c None admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 15:47:03.664414 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-192fc1c9-1045-4601-99fa-053114379ad0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 15:47:03.665072 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-192fc1c9-1045-4601-99fa-053114379ad0 None admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:47:03.665449 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-192fc1c9-1045-4601-99fa-053114379ad0 None admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:47:03.665779 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-192fc1c9-1045-4601-99fa-053114379ad0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:47:03.666520 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-192fc1c9-1045-4601-99fa-053114379ad0 None admin] RBAC: Adding query filter params (group.id=552e4a454dcf434ba77a8567804e8230, role.id=e6fc1ee3fb974125ba9b4fe55b12b45e, scope.project.id=c05dae9fd0dc44f9a2ab2d373bbc913f) {{(pid=81768) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 15:47:03.666878 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-192fc1c9-1045-4601-99fa-053114379ad0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 15:47:03.669507 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-192fc1c9-1045-4601-99fa-053114379ad0 None admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 15:47:03.677675 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 191/371] 10.222.0.15 () {56 vars in 1376 bytes} [Mon Aug 22 15:47:03 2022] GET /identity/v3/role_assignments?group.id=552e4a454dcf434ba77a8567804e8230&scope.project.id=c05dae9fd0dc44f9a2ab2d373bbc913f&role.id=e6fc1ee3fb974125ba9b4fe55b12b45e => generated 260 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 15:47:03.679211 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-192fc1c9-1045-4601-99fa-053114379ad0 None admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 15:47:03.718644 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=100 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 15:47:03.797842 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b110a588-d05c-48b4-a18e-c3247682dd47 None admin] Validating token access rules against request {{(pid=81767) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 15:47:03.866138 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6265026f-5835-42b1-bd51-8dbae8a5e79f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81767) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 15:47:03.866835 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6265026f-5835-42b1-bd51-8dbae8a5e79f None admin] REQUEST_METHOD: `GET` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:47:03.867125 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6265026f-5835-42b1-bd51-8dbae8a5e79f None admin] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:47:03.867427 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6265026f-5835-42b1-bd51-8dbae8a5e79f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:47:03.868173 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6265026f-5835-42b1-bd51-8dbae8a5e79f None admin] RBAC: Adding query filter params (role.id=640209879fb24ac0ae3f65dd107697c3, user.id=68b4a39eda4d43cb8b69acf6c3bbdc9d, scope.system=all) {{(pid=81767) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 15:47:03.868477 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6265026f-5835-42b1-bd51-8dbae8a5e79f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 15:47:03.869730 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-192fc1c9-1045-4601-99fa-053114379ad0 None admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 15:47:03.870163 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6265026f-5835-42b1-bd51-8dbae8a5e79f None admin] RBAC: Authorization granted {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 15:47:03.882162 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG dbcounter [-] [81767] Writing DB stats keystone:SELECT=100 {{(pid=81767) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 15:47:03.888148 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 181/372] 10.222.0.15 () {56 vars in 1308 bytes} [Mon Aug 22 15:47:03 2022] GET /identity/v3/role_assignments?user.id=68b4a39eda4d43cb8b69acf6c3bbdc9d&scope.system=all&role.id=640209879fb24ac0ae3f65dd107697c3 => generated 510 bytes in 273 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 15:47:03.889354 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6265026f-5835-42b1-bd51-8dbae8a5e79f None admin] Authenticating user token {{(pid=81767) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 15:47:03.952069 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bb67308-c3ba-45dd-8654-0362fa4df252 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 15:47:03.952861 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb67308-c3ba-45dd-8654-0362fa4df252 None admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:47:03.953237 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb67308-c3ba-45dd-8654-0362fa4df252 None admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:47:03.953606 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb67308-c3ba-45dd-8654-0362fa4df252 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:47:04.012207 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6265026f-5835-42b1-bd51-8dbae8a5e79f None admin] Validating token access rules against request {{(pid=81767) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 15:47:04.057343 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a93ef1d-4ba7-4ebf-8ab3-2946f95953ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81767) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 15:47:04.058714 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a93ef1d-4ba7-4ebf-8ab3-2946f95953ed None admin] REQUEST_METHOD: `GET` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:47:04.059134 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a93ef1d-4ba7-4ebf-8ab3-2946f95953ed None admin] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:47:04.059583 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a93ef1d-4ba7-4ebf-8ab3-2946f95953ed None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:47:04.143189 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG dbcounter [-] [81767] Writing DB stats keystone:SELECT=100 {{(pid=81767) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 15:47:04.146007 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a93ef1d-4ba7-4ebf-8ab3-2946f95953ed None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 15:47:04.148332 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a93ef1d-4ba7-4ebf-8ab3-2946f95953ed None admin] RBAC: Authorization granted {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 15:47:04.154055 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bb67308-c3ba-45dd-8654-0362fa4df252 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 15:47:04.155570 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bb67308-c3ba-45dd-8654-0362fa4df252 None admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 15:47:04.180412 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=100 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 15:47:04.318978 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 182/373] 10.222.0.15 () {58 vars in 1282 bytes} [Mon Aug 22 15:47:03 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 430 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 22 15:47:04.325887 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 192/374] 10.222.0.15 () {58 vars in 1282 bytes} [Mon Aug 22 15:47:03 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 648 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 22 15:47:04.326868 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a93ef1d-4ba7-4ebf-8ab3-2946f95953ed None admin] Authenticating user token {{(pid=81767) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 15:47:04.338646 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bb67308-c3ba-45dd-8654-0362fa4df252 None admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 15:47:04.504376 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a93ef1d-4ba7-4ebf-8ab3-2946f95953ed None admin] Validating token access rules against request {{(pid=81767) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 15:47:04.508136 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG dbcounter [-] [81767] Writing DB stats keystone:SELECT=100 {{(pid=81767) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 15:47:04.526120 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bb67308-c3ba-45dd-8654-0362fa4df252 None admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 15:47:04.545360 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=100 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 15:47:04.557848 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4751fb67-72a9-4d74-94a7-cdf045d42c9a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81767) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 15:47:04.558580 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4751fb67-72a9-4d74-94a7-cdf045d42c9a None admin] REQUEST_METHOD: `GET` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:47:04.558877 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4751fb67-72a9-4d74-94a7-cdf045d42c9a None admin] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:47:04.559151 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4751fb67-72a9-4d74-94a7-cdf045d42c9a None admin] PATH_INFO: `/v3/projects/9a98b1e4431940ab8d1fc95df2858a6c` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:47:04.561461 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4751fb67-72a9-4d74-94a7-cdf045d42c9a None admin] RBAC: Authorizing `identity:get_project(project_id=9a98b1e4431940ab8d1fc95df2858a6c)` {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 15:47:04.563340 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4751fb67-72a9-4d74-94a7-cdf045d42c9a None admin] RBAC: Authorization granted {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 15:47:04.565629 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 183/375] 10.222.0.15 () {56 vars in 1186 bytes} [Mon Aug 22 15:47:04 2022] GET /identity/v3/projects/9a98b1e4431940ab8d1fc95df2858a6c => generated 344 bytes in 239 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 15:47:04.566529 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21823293-f4c6-4f4c-a90e-03cbbdae0766 None None] REQUEST_METHOD: `GET` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:47:04.566801 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21823293-f4c6-4f4c-a90e-03cbbdae0766 None None] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:47:04.567069 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21823293-f4c6-4f4c-a90e-03cbbdae0766 None None] PATH_INFO: `/` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:47:04.568577 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 184/376] 10.222.0.15 () {54 vars in 927 bytes} [Mon Aug 22 15:47:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 15:47:04.574818 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed6ad662-f9ad-4f2a-9091-8f695f2122fd None None] REQUEST_METHOD: `POST` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:47:04.575063 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed6ad662-f9ad-4f2a-9091-8f695f2122fd None None] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:47:04.575299 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed6ad662-f9ad-4f2a-9091-8f695f2122fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:47:04.592024 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5df866b8-0feb-4928-8d40-ae02afe0cca2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 15:47:04.594456 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5df866b8-0feb-4928-8d40-ae02afe0cca2 None admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:47:04.594734 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5df866b8-0feb-4928-8d40-ae02afe0cca2 None admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:47:04.595005 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5df866b8-0feb-4928-8d40-ae02afe0cca2 None admin] PATH_INFO: `/v3/projects/c05dae9fd0dc44f9a2ab2d373bbc913f` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:47:04.597406 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5df866b8-0feb-4928-8d40-ae02afe0cca2 None admin] RBAC: Authorizing `identity:get_project(project_id=c05dae9fd0dc44f9a2ab2d373bbc913f)` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 15:47:04.599370 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5df866b8-0feb-4928-8d40-ae02afe0cca2 None admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 15:47:04.601422 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 193/377] 10.222.0.15 () {56 vars in 1186 bytes} [Mon Aug 22 15:47:04 2022] GET /identity/v3/projects/c05dae9fd0dc44f9a2ab2d373bbc913f => generated 302 bytes in 263 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 15:47:04.602191 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5df866b8-0feb-4928-8d40-ae02afe0cca2 None admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 15:47:04.603488 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.auth.core [None req-ed6ad662-f9ad-4f2a-9091-8f695f2122fd None None] MFA Rules not processed for user `6b2199ae0fb0480aa917ada391f5ec54`. Rule list: `[]` (Enabled: `True`). {{(pid=81767) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 15:47:04.604894 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.fernet_utils [None req-ed6ad662-f9ad-4f2a-9091-8f695f2122fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81767) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 15:47:04.703533 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 185/378] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Aug 22 15:47:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 22 15:47:04.704810 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed6ad662-f9ad-4f2a-9091-8f695f2122fd None None] Authenticating user token {{(pid=81767) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 15:47:04.781919 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5df866b8-0feb-4928-8d40-ae02afe0cca2 None admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 15:47:04.863723 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG dbcounter [-] [81767] Writing DB stats keystone:SELECT=100 {{(pid=81767) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 15:47:04.868620 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed6ad662-f9ad-4f2a-9091-8f695f2122fd None None] Validating token access rules against request {{(pid=81767) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 15:47:04.879240 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-745e67a1-bc1d-49ae-87f6-3a663259e793 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 15:47:04.879910 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-745e67a1-bc1d-49ae-87f6-3a663259e793 None admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:47:04.880196 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-745e67a1-bc1d-49ae-87f6-3a663259e793 None admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:47:04.880757 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-745e67a1-bc1d-49ae-87f6-3a663259e793 None admin] PATH_INFO: `/v3/groups/f064c547078a42a0b477e6b3f08a751d` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:47:04.897956 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-745e67a1-bc1d-49ae-87f6-3a663259e793 None admin] RBAC: Authorizing `identity:get_group(group_id=f064c547078a42a0b477e6b3f08a751d)` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 15:47:04.899461 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-745e67a1-bc1d-49ae-87f6-3a663259e793 None admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 15:47:04.904071 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 194/379] 10.222.0.15 () {56 vars in 1180 bytes} [Mon Aug 22 15:47:04 2022] GET /identity/v3/groups/f064c547078a42a0b477e6b3f08a751d => generated 229 bytes in 302 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 15:47:04.908188 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2e7326a-ae8b-47cf-b8c1-556bcfaffe0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81767) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 15:47:04.908883 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2e7326a-ae8b-47cf-b8c1-556bcfaffe0e None admin] REQUEST_METHOD: `GET` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:47:04.909158 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2e7326a-ae8b-47cf-b8c1-556bcfaffe0e None admin] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:47:04.909424 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2e7326a-ae8b-47cf-b8c1-556bcfaffe0e None admin] PATH_INFO: `/v3/groups/552e4a454dcf434ba77a8567804e8230` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:47:04.910600 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2e7326a-ae8b-47cf-b8c1-556bcfaffe0e None admin] RBAC: Authorizing `identity:get_group(group_id=552e4a454dcf434ba77a8567804e8230)` {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 15:47:04.911678 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-745e67a1-bc1d-49ae-87f6-3a663259e793 None admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 15:47:04.912172 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2e7326a-ae8b-47cf-b8c1-556bcfaffe0e None admin] RBAC: Authorization granted {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 15:47:04.914171 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 186/380] 10.222.0.15 () {56 vars in 1180 bytes} [Mon Aug 22 15:47:04 2022] GET /identity/v3/groups/552e4a454dcf434ba77a8567804e8230 => generated 226 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 15:47:04.920104 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2e7326a-ae8b-47cf-b8c1-556bcfaffe0e None admin] Authenticating user token {{(pid=81767) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 15:47:05.031186 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2e7326a-ae8b-47cf-b8c1-556bcfaffe0e None admin] Validating token access rules against request {{(pid=81767) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 15:47:05.047661 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=100 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 15:47:05.069802 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e6ed5e5-68d4-4460-9b99-bacc75c056ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81767) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 15:47:05.070371 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e6ed5e5-68d4-4460-9b99-bacc75c056ff None admin] REQUEST_METHOD: `GET` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:47:05.070709 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e6ed5e5-68d4-4460-9b99-bacc75c056ff None admin] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:47:05.070978 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e6ed5e5-68d4-4460-9b99-bacc75c056ff None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:47:05.071735 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e6ed5e5-68d4-4460-9b99-bacc75c056ff None admin] RBAC: Adding query filter params (group.id=552e4a454dcf434ba77a8567804e8230, role.id=1ce6b924c9604048906c07dbfb126da9, scope.project.id=c05dae9fd0dc44f9a2ab2d373bbc913f) {{(pid=81767) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 15:47:05.072296 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e6ed5e5-68d4-4460-9b99-bacc75c056ff None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 15:47:05.073686 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e6ed5e5-68d4-4460-9b99-bacc75c056ff None admin] RBAC: Authorization granted {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 15:47:05.085051 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 187/381] 10.222.0.15 () {56 vars in 1376 bytes} [Mon Aug 22 15:47:04 2022] GET /identity/v3/role_assignments?group.id=552e4a454dcf434ba77a8567804e8230&scope.project.id=c05dae9fd0dc44f9a2ab2d373bbc913f&role.id=1ce6b924c9604048906c07dbfb126da9 => generated 260 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 22 15:47:05.087058 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e6ed5e5-68d4-4460-9b99-bacc75c056ff None admin] Authenticating user token {{(pid=81767) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 15:47:05.141732 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-745e67a1-bc1d-49ae-87f6-3a663259e793 None admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 15:47:05.161106 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG dbcounter [-] [81767] Writing DB stats keystone:SELECT=100 {{(pid=81767) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 15:47:05.218746 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c701ec4-2ec8-44cd-b567-461070b6b246 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 15:47:05.219754 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c701ec4-2ec8-44cd-b567-461070b6b246 None admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:47:05.221326 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c701ec4-2ec8-44cd-b567-461070b6b246 None admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:47:05.222086 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c701ec4-2ec8-44cd-b567-461070b6b246 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:47:05.224522 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c701ec4-2ec8-44cd-b567-461070b6b246 None admin] RBAC: Adding query filter params (group.id=f064c547078a42a0b477e6b3f08a751d, role.id=9d9848cfd22e4b089db443c350e56670, scope.project.id=9a98b1e4431940ab8d1fc95df2858a6c) {{(pid=81768) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 15:47:05.225095 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c701ec4-2ec8-44cd-b567-461070b6b246 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 15:47:05.228121 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c701ec4-2ec8-44cd-b567-461070b6b246 None admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 15:47:05.251425 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 195/382] 10.222.0.15 () {56 vars in 1376 bytes} [Mon Aug 22 15:47:04 2022] GET /identity/v3/role_assignments?group.id=f064c547078a42a0b477e6b3f08a751d&scope.project.id=9a98b1e4431940ab8d1fc95df2858a6c&role.id=9d9848cfd22e4b089db443c350e56670 => generated 260 bytes in 334 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 15:47:05.292250 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e6ed5e5-68d4-4460-9b99-bacc75c056ff None admin] Validating token access rules against request {{(pid=81767) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 15:47:05.333133 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-287cdde4-2581-45e7-8a34-08ea0c3a8d8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81767) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 15:47:05.334137 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-287cdde4-2581-45e7-8a34-08ea0c3a8d8e None admin] REQUEST_METHOD: `GET` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:47:05.334611 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-287cdde4-2581-45e7-8a34-08ea0c3a8d8e None admin] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:47:05.335109 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-287cdde4-2581-45e7-8a34-08ea0c3a8d8e None admin] PATH_INFO: `/v3/roles/member` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:47:05.354005 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-287cdde4-2581-45e7-8a34-08ea0c3a8d8e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 22 15:47:05.360472 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application [None req-287cdde4-2581-45e7-8a34-08ea0c3a8d8e None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 22 15:47:05.360472 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 15:47:05.360472 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 15:47:05.360472 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 15:47:05.360472 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 15:47:05.360472 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 15:47:05.360472 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 15:47:05.360472 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 15:47:05.360472 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 15:47:05.360472 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 15:47:05.360472 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 15:47:05.360472 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 15:47:05.360472 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 22 15:47:05.360472 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 22 15:47:05.360472 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 22 15:47:05.360472 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 22 15:47:05.361791 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 22 15:47:05.361791 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 22 15:47:05.361791 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 22 15:47:05.361791 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 22 15:47:05.361791 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 15:47:05.361791 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 15:47:05.361791 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 15:47:05.361791 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 15:47:05.361791 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 15:47:05.361791 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 15:47:05.361791 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 15:47:05.361791 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application with Lock( Aug 22 15:47:05.361791 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 15:47:05.361791 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return self._enter() Aug 22 15:47:05.361791 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 15:47:05.361791 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 15:47:05.362719 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 15:47:05.362719 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return self.creator() Aug 22 15:47:05.362719 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 15:47:05.362719 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application created_value = creator( Aug 22 15:47:05.362719 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 22 15:47:05.362719 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 22 15:47:05.362719 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 22 15:47:05.362719 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 22 15:47:05.362719 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 22 15:47:05.362719 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 22 15:47:05.362719 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 22 15:47:05.362719 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application Aug 22 15:47:05.370610 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 188/383] 10.222.0.15 () {56 vars in 1099 bytes} [Mon Aug 22 15:47:05 2022] GET /identity/v3/roles/member => generated 84 bytes in 284 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 15:47:05.374634 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c701ec4-2ec8-44cd-b567-461070b6b246 None admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 15:47:05.500976 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c701ec4-2ec8-44cd-b567-461070b6b246 None admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 15:47:05.512852 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=100 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 15:47:05.538306 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-542ee3c5-36a2-468c-b910-263eb8093496 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 15:47:05.538983 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-542ee3c5-36a2-468c-b910-263eb8093496 None admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:47:05.539266 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-542ee3c5-36a2-468c-b910-263eb8093496 None admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:47:05.539566 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-542ee3c5-36a2-468c-b910-263eb8093496 None admin] PATH_INFO: `/v3/roles` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:47:05.540231 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-542ee3c5-36a2-468c-b910-263eb8093496 None admin] RBAC: Adding query filter params (name=member) {{(pid=81768) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 15:47:05.540530 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-542ee3c5-36a2-468c-b910-263eb8093496 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 15:47:05.541942 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-542ee3c5-36a2-468c-b910-263eb8093496 None admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 15:47:05.555865 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 196/384] 10.222.0.15 () {56 vars in 1101 bytes} [Mon Aug 22 15:47:05 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 15:47:05.559609 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-287cdde4-2581-45e7-8a34-08ea0c3a8d8e None admin] Authenticating user token {{(pid=81767) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 15:47:05.680032 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG dbcounter [-] [81767] Writing DB stats keystone:SELECT=100 {{(pid=81767) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 15:47:05.694967 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-287cdde4-2581-45e7-8a34-08ea0c3a8d8e None admin] Validating token access rules against request {{(pid=81767) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 15:47:05.713025 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-567ac995-ae38-451f-b000-04c181c43339 None None] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:47:05.713648 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-567ac995-ae38-451f-b000-04c181c43339 None None] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:47:05.714152 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-567ac995-ae38-451f-b000-04c181c43339 None None] PATH_INFO: `/` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:47:05.717653 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 197/385] 10.222.0.15 () {54 vars in 927 bytes} [Mon Aug 22 15:47:05 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 15:47:05.728794 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80b9b4eb-ef2b-4f7f-96f6-92b34d94fcb7 None None] REQUEST_METHOD: `POST` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:47:05.729398 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80b9b4eb-ef2b-4f7f-96f6-92b34d94fcb7 None None] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:47:05.729862 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80b9b4eb-ef2b-4f7f-96f6-92b34d94fcb7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:47:05.738432 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2406ab48-0a4e-452a-91e8-3fec02499fd6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81767) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 15:47:05.738432 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2406ab48-0a4e-452a-91e8-3fec02499fd6 None admin] REQUEST_METHOD: `GET` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:47:05.738432 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2406ab48-0a4e-452a-91e8-3fec02499fd6 None admin] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:47:05.738432 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2406ab48-0a4e-452a-91e8-3fec02499fd6 None admin] PATH_INFO: `/v3/groups/552e4a454dcf434ba77a8567804e8230` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:47:05.738813 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2406ab48-0a4e-452a-91e8-3fec02499fd6 None admin] RBAC: Authorizing `identity:get_group(group_id=552e4a454dcf434ba77a8567804e8230)` {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 15:47:05.741626 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2406ab48-0a4e-452a-91e8-3fec02499fd6 None admin] RBAC: Authorization granted {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 15:47:05.741910 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 189/386] 10.222.0.15 () {56 vars in 1180 bytes} [Mon Aug 22 15:47:05 2022] GET /identity/v3/groups/552e4a454dcf434ba77a8567804e8230 => generated 226 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 15:47:05.751909 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2406ab48-0a4e-452a-91e8-3fec02499fd6 None admin] Authenticating user token {{(pid=81767) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 15:47:05.771701 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.auth.core [None req-80b9b4eb-ef2b-4f7f-96f6-92b34d94fcb7 None None] MFA Rules not processed for user `6b2199ae0fb0480aa917ada391f5ec54`. Rule list: `[]` (Enabled: `True`). {{(pid=81768) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 15:47:05.775661 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-80b9b4eb-ef2b-4f7f-96f6-92b34d94fcb7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 15:47:05.915504 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 198/387] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Aug 22 15:47:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 22 15:47:05.926207 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2406ab48-0a4e-452a-91e8-3fec02499fd6 None admin] Validating token access rules against request {{(pid=81767) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 15:47:05.964708 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bad4a907-fc4e-46c4-adee-5f96c1744c6e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81767) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 15:47:05.965673 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bad4a907-fc4e-46c4-adee-5f96c1744c6e None admin] REQUEST_METHOD: `GET` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:47:05.966093 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bad4a907-fc4e-46c4-adee-5f96c1744c6e None admin] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:47:05.966507 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bad4a907-fc4e-46c4-adee-5f96c1744c6e None admin] PATH_INFO: `/v3/projects/43171f5fd3454380b88625276d7375da` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:47:05.967954 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bad4a907-fc4e-46c4-adee-5f96c1744c6e None admin] RBAC: Authorizing `identity:get_project(project_id=43171f5fd3454380b88625276d7375da)` {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 15:47:05.969824 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bad4a907-fc4e-46c4-adee-5f96c1744c6e None admin] RBAC: Authorization granted {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 15:47:05.979560 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80b9b4eb-ef2b-4f7f-96f6-92b34d94fcb7 None None] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 15:47:05.983121 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 190/388] 10.222.0.15 () {56 vars in 1186 bytes} [Mon Aug 22 15:47:05 2022] GET /identity/v3/projects/43171f5fd3454380b88625276d7375da => generated 298 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 22 15:47:06.099750 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80b9b4eb-ef2b-4f7f-96f6-92b34d94fcb7 None None] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 15:47:06.101800 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=100 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 15:47:06.141211 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e4054e8-aa6f-4141-a118-6c45ddda366c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 15:47:06.142116 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e4054e8-aa6f-4141-a118-6c45ddda366c None admin] REQUEST_METHOD: `PUT` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:47:06.142560 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e4054e8-aa6f-4141-a118-6c45ddda366c None admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:47:06.142969 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e4054e8-aa6f-4141-a118-6c45ddda366c None admin] PATH_INFO: `/v3/projects/43171f5fd3454380b88625276d7375da/groups/552e4a454dcf434ba77a8567804e8230/roles/e6fc1ee3fb974125ba9b4fe55b12b45e` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:47:06.146032 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e4054e8-aa6f-4141-a118-6c45ddda366c None admin] RBAC: Authorizing `identity:create_grant(project_id=43171f5fd3454380b88625276d7375da, group_id=552e4a454dcf434ba77a8567804e8230, role_id=e6fc1ee3fb974125ba9b4fe55b12b45e)` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 15:47:06.147797 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e4054e8-aa6f-4141-a118-6c45ddda366c None admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 15:47:06.254574 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c12110-eabe-4797-bc84-f2a5373478b2 None None] REQUEST_METHOD: `GET` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:47:06.255097 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c12110-eabe-4797-bc84-f2a5373478b2 None None] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:47:06.255533 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c12110-eabe-4797-bc84-f2a5373478b2 None None] PATH_INFO: `/` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:47:06.257074 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 191/389] 10.222.0.15 () {54 vars in 927 bytes} [Mon Aug 22 15:47:06 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 22 15:47:06.263458 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e8b4d94-a864-46e5-8c04-c65b90d76853 None None] REQUEST_METHOD: `POST` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:47:06.269728 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e8b4d94-a864-46e5-8c04-c65b90d76853 None None] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:47:06.269728 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e8b4d94-a864-46e5-8c04-c65b90d76853 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:47:06.269939 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 199/390] 10.222.0.15 () {58 vars in 1442 bytes} [Mon Aug 22 15:47:05 2022] PUT /identity/v3/projects/43171f5fd3454380b88625276d7375da/groups/552e4a454dcf434ba77a8567804e8230/roles/e6fc1ee3fb974125ba9b4fe55b12b45e => generated 0 bytes in 289 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 15:47:06.273280 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e4054e8-aa6f-4141-a118-6c45ddda366c None admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 15:47:06.296487 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.auth.core [None req-2e8b4d94-a864-46e5-8c04-c65b90d76853 None None] MFA Rules not processed for user `6b2199ae0fb0480aa917ada391f5ec54`. Rule list: `[]` (Enabled: `True`). {{(pid=81767) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 15:47:06.297697 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.fernet_utils [None req-2e8b4d94-a864-46e5-8c04-c65b90d76853 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81767) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 15:47:06.337658 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG dbcounter [-] [81767] Writing DB stats keystone:SELECT=100 {{(pid=81767) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 15:47:06.383476 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 192/391] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Aug 22 15:47:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 22 15:47:06.464189 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e4054e8-aa6f-4141-a118-6c45ddda366c None admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 15:47:06.507076 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a850af00-6e1e-458d-b804-69093f1adbb5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 15:47:06.507825 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a850af00-6e1e-458d-b804-69093f1adbb5 None admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:47:06.508103 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a850af00-6e1e-458d-b804-69093f1adbb5 None admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:47:06.508378 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a850af00-6e1e-458d-b804-69093f1adbb5 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:47:06.519919 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a850af00-6e1e-458d-b804-69093f1adbb5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 22 15:47:06.524983 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application [None req-a850af00-6e1e-458d-b804-69093f1adbb5 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 22 15:47:06.524983 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 15:47:06.524983 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 15:47:06.524983 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 15:47:06.524983 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 15:47:06.524983 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 15:47:06.524983 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 15:47:06.524983 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 15:47:06.524983 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 15:47:06.524983 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 15:47:06.524983 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 15:47:06.524983 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 15:47:06.524983 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 22 15:47:06.524983 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 22 15:47:06.524983 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 22 15:47:06.524983 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 22 15:47:06.525839 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 22 15:47:06.525839 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 22 15:47:06.525839 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 22 15:47:06.525839 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 22 15:47:06.525839 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 15:47:06.525839 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 15:47:06.525839 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 15:47:06.525839 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 15:47:06.525839 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 15:47:06.525839 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 15:47:06.525839 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 15:47:06.525839 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application with Lock( Aug 22 15:47:06.525839 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 15:47:06.525839 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self._enter() Aug 22 15:47:06.525839 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 15:47:06.525839 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 15:47:06.526812 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 15:47:06.526812 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self.creator() Aug 22 15:47:06.526812 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 15:47:06.526812 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application created_value = creator( Aug 22 15:47:06.526812 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 22 15:47:06.526812 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 22 15:47:06.526812 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 22 15:47:06.526812 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 22 15:47:06.526812 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 22 15:47:06.526812 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 22 15:47:06.526812 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 22 15:47:06.526812 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application Aug 22 15:47:06.526812 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 200/392] 10.222.0.15 () {56 vars in 1114 bytes} [Mon Aug 22 15:47:06 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 254 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 22 15:47:06.536514 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e8b4d94-a864-46e5-8c04-c65b90d76853 None None] Authenticating user token {{(pid=81767) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 15:47:06.699574 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e8b4d94-a864-46e5-8c04-c65b90d76853 None None] Validating token access rules against request {{(pid=81767) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 15:47:06.730922 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a850af00-6e1e-458d-b804-69093f1adbb5 None admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 15:47:06.768937 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38502201-d148-4a86-90d0-5904eaf41acd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81767) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 15:47:06.769796 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38502201-d148-4a86-90d0-5904eaf41acd None admin] REQUEST_METHOD: `GET` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:47:06.770262 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38502201-d148-4a86-90d0-5904eaf41acd None admin] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:47:06.770675 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38502201-d148-4a86-90d0-5904eaf41acd None admin] PATH_INFO: `/v3/roles` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:47:06.771501 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38502201-d148-4a86-90d0-5904eaf41acd None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81767) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 15:47:06.771966 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38502201-d148-4a86-90d0-5904eaf41acd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 15:47:06.773500 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 15:47:06.776485 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38502201-d148-4a86-90d0-5904eaf41acd None admin] RBAC: Authorization granted {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 15:47:06.793954 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 193/393] 10.222.0.15 () {56 vars in 1111 bytes} [Mon Aug 22 15:47:06 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 258 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 15:47:06.800975 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38502201-d148-4a86-90d0-5904eaf41acd None admin] Authenticating user token {{(pid=81767) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 15:47:06.843106 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG dbcounter [-] [81767] Writing DB stats keystone:SELECT=100 {{(pid=81767) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 15:47:06.906313 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a850af00-6e1e-458d-b804-69093f1adbb5 None admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 15:47:07.034278 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39bd9f31-aefc-4a7b-a74f-68300b4b0bac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 15:47:07.035673 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39bd9f31-aefc-4a7b-a74f-68300b4b0bac None admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:47:07.036116 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39bd9f31-aefc-4a7b-a74f-68300b4b0bac None admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:47:07.036544 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39bd9f31-aefc-4a7b-a74f-68300b4b0bac None admin] PATH_INFO: `/v3/roles/member` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:47:07.066620 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39bd9f31-aefc-4a7b-a74f-68300b4b0bac None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 22 15:47:07.072316 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application [None req-39bd9f31-aefc-4a7b-a74f-68300b4b0bac None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 22 15:47:07.072316 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 15:47:07.072316 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 15:47:07.072316 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 15:47:07.072316 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 15:47:07.072316 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 15:47:07.072316 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 15:47:07.072316 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 15:47:07.072316 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 15:47:07.072316 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 15:47:07.072316 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 15:47:07.072316 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 15:47:07.072316 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 22 15:47:07.072316 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 22 15:47:07.072316 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 22 15:47:07.072316 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 22 15:47:07.073408 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 22 15:47:07.073408 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 22 15:47:07.073408 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 22 15:47:07.073408 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 22 15:47:07.073408 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 15:47:07.073408 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 15:47:07.073408 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 15:47:07.073408 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 15:47:07.073408 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 15:47:07.073408 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 15:47:07.073408 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 15:47:07.073408 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application with Lock( Aug 22 15:47:07.073408 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 15:47:07.073408 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self._enter() Aug 22 15:47:07.073408 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 15:47:07.073408 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 15:47:07.074844 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 15:47:07.074844 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self.creator() Aug 22 15:47:07.074844 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 15:47:07.074844 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application created_value = creator( Aug 22 15:47:07.074844 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 22 15:47:07.074844 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 22 15:47:07.074844 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 22 15:47:07.074844 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 22 15:47:07.074844 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 22 15:47:07.074844 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 22 15:47:07.074844 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 22 15:47:07.074844 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application Aug 22 15:47:07.079434 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 201/394] 10.222.0.15 () {56 vars in 1099 bytes} [Mon Aug 22 15:47:06 2022] GET /identity/v3/roles/member => generated 84 bytes in 347 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 22 15:47:07.087511 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39bd9f31-aefc-4a7b-a74f-68300b4b0bac None admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 15:47:07.094252 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38502201-d148-4a86-90d0-5904eaf41acd None admin] Validating token access rules against request {{(pid=81767) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 15:47:07.164802 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18415e72-87df-46b3-9e39-2a15f30683b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81767) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 15:47:07.165608 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18415e72-87df-46b3-9e39-2a15f30683b5 None admin] REQUEST_METHOD: `GET` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:47:07.165980 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18415e72-87df-46b3-9e39-2a15f30683b5 None admin] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:47:07.166362 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18415e72-87df-46b3-9e39-2a15f30683b5 None admin] PATH_INFO: `/v3/groups/552e4a454dcf434ba77a8567804e8230` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:47:07.172254 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18415e72-87df-46b3-9e39-2a15f30683b5 None admin] RBAC: Authorizing `identity:get_group(group_id=552e4a454dcf434ba77a8567804e8230)` {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 15:47:07.173929 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18415e72-87df-46b3-9e39-2a15f30683b5 None admin] RBAC: Authorization granted {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 15:47:07.177478 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 194/395] 10.222.0.15 () {56 vars in 1180 bytes} [Mon Aug 22 15:47:06 2022] GET /identity/v3/groups/552e4a454dcf434ba77a8567804e8230 => generated 226 bytes in 376 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 15:47:07.182118 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18415e72-87df-46b3-9e39-2a15f30683b5 None admin] Authenticating user token {{(pid=81767) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 15:47:07.206897 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=100 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 15:47:07.226786 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39bd9f31-aefc-4a7b-a74f-68300b4b0bac None admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 15:47:07.272441 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-928cfad5-d1e0-4811-9f33-4efbdaf4770e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 15:47:07.273210 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-928cfad5-d1e0-4811-9f33-4efbdaf4770e None admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:47:07.273520 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-928cfad5-d1e0-4811-9f33-4efbdaf4770e None admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:47:07.273790 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-928cfad5-d1e0-4811-9f33-4efbdaf4770e None admin] PATH_INFO: `/v3/roles` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:47:07.274460 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-928cfad5-d1e0-4811-9f33-4efbdaf4770e None admin] RBAC: Adding query filter params (name=member) {{(pid=81768) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 15:47:07.274862 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-928cfad5-d1e0-4811-9f33-4efbdaf4770e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 15:47:07.276480 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-928cfad5-d1e0-4811-9f33-4efbdaf4770e None admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 15:47:07.291821 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 202/396] 10.222.0.15 () {56 vars in 1101 bytes} [Mon Aug 22 15:47:07 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 15:47:07.294100 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-928cfad5-d1e0-4811-9f33-4efbdaf4770e None admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 15:47:07.311834 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG dbcounter [-] [81767] Writing DB stats keystone:SELECT=100 {{(pid=81767) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 15:47:07.329863 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18415e72-87df-46b3-9e39-2a15f30683b5 None admin] Validating token access rules against request {{(pid=81767) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 15:47:07.376094 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54e3f0f2-a02c-41f0-b74a-1ab87e77fe12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81767) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 15:47:07.376807 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54e3f0f2-a02c-41f0-b74a-1ab87e77fe12 None admin] REQUEST_METHOD: `GET` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:47:07.377082 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54e3f0f2-a02c-41f0-b74a-1ab87e77fe12 None admin] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:47:07.377343 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54e3f0f2-a02c-41f0-b74a-1ab87e77fe12 None admin] PATH_INFO: `/v3/projects/43171f5fd3454380b88625276d7375da` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:47:07.378812 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54e3f0f2-a02c-41f0-b74a-1ab87e77fe12 None admin] RBAC: Authorizing `identity:get_project(project_id=43171f5fd3454380b88625276d7375da)` {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 15:47:07.380317 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54e3f0f2-a02c-41f0-b74a-1ab87e77fe12 None admin] RBAC: Authorization granted {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 15:47:07.382257 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 195/397] 10.222.0.15 () {56 vars in 1186 bytes} [Mon Aug 22 15:47:07 2022] GET /identity/v3/projects/43171f5fd3454380b88625276d7375da => generated 298 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 22 15:47:07.388806 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54e3f0f2-a02c-41f0-b74a-1ab87e77fe12 None admin] Authenticating user token {{(pid=81767) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 15:47:07.467795 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-928cfad5-d1e0-4811-9f33-4efbdaf4770e None admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 15:47:07.518391 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca260c99-46d6-4ed4-99fa-04de49741cdd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 15:47:07.519040 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca260c99-46d6-4ed4-99fa-04de49741cdd None admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:47:07.519339 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca260c99-46d6-4ed4-99fa-04de49741cdd None admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:47:07.519619 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca260c99-46d6-4ed4-99fa-04de49741cdd None admin] PATH_INFO: `/v3/groups/552e4a454dcf434ba77a8567804e8230` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:47:07.523966 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca260c99-46d6-4ed4-99fa-04de49741cdd None admin] RBAC: Authorizing `identity:get_group(group_id=552e4a454dcf434ba77a8567804e8230)` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 15:47:07.524252 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54e3f0f2-a02c-41f0-b74a-1ab87e77fe12 None admin] Validating token access rules against request {{(pid=81767) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 15:47:07.525550 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca260c99-46d6-4ed4-99fa-04de49741cdd None admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 15:47:07.527887 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 203/398] 10.222.0.15 () {56 vars in 1180 bytes} [Mon Aug 22 15:47:07 2022] GET /identity/v3/groups/552e4a454dcf434ba77a8567804e8230 => generated 226 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 22 15:47:07.533357 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca260c99-46d6-4ed4-99fa-04de49741cdd None admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 15:47:07.562267 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0b6b07f-aae0-4d49-9f06-36d572876610 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81767) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 15:47:07.562845 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b6b07f-aae0-4d49-9f06-36d572876610 None admin] REQUEST_METHOD: `PUT` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:47:07.563116 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b6b07f-aae0-4d49-9f06-36d572876610 None admin] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:47:07.563430 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b6b07f-aae0-4d49-9f06-36d572876610 None admin] PATH_INFO: `/v3/projects/43171f5fd3454380b88625276d7375da/groups/552e4a454dcf434ba77a8567804e8230/roles/1ce6b924c9604048906c07dbfb126da9` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:47:07.565581 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0b6b07f-aae0-4d49-9f06-36d572876610 None admin] RBAC: Authorizing `identity:create_grant(project_id=43171f5fd3454380b88625276d7375da, group_id=552e4a454dcf434ba77a8567804e8230, role_id=1ce6b924c9604048906c07dbfb126da9)` {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 15:47:07.567009 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0b6b07f-aae0-4d49-9f06-36d572876610 None admin] RBAC: Authorization granted {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 15:47:07.572011 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=100 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 15:47:07.665495 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca260c99-46d6-4ed4-99fa-04de49741cdd None admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 15:47:07.684174 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 196/399] 10.222.0.15 () {58 vars in 1442 bytes} [Mon Aug 22 15:47:07 2022] PUT /identity/v3/projects/43171f5fd3454380b88625276d7375da/groups/552e4a454dcf434ba77a8567804e8230/roles/1ce6b924c9604048906c07dbfb126da9 => generated 0 bytes in 296 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 15:47:07.685574 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33609695-cb12-445e-908b-c7f26cb39ad3 None None] REQUEST_METHOD: `GET` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:47:07.685990 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33609695-cb12-445e-908b-c7f26cb39ad3 None None] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:47:07.686361 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33609695-cb12-445e-908b-c7f26cb39ad3 None None] PATH_INFO: `/` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:47:07.696509 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 197/400] 10.222.0.15 () {54 vars in 927 bytes} [Mon Aug 22 15:47:07 2022] GET /identity => generated 270 bytes in 12 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 15:47:07.703170 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-589dfef0-5911-4d09-9ffe-87ded2d77a43 None None] REQUEST_METHOD: `POST` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:47:07.703674 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-589dfef0-5911-4d09-9ffe-87ded2d77a43 None None] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:47:07.704375 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-589dfef0-5911-4d09-9ffe-87ded2d77a43 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:47:07.723789 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d3b44a7-245d-48ea-b6da-f0594616e865 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 15:47:07.724718 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d3b44a7-245d-48ea-b6da-f0594616e865 None admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:47:07.725166 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d3b44a7-245d-48ea-b6da-f0594616e865 None admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:47:07.725617 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d3b44a7-245d-48ea-b6da-f0594616e865 None admin] PATH_INFO: `/v3/projects/c05dae9fd0dc44f9a2ab2d373bbc913f` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:47:07.727592 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d3b44a7-245d-48ea-b6da-f0594616e865 None admin] RBAC: Authorizing `identity:get_project(project_id=c05dae9fd0dc44f9a2ab2d373bbc913f)` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 15:47:07.733144 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d3b44a7-245d-48ea-b6da-f0594616e865 None admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 15:47:07.733144 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 204/401] 10.222.0.15 () {56 vars in 1186 bytes} [Mon Aug 22 15:47:07 2022] GET /identity/v3/projects/c05dae9fd0dc44f9a2ab2d373bbc913f => generated 302 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 22 15:47:07.737860 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d3b44a7-245d-48ea-b6da-f0594616e865 None admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 15:47:07.738830 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.auth.core [None req-589dfef0-5911-4d09-9ffe-87ded2d77a43 None None] MFA Rules not processed for user `6b2199ae0fb0480aa917ada391f5ec54`. Rule list: `[]` (Enabled: `True`). {{(pid=81767) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 15:47:07.740333 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.fernet_utils [None req-589dfef0-5911-4d09-9ffe-87ded2d77a43 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81767) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 15:47:07.753573 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG dbcounter [-] [81767] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81767) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 15:47:07.827131 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 198/402] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Aug 22 15:47:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 22 15:47:07.867679 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=100 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 15:47:07.884547 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d3b44a7-245d-48ea-b6da-f0594616e865 None admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 15:47:07.925066 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b33904a-3bdb-452e-bb69-972f0c1a9174 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 15:47:07.925748 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b33904a-3bdb-452e-bb69-972f0c1a9174 None admin] REQUEST_METHOD: `PUT` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:47:07.926024 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b33904a-3bdb-452e-bb69-972f0c1a9174 None admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:47:07.926294 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b33904a-3bdb-452e-bb69-972f0c1a9174 None admin] PATH_INFO: `/v3/projects/c05dae9fd0dc44f9a2ab2d373bbc913f/groups/552e4a454dcf434ba77a8567804e8230/roles/e6fc1ee3fb974125ba9b4fe55b12b45e` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:47:07.933030 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b33904a-3bdb-452e-bb69-972f0c1a9174 None admin] RBAC: Authorizing `identity:create_grant(project_id=c05dae9fd0dc44f9a2ab2d373bbc913f, group_id=552e4a454dcf434ba77a8567804e8230, role_id=e6fc1ee3fb974125ba9b4fe55b12b45e)` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 15:47:07.935144 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b33904a-3bdb-452e-bb69-972f0c1a9174 None admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 15:47:08.068979 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 205/403] 10.222.0.15 () {58 vars in 1442 bytes} [Mon Aug 22 15:47:07 2022] PUT /identity/v3/projects/c05dae9fd0dc44f9a2ab2d373bbc913f/groups/552e4a454dcf434ba77a8567804e8230/roles/e6fc1ee3fb974125ba9b4fe55b12b45e => generated 0 bytes in 331 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 15:47:08.147561 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-589dfef0-5911-4d09-9ffe-87ded2d77a43 None None] Authenticating user token {{(pid=81767) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 15:47:08.176125 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68942935-19bf-4987-8b76-6e5ed8f9c24d None None] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:47:08.176438 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68942935-19bf-4987-8b76-6e5ed8f9c24d None None] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:47:08.176660 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68942935-19bf-4987-8b76-6e5ed8f9c24d None None] PATH_INFO: `/` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:47:08.178201 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 206/404] 10.222.0.15 () {54 vars in 927 bytes} [Mon Aug 22 15:47:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 22 15:47:08.183908 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb847f5c-b3bb-4e8f-96a6-d35106940705 None None] REQUEST_METHOD: `POST` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:47:08.184195 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb847f5c-b3bb-4e8f-96a6-d35106940705 None None] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:47:08.184463 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb847f5c-b3bb-4e8f-96a6-d35106940705 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:47:08.215050 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.auth.core [None req-fb847f5c-b3bb-4e8f-96a6-d35106940705 None None] MFA Rules not processed for user `6b2199ae0fb0480aa917ada391f5ec54`. Rule list: `[]` (Enabled: `True`). {{(pid=81768) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 15:47:08.216329 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-fb847f5c-b3bb-4e8f-96a6-d35106940705 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 15:47:08.285133 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-589dfef0-5911-4d09-9ffe-87ded2d77a43 None None] Validating token access rules against request {{(pid=81767) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 15:47:08.314266 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 207/405] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Aug 22 15:47:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 22 15:47:08.335629 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fa05f01-81a7-486b-a2de-740763f6e851 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81767) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 15:47:08.336362 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fa05f01-81a7-486b-a2de-740763f6e851 None admin] REQUEST_METHOD: `GET` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:47:08.336576 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fa05f01-81a7-486b-a2de-740763f6e851 None admin] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:47:08.336846 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fa05f01-81a7-486b-a2de-740763f6e851 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:47:08.350887 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fa05f01-81a7-486b-a2de-740763f6e851 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 22 15:47:08.356440 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application [None req-2fa05f01-81a7-486b-a2de-740763f6e851 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 22 15:47:08.356440 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 15:47:08.356440 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 15:47:08.356440 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 15:47:08.356440 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 15:47:08.356440 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 15:47:08.356440 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 15:47:08.356440 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 15:47:08.356440 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 15:47:08.356440 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 15:47:08.356440 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 15:47:08.356440 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 15:47:08.356440 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 22 15:47:08.356440 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 22 15:47:08.356440 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 22 15:47:08.356440 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 22 15:47:08.357506 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 22 15:47:08.357506 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 22 15:47:08.357506 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 22 15:47:08.357506 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 22 15:47:08.357506 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 15:47:08.357506 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 15:47:08.357506 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 15:47:08.357506 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 15:47:08.357506 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 15:47:08.357506 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 15:47:08.357506 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 15:47:08.357506 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application with Lock( Aug 22 15:47:08.357506 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 15:47:08.357506 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return self._enter() Aug 22 15:47:08.357506 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 15:47:08.357506 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 15:47:08.358431 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 15:47:08.358431 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return self.creator() Aug 22 15:47:08.358431 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 15:47:08.358431 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application created_value = creator( Aug 22 15:47:08.358431 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 22 15:47:08.358431 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 22 15:47:08.358431 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 22 15:47:08.358431 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 22 15:47:08.358431 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 22 15:47:08.358431 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 22 15:47:08.358431 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 22 15:47:08.358431 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application Aug 22 15:47:08.360623 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 199/406] 10.222.0.15 () {56 vars in 1114 bytes} [Mon Aug 22 15:47:08 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 15:47:08.371306 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb847f5c-b3bb-4e8f-96a6-d35106940705 None None] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 15:47:08.482426 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 15:47:08.527753 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb847f5c-b3bb-4e8f-96a6-d35106940705 None None] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 15:47:08.571965 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aab0f79-d864-42ac-a22e-0367b29817bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 15:47:08.572588 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aab0f79-d864-42ac-a22e-0367b29817bd None admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:47:08.572915 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aab0f79-d864-42ac-a22e-0367b29817bd None admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:47:08.573260 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aab0f79-d864-42ac-a22e-0367b29817bd None admin] PATH_INFO: `/v3/roles` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:47:08.574005 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aab0f79-d864-42ac-a22e-0367b29817bd None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81768) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 15:47:08.574308 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aab0f79-d864-42ac-a22e-0367b29817bd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 15:47:08.575730 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aab0f79-d864-42ac-a22e-0367b29817bd None admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 15:47:08.587796 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 208/407] 10.222.0.15 () {56 vars in 1111 bytes} [Mon Aug 22 15:47:08 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 15:47:08.593753 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fa05f01-81a7-486b-a2de-740763f6e851 None admin] Authenticating user token {{(pid=81767) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 15:47:08.604878 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aab0f79-d864-42ac-a22e-0367b29817bd None admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 15:47:08.606795 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG dbcounter [-] [81767] Writing DB stats keystone:SELECT=100 {{(pid=81767) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 15:47:08.711188 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aab0f79-d864-42ac-a22e-0367b29817bd None admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 15:47:08.744457 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=100 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 15:47:08.748488 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73e41357-d337-4a4e-a0e3-e4af4c206db4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 15:47:08.749371 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73e41357-d337-4a4e-a0e3-e4af4c206db4 None admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:47:08.750047 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73e41357-d337-4a4e-a0e3-e4af4c206db4 None admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:47:08.751684 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73e41357-d337-4a4e-a0e3-e4af4c206db4 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:47:08.772602 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73e41357-d337-4a4e-a0e3-e4af4c206db4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 22 15:47:08.779099 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application [None req-73e41357-d337-4a4e-a0e3-e4af4c206db4 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 22 15:47:08.779099 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 15:47:08.779099 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 15:47:08.779099 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 15:47:08.779099 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 15:47:08.779099 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 15:47:08.779099 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 15:47:08.779099 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 15:47:08.779099 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 15:47:08.779099 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 15:47:08.779099 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 15:47:08.779099 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 15:47:08.779099 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 22 15:47:08.779099 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 22 15:47:08.779099 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 22 15:47:08.779099 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 22 15:47:08.780015 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 22 15:47:08.780015 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 22 15:47:08.780015 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 22 15:47:08.780015 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 22 15:47:08.780015 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 15:47:08.780015 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 15:47:08.780015 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 15:47:08.780015 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 15:47:08.780015 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 15:47:08.780015 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 15:47:08.780015 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 15:47:08.780015 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application with Lock( Aug 22 15:47:08.780015 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 15:47:08.780015 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self._enter() Aug 22 15:47:08.780015 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 15:47:08.780015 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 15:47:08.780781 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 15:47:08.780781 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self.creator() Aug 22 15:47:08.780781 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 15:47:08.780781 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application created_value = creator( Aug 22 15:47:08.780781 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 22 15:47:08.780781 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 22 15:47:08.780781 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 22 15:47:08.780781 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 22 15:47:08.780781 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 22 15:47:08.780781 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 22 15:47:08.780781 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 22 15:47:08.780781 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application Aug 22 15:47:08.790246 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 209/408] 10.222.0.15 () {56 vars in 1096 bytes} [Mon Aug 22 15:47:08 2022] GET /identity/v3/roles/admin => generated 83 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 22 15:47:08.798829 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73e41357-d337-4a4e-a0e3-e4af4c206db4 None admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 15:47:08.834887 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fa05f01-81a7-486b-a2de-740763f6e851 None admin] Validating token access rules against request {{(pid=81767) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 15:47:08.881190 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55966660-13c5-4dbe-ba5f-571a63e90884 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81767) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 15:47:08.881870 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55966660-13c5-4dbe-ba5f-571a63e90884 None admin] REQUEST_METHOD: `GET` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:47:08.882172 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55966660-13c5-4dbe-ba5f-571a63e90884 None admin] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:47:08.882482 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55966660-13c5-4dbe-ba5f-571a63e90884 None admin] PATH_INFO: `/v3/groups/552e4a454dcf434ba77a8567804e8230` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:47:08.883885 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55966660-13c5-4dbe-ba5f-571a63e90884 None admin] RBAC: Authorizing `identity:get_group(group_id=552e4a454dcf434ba77a8567804e8230)` {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 15:47:08.885596 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55966660-13c5-4dbe-ba5f-571a63e90884 None admin] RBAC: Authorization granted {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 15:47:08.892678 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 200/409] 10.222.0.15 () {56 vars in 1180 bytes} [Mon Aug 22 15:47:08 2022] GET /identity/v3/groups/552e4a454dcf434ba77a8567804e8230 => generated 226 bytes in 295 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 15:47:08.894199 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36dfcd17-3c6f-40fe-8e51-d78eca90c80c None None] REQUEST_METHOD: `GET` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:47:08.894488 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36dfcd17-3c6f-40fe-8e51-d78eca90c80c None None] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:47:08.894769 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36dfcd17-3c6f-40fe-8e51-d78eca90c80c None None] PATH_INFO: `/` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:47:08.896911 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 201/410] 10.222.0.15 () {54 vars in 927 bytes} [Mon Aug 22 15:47:08 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 15:47:08.897770 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36dfcd17-3c6f-40fe-8e51-d78eca90c80c None None] Authenticating user token {{(pid=81767) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 15:47:08.917357 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73e41357-d337-4a4e-a0e3-e4af4c206db4 None admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 15:47:08.953297 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52312d66-d5c8-401e-8e08-9784018ec441 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 15:47:08.954277 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52312d66-d5c8-401e-8e08-9784018ec441 None admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:47:08.954746 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52312d66-d5c8-401e-8e08-9784018ec441 None admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:47:08.955206 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52312d66-d5c8-401e-8e08-9784018ec441 None admin] PATH_INFO: `/v3/roles` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:47:08.956084 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52312d66-d5c8-401e-8e08-9784018ec441 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81768) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 15:47:08.956572 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52312d66-d5c8-401e-8e08-9784018ec441 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 15:47:08.958163 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52312d66-d5c8-401e-8e08-9784018ec441 None admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 15:47:08.967322 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 210/411] 10.222.0.15 () {56 vars in 1099 bytes} [Mon Aug 22 15:47:08 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 22 15:47:08.973070 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a08a7780-ea6a-4634-8ff4-e15a8c8699ae None None] REQUEST_METHOD: `POST` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:47:08.973540 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a08a7780-ea6a-4634-8ff4-e15a8c8699ae None None] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:47:08.973908 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a08a7780-ea6a-4634-8ff4-e15a8c8699ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:47:08.987516 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG dbcounter [-] [81767] Writing DB stats keystone:SELECT=100 {{(pid=81767) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 15:47:08.999489 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.auth.core [None req-a08a7780-ea6a-4634-8ff4-e15a8c8699ae None None] MFA Rules not processed for user `6b2199ae0fb0480aa917ada391f5ec54`. Rule list: `[]` (Enabled: `True`). {{(pid=81768) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 15:47:08.999489 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-a08a7780-ea6a-4634-8ff4-e15a8c8699ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 15:47:09.025703 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36dfcd17-3c6f-40fe-8e51-d78eca90c80c None None] Validating token access rules against request {{(pid=81767) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 15:47:09.048161 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=100 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 15:47:09.072715 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 211/412] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Aug 22 15:47:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 22 15:47:09.072715 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a08a7780-ea6a-4634-8ff4-e15a8c8699ae None None] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 15:47:09.077653 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69774d6c-24c0-48be-bef1-794c552af1de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81767) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 15:47:09.078636 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69774d6c-24c0-48be-bef1-794c552af1de None admin] REQUEST_METHOD: `GET` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:47:09.079090 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69774d6c-24c0-48be-bef1-794c552af1de None admin] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:47:09.084654 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69774d6c-24c0-48be-bef1-794c552af1de None admin] PATH_INFO: `/v3/projects/c05dae9fd0dc44f9a2ab2d373bbc913f` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:47:09.086259 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69774d6c-24c0-48be-bef1-794c552af1de None admin] RBAC: Authorizing `identity:get_project(project_id=c05dae9fd0dc44f9a2ab2d373bbc913f)` {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 15:47:09.088067 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69774d6c-24c0-48be-bef1-794c552af1de None admin] RBAC: Authorization granted {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 15:47:09.090547 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 202/413] 10.222.0.15 () {56 vars in 1186 bytes} [Mon Aug 22 15:47:08 2022] GET /identity/v3/projects/c05dae9fd0dc44f9a2ab2d373bbc913f => generated 302 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 15:47:09.096736 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69774d6c-24c0-48be-bef1-794c552af1de None admin] Authenticating user token {{(pid=81767) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 15:47:09.175289 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a08a7780-ea6a-4634-8ff4-e15a8c8699ae None None] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 15:47:09.211336 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54558339-0ffd-4f0a-b363-cc04e2d36313 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 15:47:09.212081 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54558339-0ffd-4f0a-b363-cc04e2d36313 None admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:47:09.212420 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54558339-0ffd-4f0a-b363-cc04e2d36313 None admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:47:09.212761 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54558339-0ffd-4f0a-b363-cc04e2d36313 None admin] PATH_INFO: `/v3/groups/f064c547078a42a0b477e6b3f08a751d` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:47:09.214062 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54558339-0ffd-4f0a-b363-cc04e2d36313 None admin] RBAC: Authorizing `identity:get_group(group_id=f064c547078a42a0b477e6b3f08a751d)` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 15:47:09.216102 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54558339-0ffd-4f0a-b363-cc04e2d36313 None admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 15:47:09.218034 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 212/414] 10.222.0.15 () {56 vars in 1180 bytes} [Mon Aug 22 15:47:09 2022] GET /identity/v3/groups/f064c547078a42a0b477e6b3f08a751d => generated 229 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 15:47:09.220677 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69774d6c-24c0-48be-bef1-794c552af1de None admin] Validating token access rules against request {{(pid=81767) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 15:47:09.223304 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54558339-0ffd-4f0a-b363-cc04e2d36313 None admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 15:47:09.263496 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81ab56f2-9d41-4781-8bae-aabfe69d0efb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81767) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 15:47:09.264591 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81ab56f2-9d41-4781-8bae-aabfe69d0efb None admin] REQUEST_METHOD: `PUT` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:47:09.265089 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81ab56f2-9d41-4781-8bae-aabfe69d0efb None admin] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:47:09.265571 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81ab56f2-9d41-4781-8bae-aabfe69d0efb None admin] PATH_INFO: `/v3/projects/c05dae9fd0dc44f9a2ab2d373bbc913f/groups/552e4a454dcf434ba77a8567804e8230/roles/1ce6b924c9604048906c07dbfb126da9` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:47:09.268241 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81ab56f2-9d41-4781-8bae-aabfe69d0efb None admin] RBAC: Authorizing `identity:create_grant(project_id=c05dae9fd0dc44f9a2ab2d373bbc913f, group_id=552e4a454dcf434ba77a8567804e8230, role_id=1ce6b924c9604048906c07dbfb126da9)` {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 15:47:09.270113 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81ab56f2-9d41-4781-8bae-aabfe69d0efb None admin] RBAC: Authorization granted {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 15:47:09.283879 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=100 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 15:47:09.329806 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54558339-0ffd-4f0a-b363-cc04e2d36313 None admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 15:47:09.365143 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1db8c5de-c593-42c9-a8ae-efd9823ff482 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 15:47:09.365777 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1db8c5de-c593-42c9-a8ae-efd9823ff482 None admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:47:09.366049 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1db8c5de-c593-42c9-a8ae-efd9823ff482 None admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:47:09.366312 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1db8c5de-c593-42c9-a8ae-efd9823ff482 None admin] PATH_INFO: `/v3/projects/9a98b1e4431940ab8d1fc95df2858a6c` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:47:09.367461 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1db8c5de-c593-42c9-a8ae-efd9823ff482 None admin] RBAC: Authorizing `identity:get_project(project_id=9a98b1e4431940ab8d1fc95df2858a6c)` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 15:47:09.368905 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1db8c5de-c593-42c9-a8ae-efd9823ff482 None admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 15:47:09.375407 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 213/415] 10.222.0.15 () {56 vars in 1186 bytes} [Mon Aug 22 15:47:09 2022] GET /identity/v3/projects/9a98b1e4431940ab8d1fc95df2858a6c => generated 344 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 15:47:09.383334 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG dbcounter [-] [81767] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81767) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 15:47:09.383797 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1db8c5de-c593-42c9-a8ae-efd9823ff482 None admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 15:47:09.389526 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 203/416] 10.222.0.15 () {58 vars in 1442 bytes} [Mon Aug 22 15:47:09 2022] PUT /identity/v3/projects/c05dae9fd0dc44f9a2ab2d373bbc913f/groups/552e4a454dcf434ba77a8567804e8230/roles/1ce6b924c9604048906c07dbfb126da9 => generated 0 bytes in 293 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 22 15:47:09.449311 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81ab56f2-9d41-4781-8bae-aabfe69d0efb None admin] Authenticating user token {{(pid=81767) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 15:47:09.493949 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1db8c5de-c593-42c9-a8ae-efd9823ff482 None admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 15:47:09.519363 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=100 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 15:47:09.537897 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4af0f75a-79f6-444c-9f17-62a9353b17a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 15:47:09.539250 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4af0f75a-79f6-444c-9f17-62a9353b17a9 None admin] REQUEST_METHOD: `PUT` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:47:09.539250 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4af0f75a-79f6-444c-9f17-62a9353b17a9 None admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:47:09.539250 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4af0f75a-79f6-444c-9f17-62a9353b17a9 None admin] PATH_INFO: `/v3/projects/9a98b1e4431940ab8d1fc95df2858a6c/groups/f064c547078a42a0b477e6b3f08a751d/roles/9d9848cfd22e4b089db443c350e56670` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:47:09.546257 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4af0f75a-79f6-444c-9f17-62a9353b17a9 None admin] RBAC: Authorizing `identity:create_grant(project_id=9a98b1e4431940ab8d1fc95df2858a6c, group_id=f064c547078a42a0b477e6b3f08a751d, role_id=9d9848cfd22e4b089db443c350e56670)` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 15:47:09.547752 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4af0f75a-79f6-444c-9f17-62a9353b17a9 None admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 15:47:09.627194 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81ab56f2-9d41-4781-8bae-aabfe69d0efb None admin] Validating token access rules against request {{(pid=81767) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 15:47:09.665154 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3884f95d-cbf2-4cdf-88e2-5c091e330363 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81767) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 15:47:09.666149 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3884f95d-cbf2-4cdf-88e2-5c091e330363 None admin] REQUEST_METHOD: `GET` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:47:09.666657 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3884f95d-cbf2-4cdf-88e2-5c091e330363 None admin] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:47:09.667176 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3884f95d-cbf2-4cdf-88e2-5c091e330363 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:47:09.679669 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3884f95d-cbf2-4cdf-88e2-5c091e330363 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 22 15:47:09.685021 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 214/417] 10.222.0.15 () {58 vars in 1442 bytes} [Mon Aug 22 15:47:09 2022] PUT /identity/v3/projects/9a98b1e4431940ab8d1fc95df2858a6c/groups/f064c547078a42a0b477e6b3f08a751d/roles/9d9848cfd22e4b089db443c350e56670 => generated 0 bytes in 308 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 22 15:47:09.686514 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application [None req-3884f95d-cbf2-4cdf-88e2-5c091e330363 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 22 15:47:09.686514 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 15:47:09.686514 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 15:47:09.686514 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 15:47:09.686514 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 15:47:09.686514 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 15:47:09.686514 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 15:47:09.686514 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 15:47:09.686514 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 15:47:09.686514 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 15:47:09.686514 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 15:47:09.686514 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 15:47:09.686514 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 22 15:47:09.686514 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 22 15:47:09.686514 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 22 15:47:09.686514 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 22 15:47:09.687435 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 22 15:47:09.687435 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 22 15:47:09.687435 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 22 15:47:09.687435 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 22 15:47:09.687435 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 15:47:09.687435 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 15:47:09.687435 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 15:47:09.687435 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 15:47:09.687435 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 15:47:09.687435 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 15:47:09.687435 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 15:47:09.687435 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application with Lock( Aug 22 15:47:09.687435 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 15:47:09.687435 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return self._enter() Aug 22 15:47:09.687435 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 15:47:09.687435 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 15:47:09.688262 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 15:47:09.688262 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return self.creator() Aug 22 15:47:09.688262 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 15:47:09.688262 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application created_value = creator( Aug 22 15:47:09.688262 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 22 15:47:09.688262 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 22 15:47:09.688262 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 22 15:47:09.688262 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 22 15:47:09.688262 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 22 15:47:09.688262 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 22 15:47:09.688262 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 22 15:47:09.688262 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application Aug 22 15:47:09.691207 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 204/418] 10.222.0.15 () {56 vars in 1099 bytes} [Mon Aug 22 15:47:09 2022] GET /identity/v3/roles/member => generated 84 bytes in 243 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 22 15:47:09.699292 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4af0f75a-79f6-444c-9f17-62a9353b17a9 None admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 15:47:09.857560 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4af0f75a-79f6-444c-9f17-62a9353b17a9 None admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 15:47:09.909987 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90664112-b5d5-4487-8822-3498a4025abb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 15:47:09.910610 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90664112-b5d5-4487-8822-3498a4025abb None admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:47:09.910891 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90664112-b5d5-4487-8822-3498a4025abb None admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:47:09.911165 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90664112-b5d5-4487-8822-3498a4025abb None admin] PATH_INFO: `/v3/roles` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:47:09.911869 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90664112-b5d5-4487-8822-3498a4025abb None admin] RBAC: Adding query filter params (name=member) {{(pid=81768) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 15:47:09.912174 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90664112-b5d5-4487-8822-3498a4025abb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 15:47:09.913607 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90664112-b5d5-4487-8822-3498a4025abb None admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 15:47:09.928041 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 215/419] 10.222.0.15 () {56 vars in 1101 bytes} [Mon Aug 22 15:47:09 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 15:47:09.934183 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3884f95d-cbf2-4cdf-88e2-5c091e330363 None admin] Authenticating user token {{(pid=81767) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 15:47:10.028436 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG dbcounter [-] [81767] Writing DB stats keystone:SELECT=100 {{(pid=81767) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 15:47:10.073982 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3884f95d-cbf2-4cdf-88e2-5c091e330363 None admin] Validating token access rules against request {{(pid=81767) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 15:47:10.112897 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c3ed517-944a-492c-812a-307892e14881 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81767) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 15:47:10.113477 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c3ed517-944a-492c-812a-307892e14881 None admin] REQUEST_METHOD: `GET` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:47:10.113825 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c3ed517-944a-492c-812a-307892e14881 None admin] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:47:10.114152 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c3ed517-944a-492c-812a-307892e14881 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:47:10.160021 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ca42252-31ab-450b-9a44-fece3e229141 None None] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:47:10.160971 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ca42252-31ab-450b-9a44-fece3e229141 None None] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:47:10.161466 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ca42252-31ab-450b-9a44-fece3e229141 None None] PATH_INFO: `/` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:47:10.163012 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 216/420] 10.222.0.15 () {54 vars in 927 bytes} [Mon Aug 22 15:47:10 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 15:47:10.168492 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef581048-9aee-4b42-b99e-243937b6ad8e None None] REQUEST_METHOD: `POST` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:47:10.168771 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef581048-9aee-4b42-b99e-243937b6ad8e None None] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:47:10.169042 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef581048-9aee-4b42-b99e-243937b6ad8e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:47:10.205551 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.auth.core [None req-ef581048-9aee-4b42-b99e-243937b6ad8e None None] MFA Rules not processed for user `6b2199ae0fb0480aa917ada391f5ec54`. Rule list: `[]` (Enabled: `True`). {{(pid=81768) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 15:47:10.206749 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-ef581048-9aee-4b42-b99e-243937b6ad8e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 15:47:10.210371 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c3ed517-944a-492c-812a-307892e14881 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 15:47:10.211885 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c3ed517-944a-492c-812a-307892e14881 None admin] RBAC: Authorization granted {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 15:47:10.263057 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 15:47:10.303501 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 217/421] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Aug 22 15:47:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 22 15:47:10.318746 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG dbcounter [-] [81767] Writing DB stats keystone:SELECT=100 {{(pid=81767) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 15:47:10.363872 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 205/422] 10.222.0.15 () {58 vars in 1282 bytes} [Mon Aug 22 15:47:09 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 431 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 22 15:47:10.369827 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef581048-9aee-4b42-b99e-243937b6ad8e None None] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 15:47:10.505202 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef581048-9aee-4b42-b99e-243937b6ad8e None None] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 15:47:10.540831 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc15122-4290-4c94-b9b2-7a9545adc631 None None] REQUEST_METHOD: `GET` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:47:10.541348 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc15122-4290-4c94-b9b2-7a9545adc631 None None] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:47:10.541348 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc15122-4290-4c94-b9b2-7a9545adc631 None None] PATH_INFO: `/` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:47:10.542936 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 206/423] 10.222.0.15 () {54 vars in 927 bytes} [Mon Aug 22 15:47:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 15:47:10.550154 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-043fdd11-8763-4843-8859-dba36e48f59b None None] REQUEST_METHOD: `POST` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:47:10.550310 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-043fdd11-8763-4843-8859-dba36e48f59b None None] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:47:10.550579 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-043fdd11-8763-4843-8859-dba36e48f59b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:47:10.552750 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6710ad7-288e-419a-8e1f-e201bf287446 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 15:47:10.553893 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6710ad7-288e-419a-8e1f-e201bf287446 None admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:47:10.554375 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6710ad7-288e-419a-8e1f-e201bf287446 None admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:47:10.554858 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6710ad7-288e-419a-8e1f-e201bf287446 None admin] PATH_INFO: `/v3/projects/43171f5fd3454380b88625276d7375da` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:47:10.556213 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6710ad7-288e-419a-8e1f-e201bf287446 None admin] RBAC: Authorizing `identity:get_project(project_id=43171f5fd3454380b88625276d7375da)` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 15:47:10.558222 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6710ad7-288e-419a-8e1f-e201bf287446 None admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 15:47:10.560496 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 218/424] 10.222.0.15 () {56 vars in 1186 bytes} [Mon Aug 22 15:47:10 2022] GET /identity/v3/projects/43171f5fd3454380b88625276d7375da => generated 298 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 15:47:10.572355 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6710ad7-288e-419a-8e1f-e201bf287446 None admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 15:47:10.581325 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.auth.core [None req-043fdd11-8763-4843-8859-dba36e48f59b None None] MFA Rules not processed for user `6b2199ae0fb0480aa917ada391f5ec54`. Rule list: `[]` (Enabled: `True`). {{(pid=81767) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 15:47:10.591299 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.fernet_utils [None req-043fdd11-8763-4843-8859-dba36e48f59b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81767) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 15:47:10.645660 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=100 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 15:47:10.663869 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 207/425] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Aug 22 15:47:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 22 15:47:10.671460 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-043fdd11-8763-4843-8859-dba36e48f59b None None] Authenticating user token {{(pid=81767) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 15:47:10.748385 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6710ad7-288e-419a-8e1f-e201bf287446 None admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 15:47:10.799793 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a33d432-3364-4c51-b6d0-60b7325c73ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 15:47:10.800436 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a33d432-3364-4c51-b6d0-60b7325c73ea None admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:47:10.800718 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a33d432-3364-4c51-b6d0-60b7325c73ea None admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:47:10.800997 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a33d432-3364-4c51-b6d0-60b7325c73ea None admin] PATH_INFO: `/v3/groups/552e4a454dcf434ba77a8567804e8230` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:47:10.803259 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a33d432-3364-4c51-b6d0-60b7325c73ea None admin] RBAC: Authorizing `identity:get_group(group_id=552e4a454dcf434ba77a8567804e8230)` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 15:47:10.804808 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a33d432-3364-4c51-b6d0-60b7325c73ea None admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 15:47:10.806658 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 219/426] 10.222.0.15 () {56 vars in 1180 bytes} [Mon Aug 22 15:47:10 2022] GET /identity/v3/groups/552e4a454dcf434ba77a8567804e8230 => generated 226 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 15:47:10.815862 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a33d432-3364-4c51-b6d0-60b7325c73ea None admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 15:47:10.835026 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG dbcounter [-] [81767] Writing DB stats keystone:SELECT=100 {{(pid=81767) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 15:47:10.846139 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-043fdd11-8763-4843-8859-dba36e48f59b None None] Validating token access rules against request {{(pid=81767) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 15:47:10.921020 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-094aed9c-50ac-47c5-8a1d-45f18fb49336 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81767) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 15:47:10.921586 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-094aed9c-50ac-47c5-8a1d-45f18fb49336 None admin] REQUEST_METHOD: `GET` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:47:10.921863 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-094aed9c-50ac-47c5-8a1d-45f18fb49336 None admin] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:47:10.922131 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-094aed9c-50ac-47c5-8a1d-45f18fb49336 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:47:10.948753 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-094aed9c-50ac-47c5-8a1d-45f18fb49336 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 22 15:47:10.954431 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application [None req-094aed9c-50ac-47c5-8a1d-45f18fb49336 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 22 15:47:10.954431 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 15:47:10.954431 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 15:47:10.954431 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 15:47:10.954431 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 15:47:10.954431 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 15:47:10.954431 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 15:47:10.954431 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 15:47:10.954431 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 15:47:10.954431 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 15:47:10.954431 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 15:47:10.954431 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 15:47:10.954431 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 22 15:47:10.954431 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 22 15:47:10.954431 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 22 15:47:10.954431 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 22 15:47:10.955314 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 22 15:47:10.955314 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 22 15:47:10.955314 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 22 15:47:10.955314 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 22 15:47:10.955314 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 15:47:10.955314 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 15:47:10.955314 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 15:47:10.955314 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 15:47:10.955314 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 15:47:10.955314 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 15:47:10.955314 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 15:47:10.955314 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application with Lock( Aug 22 15:47:10.955314 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 15:47:10.955314 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return self._enter() Aug 22 15:47:10.955314 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 15:47:10.955314 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 15:47:10.956111 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 15:47:10.956111 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return self.creator() Aug 22 15:47:10.956111 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 15:47:10.956111 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application created_value = creator( Aug 22 15:47:10.956111 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 22 15:47:10.956111 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 22 15:47:10.956111 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 22 15:47:10.956111 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 22 15:47:10.956111 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 22 15:47:10.956111 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 22 15:47:10.956111 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 22 15:47:10.956111 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application Aug 22 15:47:10.956111 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 208/427] 10.222.0.15 () {56 vars in 1114 bytes} [Mon Aug 22 15:47:10 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 288 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 15:47:10.963404 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-094aed9c-50ac-47c5-8a1d-45f18fb49336 None admin] Authenticating user token {{(pid=81767) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 15:47:10.978203 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a33d432-3364-4c51-b6d0-60b7325c73ea None admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 15:47:10.991380 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=100 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 15:47:11.021408 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f903ea6a-21c6-434f-8ce6-f8ca09ed785d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 15:47:11.022032 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f903ea6a-21c6-434f-8ce6-f8ca09ed785d None admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:47:11.022296 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f903ea6a-21c6-434f-8ce6-f8ca09ed785d None admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:47:11.022567 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f903ea6a-21c6-434f-8ce6-f8ca09ed785d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:47:11.023307 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f903ea6a-21c6-434f-8ce6-f8ca09ed785d None admin] RBAC: Adding query filter params (group.id=552e4a454dcf434ba77a8567804e8230, role.id=e6fc1ee3fb974125ba9b4fe55b12b45e, scope.project.id=43171f5fd3454380b88625276d7375da) {{(pid=81768) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 15:47:11.023673 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f903ea6a-21c6-434f-8ce6-f8ca09ed785d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 15:47:11.025164 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f903ea6a-21c6-434f-8ce6-f8ca09ed785d None admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 15:47:11.043404 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 220/428] 10.222.0.15 () {56 vars in 1376 bytes} [Mon Aug 22 15:47:10 2022] GET /identity/v3/role_assignments?group.id=552e4a454dcf434ba77a8567804e8230&scope.project.id=43171f5fd3454380b88625276d7375da&role.id=e6fc1ee3fb974125ba9b4fe55b12b45e => generated 611 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 15:47:11.044320 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f903ea6a-21c6-434f-8ce6-f8ca09ed785d None admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 15:47:11.175886 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f903ea6a-21c6-434f-8ce6-f8ca09ed785d None admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 15:47:11.219188 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d623d5d-9477-4ff3-ae03-cd15dc5a7f43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 15:47:11.220001 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d623d5d-9477-4ff3-ae03-cd15dc5a7f43 None admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:47:11.220281 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d623d5d-9477-4ff3-ae03-cd15dc5a7f43 None admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:47:11.220552 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d623d5d-9477-4ff3-ae03-cd15dc5a7f43 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:47:11.234678 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d623d5d-9477-4ff3-ae03-cd15dc5a7f43 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 22 15:47:11.240756 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application [None req-8d623d5d-9477-4ff3-ae03-cd15dc5a7f43 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 22 15:47:11.240756 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 15:47:11.240756 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 15:47:11.240756 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 15:47:11.240756 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 15:47:11.240756 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 15:47:11.240756 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 15:47:11.240756 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 15:47:11.240756 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 15:47:11.240756 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 15:47:11.240756 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 15:47:11.240756 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 15:47:11.240756 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 22 15:47:11.240756 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 22 15:47:11.240756 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 22 15:47:11.240756 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 22 15:47:11.241689 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 22 15:47:11.241689 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 22 15:47:11.241689 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 22 15:47:11.241689 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 22 15:47:11.241689 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 15:47:11.241689 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 15:47:11.241689 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 15:47:11.241689 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 15:47:11.241689 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 15:47:11.241689 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 15:47:11.241689 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 15:47:11.241689 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application with Lock( Aug 22 15:47:11.241689 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 15:47:11.241689 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self._enter() Aug 22 15:47:11.241689 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 15:47:11.241689 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 15:47:11.242458 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 15:47:11.242458 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self.creator() Aug 22 15:47:11.242458 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 15:47:11.242458 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application created_value = creator( Aug 22 15:47:11.242458 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 22 15:47:11.242458 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 22 15:47:11.242458 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 22 15:47:11.242458 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 22 15:47:11.242458 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 22 15:47:11.242458 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 22 15:47:11.242458 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 22 15:47:11.242458 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application Aug 22 15:47:11.242458 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 221/429] 10.222.0.15 () {56 vars in 1099 bytes} [Mon Aug 22 15:47:11 2022] GET /identity/v3/roles/member => generated 84 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 15:47:11.249964 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d623d5d-9477-4ff3-ae03-cd15dc5a7f43 None admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 15:47:11.255441 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-094aed9c-50ac-47c5-8a1d-45f18fb49336 None admin] Validating token access rules against request {{(pid=81767) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 15:47:11.291646 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=100 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 15:47:11.313439 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-028244d2-1b44-43d4-a534-a581fa831322 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81767) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 15:47:11.314048 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-028244d2-1b44-43d4-a534-a581fa831322 None admin] REQUEST_METHOD: `GET` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:47:11.314358 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-028244d2-1b44-43d4-a534-a581fa831322 None admin] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:47:11.314663 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-028244d2-1b44-43d4-a534-a581fa831322 None admin] PATH_INFO: `/v3/roles` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:47:11.315361 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-028244d2-1b44-43d4-a534-a581fa831322 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81767) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 15:47:11.315656 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-028244d2-1b44-43d4-a534-a581fa831322 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 15:47:11.317029 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-028244d2-1b44-43d4-a534-a581fa831322 None admin] RBAC: Authorization granted {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 15:47:11.327508 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 209/430] 10.222.0.15 () {56 vars in 1111 bytes} [Mon Aug 22 15:47:10 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 364 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 22 15:47:11.336127 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-028244d2-1b44-43d4-a534-a581fa831322 None admin] Authenticating user token {{(pid=81767) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 15:47:11.371712 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG dbcounter [-] [81767] Writing DB stats keystone:SELECT=100 {{(pid=81767) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 15:47:11.395085 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d623d5d-9477-4ff3-ae03-cd15dc5a7f43 None admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 15:47:11.441115 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55d70a10-cd95-43be-a034-05f18c46e685 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 15:47:11.441798 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55d70a10-cd95-43be-a034-05f18c46e685 None admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:47:11.442172 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55d70a10-cd95-43be-a034-05f18c46e685 None admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:47:11.442538 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55d70a10-cd95-43be-a034-05f18c46e685 None admin] PATH_INFO: `/v3/roles` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:47:11.443686 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55d70a10-cd95-43be-a034-05f18c46e685 None admin] RBAC: Adding query filter params (name=member) {{(pid=81768) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 15:47:11.444083 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55d70a10-cd95-43be-a034-05f18c46e685 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 15:47:11.446130 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55d70a10-cd95-43be-a034-05f18c46e685 None admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 15:47:11.458064 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 222/431] 10.222.0.15 () {56 vars in 1101 bytes} [Mon Aug 22 15:47:11 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 15:47:11.464960 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55d70a10-cd95-43be-a034-05f18c46e685 None admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 15:47:11.474255 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-028244d2-1b44-43d4-a534-a581fa831322 None admin] Validating token access rules against request {{(pid=81767) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 15:47:11.524366 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0ec621e-dd91-441f-9fc0-ae269b9f46f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81767) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 15:47:11.525015 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0ec621e-dd91-441f-9fc0-ae269b9f46f3 None admin] REQUEST_METHOD: `GET` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:47:11.525293 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0ec621e-dd91-441f-9fc0-ae269b9f46f3 None admin] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:47:11.525721 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0ec621e-dd91-441f-9fc0-ae269b9f46f3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:47:11.585354 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=100 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 15:47:11.592887 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55d70a10-cd95-43be-a034-05f18c46e685 None admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 15:47:11.610853 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0ec621e-dd91-441f-9fc0-ae269b9f46f3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 15:47:11.612876 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0ec621e-dd91-441f-9fc0-ae269b9f46f3 None admin] RBAC: Authorization granted {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 15:47:11.629707 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG dbcounter [-] [81767] Writing DB stats keystone:SELECT=100 {{(pid=81767) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 15:47:11.630494 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b917faa2-0f4d-4095-8048-d43acce1b778 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 15:47:11.630898 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b917faa2-0f4d-4095-8048-d43acce1b778 None admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:47:11.631184 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b917faa2-0f4d-4095-8048-d43acce1b778 None admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:47:11.631493 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b917faa2-0f4d-4095-8048-d43acce1b778 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:47:11.709541 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b917faa2-0f4d-4095-8048-d43acce1b778 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 15:47:11.710942 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b917faa2-0f4d-4095-8048-d43acce1b778 None admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 15:47:11.736319 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 210/432] 10.222.0.15 () {58 vars in 1282 bytes} [Mon Aug 22 15:47:11 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 404 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 22 15:47:11.737612 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f18a3de0-3ce6-41cf-8f65-03fa297139a0 None None] REQUEST_METHOD: `GET` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:47:11.737893 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f18a3de0-3ce6-41cf-8f65-03fa297139a0 None None] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:47:11.738163 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f18a3de0-3ce6-41cf-8f65-03fa297139a0 None None] PATH_INFO: `/` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:47:11.741565 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 211/433] 10.222.0.15 () {54 vars in 927 bytes} [Mon Aug 22 15:47:11 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 15:47:11.743758 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f18a3de0-3ce6-41cf-8f65-03fa297139a0 None None] Authenticating user token {{(pid=81767) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 15:47:11.841033 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=100 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 15:47:11.849559 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 223/434] 10.222.0.15 () {58 vars in 1282 bytes} [Mon Aug 22 15:47:11 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 386 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 22 15:47:11.851093 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f44150-6242-4545-938b-ee4ab850f048 None None] REQUEST_METHOD: `POST` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:47:11.853143 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f44150-6242-4545-938b-ee4ab850f048 None None] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:47:11.853143 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f44150-6242-4545-938b-ee4ab850f048 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:47:11.877597 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.auth.core [None req-c1f44150-6242-4545-938b-ee4ab850f048 None None] MFA Rules not processed for user `6b2199ae0fb0480aa917ada391f5ec54`. Rule list: `[]` (Enabled: `True`). {{(pid=81768) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 15:47:11.878860 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-c1f44150-6242-4545-938b-ee4ab850f048 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 15:47:11.883998 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f18a3de0-3ce6-41cf-8f65-03fa297139a0 None None] Validating token access rules against request {{(pid=81767) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 15:47:11.898465 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG dbcounter [-] [81767] Writing DB stats keystone:SELECT=100 {{(pid=81767) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 15:47:11.923367 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00ae43c3-3cd4-438a-abed-ddf724fad4d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81767) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 15:47:11.924000 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00ae43c3-3cd4-438a-abed-ddf724fad4d6 None admin] REQUEST_METHOD: `GET` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:47:11.924277 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00ae43c3-3cd4-438a-abed-ddf724fad4d6 None admin] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:47:11.924549 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00ae43c3-3cd4-438a-abed-ddf724fad4d6 None admin] PATH_INFO: `/v3/projects/43171f5fd3454380b88625276d7375da` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:47:11.925694 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00ae43c3-3cd4-438a-abed-ddf724fad4d6 None admin] RBAC: Authorizing `identity:get_project(project_id=43171f5fd3454380b88625276d7375da)` {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 15:47:11.927131 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00ae43c3-3cd4-438a-abed-ddf724fad4d6 None admin] RBAC: Authorization granted {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 15:47:11.933643 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 212/435] 10.222.0.15 () {56 vars in 1186 bytes} [Mon Aug 22 15:47:11 2022] GET /identity/v3/projects/43171f5fd3454380b88625276d7375da => generated 298 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 15:47:11.933643 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00ae43c3-3cd4-438a-abed-ddf724fad4d6 None admin] Authenticating user token {{(pid=81767) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 15:47:11.971664 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 224/436] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Aug 22 15:47:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 22 15:47:11.972474 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1f44150-6242-4545-938b-ee4ab850f048 None None] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 15:47:12.046700 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00ae43c3-3cd4-438a-abed-ddf724fad4d6 None admin] Validating token access rules against request {{(pid=81767) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 15:47:12.084120 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2735ef68-2898-4cc1-b454-be4fc060eb29 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81767) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 15:47:12.085071 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2735ef68-2898-4cc1-b454-be4fc060eb29 None admin] REQUEST_METHOD: `GET` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:47:12.085447 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2735ef68-2898-4cc1-b454-be4fc060eb29 None admin] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:47:12.085802 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2735ef68-2898-4cc1-b454-be4fc060eb29 None admin] PATH_INFO: `/v3/projects/c05dae9fd0dc44f9a2ab2d373bbc913f` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:47:12.086984 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2735ef68-2898-4cc1-b454-be4fc060eb29 None admin] RBAC: Authorizing `identity:get_project(project_id=c05dae9fd0dc44f9a2ab2d373bbc913f)` {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 15:47:12.088561 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1f44150-6242-4545-938b-ee4ab850f048 None None] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 15:47:12.088674 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2735ef68-2898-4cc1-b454-be4fc060eb29 None admin] RBAC: Authorization granted {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 15:47:12.091335 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 213/437] 10.222.0.15 () {56 vars in 1186 bytes} [Mon Aug 22 15:47:11 2022] GET /identity/v3/projects/c05dae9fd0dc44f9a2ab2d373bbc913f => generated 302 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 15:47:12.096978 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2735ef68-2898-4cc1-b454-be4fc060eb29 None admin] Authenticating user token {{(pid=81767) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 15:47:12.116240 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=100 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 15:47:12.128074 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94a06751-a27d-43ec-a173-06712a56c989 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 15:47:12.129025 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94a06751-a27d-43ec-a173-06712a56c989 None admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:47:12.140987 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94a06751-a27d-43ec-a173-06712a56c989 None admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:47:12.140987 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94a06751-a27d-43ec-a173-06712a56c989 None admin] PATH_INFO: `/v3/groups/552e4a454dcf434ba77a8567804e8230` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:47:12.140987 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94a06751-a27d-43ec-a173-06712a56c989 None admin] RBAC: Authorizing `identity:get_group(group_id=552e4a454dcf434ba77a8567804e8230)` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 15:47:12.140987 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94a06751-a27d-43ec-a173-06712a56c989 None admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 15:47:12.140987 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 225/438] 10.222.0.15 () {56 vars in 1180 bytes} [Mon Aug 22 15:47:11 2022] GET /identity/v3/groups/552e4a454dcf434ba77a8567804e8230 => generated 226 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 15:47:12.140987 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94a06751-a27d-43ec-a173-06712a56c989 None admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 15:47:12.163396 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG dbcounter [-] [81767] Writing DB stats keystone:SELECT=100 {{(pid=81767) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 15:47:12.241742 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2735ef68-2898-4cc1-b454-be4fc060eb29 None admin] Validating token access rules against request {{(pid=81767) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 15:47:12.264349 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94a06751-a27d-43ec-a173-06712a56c989 None admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 15:47:12.276478 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a62c9a6-a8a4-42a2-be40-d757a0790020 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81767) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 15:47:12.277639 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a62c9a6-a8a4-42a2-be40-d757a0790020 None admin] REQUEST_METHOD: `GET` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:47:12.277912 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a62c9a6-a8a4-42a2-be40-d757a0790020 None admin] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:47:12.278174 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a62c9a6-a8a4-42a2-be40-d757a0790020 None admin] PATH_INFO: `/v3/groups/552e4a454dcf434ba77a8567804e8230` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:47:12.279340 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a62c9a6-a8a4-42a2-be40-d757a0790020 None admin] RBAC: Authorizing `identity:get_group(group_id=552e4a454dcf434ba77a8567804e8230)` {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 15:47:12.280771 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a62c9a6-a8a4-42a2-be40-d757a0790020 None admin] RBAC: Authorization granted {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 15:47:12.287402 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 214/439] 10.222.0.15 () {56 vars in 1180 bytes} [Mon Aug 22 15:47:12 2022] GET /identity/v3/groups/552e4a454dcf434ba77a8567804e8230 => generated 226 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 15:47:12.287402 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a62c9a6-a8a4-42a2-be40-d757a0790020 None admin] Authenticating user token {{(pid=81767) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 15:47:12.308950 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55506b0b-8170-4afa-be96-8658d83796fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 15:47:12.309553 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55506b0b-8170-4afa-be96-8658d83796fe None admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:47:12.309831 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55506b0b-8170-4afa-be96-8658d83796fe None admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:47:12.310109 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55506b0b-8170-4afa-be96-8658d83796fe None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:47:12.310860 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55506b0b-8170-4afa-be96-8658d83796fe None admin] RBAC: Adding query filter params (group.id=552e4a454dcf434ba77a8567804e8230, role.id=1ce6b924c9604048906c07dbfb126da9, scope.project.id=43171f5fd3454380b88625276d7375da) {{(pid=81768) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 15:47:12.311238 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55506b0b-8170-4afa-be96-8658d83796fe None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 15:47:12.312709 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55506b0b-8170-4afa-be96-8658d83796fe None admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 15:47:12.318458 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 226/440] 10.222.0.15 () {56 vars in 1376 bytes} [Mon Aug 22 15:47:12 2022] GET /identity/v3/role_assignments?group.id=552e4a454dcf434ba77a8567804e8230&scope.project.id=43171f5fd3454380b88625276d7375da&role.id=1ce6b924c9604048906c07dbfb126da9 => generated 611 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 15:47:12.319619 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9bf3713-b9dd-4bc9-8beb-2814f01c35ff None None] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:47:12.319898 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9bf3713-b9dd-4bc9-8beb-2814f01c35ff None None] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:47:12.320166 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9bf3713-b9dd-4bc9-8beb-2814f01c35ff None None] PATH_INFO: `/` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:47:12.321363 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 227/441] 10.222.0.15 () {54 vars in 927 bytes} [Mon Aug 22 15:47:12 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 15:47:12.322084 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9bf3713-b9dd-4bc9-8beb-2814f01c35ff None None] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 15:47:12.408138 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=100 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 15:47:12.421923 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a62c9a6-a8a4-42a2-be40-d757a0790020 None admin] Validating token access rules against request {{(pid=81767) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 15:47:12.442389 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG dbcounter [-] [81767] Writing DB stats keystone:SELECT=100 {{(pid=81767) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 15:47:12.462361 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-631f4b32-a977-45b1-925b-122d894c7bcf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81767) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 15:47:12.462747 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-631f4b32-a977-45b1-925b-122d894c7bcf None admin] REQUEST_METHOD: `GET` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:47:12.463025 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-631f4b32-a977-45b1-925b-122d894c7bcf None admin] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:47:12.463331 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-631f4b32-a977-45b1-925b-122d894c7bcf None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:47:12.471764 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9bf3713-b9dd-4bc9-8beb-2814f01c35ff None None] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 15:47:12.477139 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-631f4b32-a977-45b1-925b-122d894c7bcf None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 22 15:47:12.484272 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application [None req-631f4b32-a977-45b1-925b-122d894c7bcf None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 22 15:47:12.484272 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 15:47:12.484272 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 15:47:12.484272 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 15:47:12.484272 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 15:47:12.484272 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 15:47:12.484272 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 15:47:12.484272 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 15:47:12.484272 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 15:47:12.484272 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 15:47:12.484272 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 15:47:12.484272 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 15:47:12.484272 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 22 15:47:12.484272 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 22 15:47:12.484272 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 22 15:47:12.484272 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 22 15:47:12.485364 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 22 15:47:12.485364 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 22 15:47:12.485364 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 22 15:47:12.485364 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 22 15:47:12.485364 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 15:47:12.485364 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 15:47:12.485364 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 15:47:12.485364 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 15:47:12.485364 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 15:47:12.485364 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 15:47:12.485364 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 15:47:12.485364 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application with Lock( Aug 22 15:47:12.485364 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 15:47:12.485364 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return self._enter() Aug 22 15:47:12.485364 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 15:47:12.485364 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 15:47:12.486239 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 15:47:12.486239 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return self.creator() Aug 22 15:47:12.486239 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 15:47:12.486239 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application created_value = creator( Aug 22 15:47:12.486239 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 22 15:47:12.486239 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 22 15:47:12.486239 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 22 15:47:12.486239 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 22 15:47:12.486239 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 22 15:47:12.486239 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 22 15:47:12.486239 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 22 15:47:12.486239 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application Aug 22 15:47:12.487642 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 215/442] 10.222.0.15 () {56 vars in 1114 bytes} [Mon Aug 22 15:47:12 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 15:47:12.489060 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8299c9e-f869-4a2f-9887-26f659f8c203 None None] REQUEST_METHOD: `POST` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:47:12.489339 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8299c9e-f869-4a2f-9887-26f659f8c203 None None] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:47:12.489605 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8299c9e-f869-4a2f-9887-26f659f8c203 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:47:12.513697 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4571177b-6e46-4c45-a1d3-51b18f47a989 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 15:47:12.517074 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4571177b-6e46-4c45-a1d3-51b18f47a989 None admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:47:12.517223 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.auth.core [None req-d8299c9e-f869-4a2f-9887-26f659f8c203 None None] MFA Rules not processed for user `6b2199ae0fb0480aa917ada391f5ec54`. Rule list: `[]` (Enabled: `True`). {{(pid=81767) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 15:47:12.517305 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4571177b-6e46-4c45-a1d3-51b18f47a989 None admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:47:12.517305 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4571177b-6e46-4c45-a1d3-51b18f47a989 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:47:12.517432 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.fernet_utils [None req-d8299c9e-f869-4a2f-9887-26f659f8c203 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81767) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 15:47:12.517501 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4571177b-6e46-4c45-a1d3-51b18f47a989 None admin] RBAC: Adding query filter params (group.id=552e4a454dcf434ba77a8567804e8230, role.id=e6fc1ee3fb974125ba9b4fe55b12b45e, scope.project.id=c05dae9fd0dc44f9a2ab2d373bbc913f) {{(pid=81768) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 15:47:12.517501 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4571177b-6e46-4c45-a1d3-51b18f47a989 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 15:47:12.517831 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4571177b-6e46-4c45-a1d3-51b18f47a989 None admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 15:47:12.523836 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 228/443] 10.222.0.15 () {56 vars in 1376 bytes} [Mon Aug 22 15:47:12 2022] GET /identity/v3/role_assignments?group.id=552e4a454dcf434ba77a8567804e8230&scope.project.id=c05dae9fd0dc44f9a2ab2d373bbc913f&role.id=e6fc1ee3fb974125ba9b4fe55b12b45e => generated 611 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 15:47:12.524635 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4571177b-6e46-4c45-a1d3-51b18f47a989 None admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 15:47:12.617654 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 216/444] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Aug 22 15:47:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 22 15:47:12.673282 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4571177b-6e46-4c45-a1d3-51b18f47a989 None admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 15:47:12.695556 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=100 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 15:47:12.711632 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d875c5d-e730-4324-9925-8d7ecb09a8ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 15:47:12.712620 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d875c5d-e730-4324-9925-8d7ecb09a8ae None admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:47:12.713133 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d875c5d-e730-4324-9925-8d7ecb09a8ae None admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:47:12.713627 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d875c5d-e730-4324-9925-8d7ecb09a8ae None admin] PATH_INFO: `/v3/roles` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:47:12.714557 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d875c5d-e730-4324-9925-8d7ecb09a8ae None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81768) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 15:47:12.715090 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d875c5d-e730-4324-9925-8d7ecb09a8ae None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 15:47:12.716504 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d875c5d-e730-4324-9925-8d7ecb09a8ae None admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 15:47:12.726153 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 229/445] 10.222.0.15 () {56 vars in 1111 bytes} [Mon Aug 22 15:47:12 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 15:47:12.732390 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8299c9e-f869-4a2f-9887-26f659f8c203 None None] Authenticating user token {{(pid=81767) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 15:47:12.854388 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8299c9e-f869-4a2f-9887-26f659f8c203 None None] Validating token access rules against request {{(pid=81767) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 15:47:12.863258 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG dbcounter [-] [81767] Writing DB stats keystone:SELECT=100 {{(pid=81767) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 15:47:12.892901 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1567acf5-281a-4e4c-b33c-e3b712fdd6fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81767) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 15:47:12.893509 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1567acf5-281a-4e4c-b33c-e3b712fdd6fe None admin] REQUEST_METHOD: `GET` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:47:12.893788 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1567acf5-281a-4e4c-b33c-e3b712fdd6fe None admin] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:47:12.894058 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1567acf5-281a-4e4c-b33c-e3b712fdd6fe None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:47:12.905824 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d875c5d-e730-4324-9925-8d7ecb09a8ae None admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 15:47:12.970194 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1567acf5-281a-4e4c-b33c-e3b712fdd6fe None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 15:47:12.971720 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1567acf5-281a-4e4c-b33c-e3b712fdd6fe None admin] RBAC: Authorization granted {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 15:47:13.016098 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d875c5d-e730-4324-9925-8d7ecb09a8ae None admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 15:47:13.050196 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b30153bc-89af-4a86-b556-523c73a801f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 15:47:13.050846 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b30153bc-89af-4a86-b556-523c73a801f2 None admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:47:13.051124 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b30153bc-89af-4a86-b556-523c73a801f2 None admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:47:13.051424 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b30153bc-89af-4a86-b556-523c73a801f2 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:47:13.062970 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b30153bc-89af-4a86-b556-523c73a801f2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 22 15:47:13.068727 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application [None req-b30153bc-89af-4a86-b556-523c73a801f2 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 22 15:47:13.068727 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 15:47:13.068727 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 15:47:13.068727 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 15:47:13.068727 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 15:47:13.068727 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 15:47:13.068727 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 15:47:13.068727 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 15:47:13.068727 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 15:47:13.068727 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 15:47:13.068727 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 15:47:13.068727 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 15:47:13.068727 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 22 15:47:13.068727 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 22 15:47:13.068727 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 22 15:47:13.068727 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 22 15:47:13.069782 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 22 15:47:13.069782 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 22 15:47:13.069782 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 22 15:47:13.069782 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 22 15:47:13.069782 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 15:47:13.069782 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 15:47:13.069782 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 15:47:13.069782 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 15:47:13.069782 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 15:47:13.069782 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 15:47:13.069782 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 15:47:13.069782 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application with Lock( Aug 22 15:47:13.069782 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 15:47:13.069782 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self._enter() Aug 22 15:47:13.069782 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 15:47:13.069782 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 15:47:13.070599 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 15:47:13.070599 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self.creator() Aug 22 15:47:13.070599 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 15:47:13.070599 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application created_value = creator( Aug 22 15:47:13.070599 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 22 15:47:13.070599 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 22 15:47:13.070599 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 22 15:47:13.070599 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 22 15:47:13.070599 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 22 15:47:13.070599 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 22 15:47:13.070599 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 22 15:47:13.070599 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application Aug 22 15:47:13.071198 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 217/446] 10.222.0.15 () {58 vars in 1282 bytes} [Mon Aug 22 15:47:12 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 337 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 22 15:47:13.071269 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 230/447] 10.222.0.15 () {56 vars in 1096 bytes} [Mon Aug 22 15:47:12 2022] GET /identity/v3/roles/admin => generated 83 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 22 15:47:13.075703 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1567acf5-281a-4e4c-b33c-e3b712fdd6fe None admin] Authenticating user token {{(pid=81767) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 15:47:13.076159 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b30153bc-89af-4a86-b556-523c73a801f2 None admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 15:47:13.083538 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG dbcounter [-] [81767] Writing DB stats keystone:SELECT=100 {{(pid=81767) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 15:47:13.119767 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=100 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 15:47:13.178222 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1567acf5-281a-4e4c-b33c-e3b712fdd6fe None admin] Validating token access rules against request {{(pid=81767) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 15:47:13.178564 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b30153bc-89af-4a86-b556-523c73a801f2 None admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 15:47:13.213909 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bff85591-a980-49d8-bb58-85eec07f0852 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 15:47:13.214528 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bff85591-a980-49d8-bb58-85eec07f0852 None admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:47:13.214714 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bfb9a41-5d29-4bf2-bb4f-c16be94a2a48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81767) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 15:47:13.214820 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bff85591-a980-49d8-bb58-85eec07f0852 None admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:47:13.215060 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bff85591-a980-49d8-bb58-85eec07f0852 None admin] PATH_INFO: `/v3/roles` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:47:13.215361 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bfb9a41-5d29-4bf2-bb4f-c16be94a2a48 None admin] REQUEST_METHOD: `GET` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:47:13.215636 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bfb9a41-5d29-4bf2-bb4f-c16be94a2a48 None admin] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:47:13.215749 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bff85591-a980-49d8-bb58-85eec07f0852 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81768) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 15:47:13.215904 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bfb9a41-5d29-4bf2-bb4f-c16be94a2a48 None admin] PATH_INFO: `/v3/projects/c05dae9fd0dc44f9a2ab2d373bbc913f` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:47:13.216047 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bff85591-a980-49d8-bb58-85eec07f0852 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 15:47:13.217032 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bfb9a41-5d29-4bf2-bb4f-c16be94a2a48 None admin] RBAC: Authorizing `identity:get_project(project_id=c05dae9fd0dc44f9a2ab2d373bbc913f)` {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 15:47:13.217419 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bff85591-a980-49d8-bb58-85eec07f0852 None admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 15:47:13.218443 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bfb9a41-5d29-4bf2-bb4f-c16be94a2a48 None admin] RBAC: Authorization granted {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 15:47:13.220120 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 218/448] 10.222.0.15 () {56 vars in 1186 bytes} [Mon Aug 22 15:47:13 2022] GET /identity/v3/projects/c05dae9fd0dc44f9a2ab2d373bbc913f => generated 302 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 22 15:47:13.225581 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bfb9a41-5d29-4bf2-bb4f-c16be94a2a48 None admin] Authenticating user token {{(pid=81767) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 15:47:13.226078 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 231/449] 10.222.0.15 () {56 vars in 1099 bytes} [Mon Aug 22 15:47:13 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 22 15:47:13.231412 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bff85591-a980-49d8-bb58-85eec07f0852 None admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 15:47:13.307904 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG dbcounter [-] [81767] Writing DB stats keystone:SELECT=100 {{(pid=81767) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 15:47:13.332124 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bfb9a41-5d29-4bf2-bb4f-c16be94a2a48 None admin] Validating token access rules against request {{(pid=81767) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 15:47:13.332314 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bff85591-a980-49d8-bb58-85eec07f0852 None admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 15:47:13.336066 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=100 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 15:47:13.366402 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cad7a94d-617d-4a00-937c-fda7eba2a3c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 15:47:13.367008 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cad7a94d-617d-4a00-937c-fda7eba2a3c8 None admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:47:13.367309 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cad7a94d-617d-4a00-937c-fda7eba2a3c8 None admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:47:13.367574 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cad7a94d-617d-4a00-937c-fda7eba2a3c8 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:47:13.367698 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43cd54ec-1656-48bd-afe0-119957e2116d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81767) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 15:47:13.368258 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43cd54ec-1656-48bd-afe0-119957e2116d None admin] REQUEST_METHOD: `GET` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:47:13.368576 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43cd54ec-1656-48bd-afe0-119957e2116d None admin] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:47:13.368935 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43cd54ec-1656-48bd-afe0-119957e2116d None admin] PATH_INFO: `/v3/groups/552e4a454dcf434ba77a8567804e8230` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:47:13.373718 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43cd54ec-1656-48bd-afe0-119957e2116d None admin] RBAC: Authorizing `identity:get_group(group_id=552e4a454dcf434ba77a8567804e8230)` {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 15:47:13.373718 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43cd54ec-1656-48bd-afe0-119957e2116d None admin] RBAC: Authorization granted {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 15:47:13.373718 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 219/450] 10.222.0.15 () {56 vars in 1180 bytes} [Mon Aug 22 15:47:13 2022] GET /identity/v3/groups/552e4a454dcf434ba77a8567804e8230 => generated 226 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 22 15:47:13.379688 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43cd54ec-1656-48bd-afe0-119957e2116d None admin] Authenticating user token {{(pid=81767) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 15:47:13.438123 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cad7a94d-617d-4a00-937c-fda7eba2a3c8 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 15:47:13.439697 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cad7a94d-617d-4a00-937c-fda7eba2a3c8 None admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 15:47:13.480746 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43cd54ec-1656-48bd-afe0-119957e2116d None admin] Validating token access rules against request {{(pid=81767) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 15:47:13.516043 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f52de86-d71b-4712-b1ec-aba7bc098320 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81767) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 15:47:13.516636 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f52de86-d71b-4712-b1ec-aba7bc098320 None admin] REQUEST_METHOD: `GET` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:47:13.516910 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f52de86-d71b-4712-b1ec-aba7bc098320 None admin] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:47:13.517180 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f52de86-d71b-4712-b1ec-aba7bc098320 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:47:13.517896 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f52de86-d71b-4712-b1ec-aba7bc098320 None admin] RBAC: Adding query filter params (group.id=552e4a454dcf434ba77a8567804e8230, role.id=1ce6b924c9604048906c07dbfb126da9, scope.project.id=c05dae9fd0dc44f9a2ab2d373bbc913f) {{(pid=81767) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 15:47:13.518203 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f52de86-d71b-4712-b1ec-aba7bc098320 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 15:47:13.519651 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f52de86-d71b-4712-b1ec-aba7bc098320 None admin] RBAC: Authorization granted {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 15:47:13.525358 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 220/451] 10.222.0.15 () {56 vars in 1376 bytes} [Mon Aug 22 15:47:13 2022] GET /identity/v3/role_assignments?group.id=552e4a454dcf434ba77a8567804e8230&scope.project.id=c05dae9fd0dc44f9a2ab2d373bbc913f&role.id=1ce6b924c9604048906c07dbfb126da9 => generated 611 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 22 15:47:13.539778 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 232/452] 10.222.0.15 () {58 vars in 1282 bytes} [Mon Aug 22 15:47:13 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 22 15:47:13.546668 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f52de86-d71b-4712-b1ec-aba7bc098320 None admin] Authenticating user token {{(pid=81767) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 15:47:13.556995 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG dbcounter [-] [81767] Writing DB stats keystone:SELECT=100 {{(pid=81767) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 15:47:13.662315 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f52de86-d71b-4712-b1ec-aba7bc098320 None admin] Validating token access rules against request {{(pid=81767) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 15:47:13.700042 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c95a36a3-a758-4000-8583-c27dd790bf23 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81767) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 15:47:13.700693 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c95a36a3-a758-4000-8583-c27dd790bf23 None admin] REQUEST_METHOD: `GET` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:47:13.700970 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c95a36a3-a758-4000-8583-c27dd790bf23 None admin] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:47:13.701240 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c95a36a3-a758-4000-8583-c27dd790bf23 None admin] PATH_INFO: `/v3/projects/9a98b1e4431940ab8d1fc95df2858a6c` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:47:13.702486 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c95a36a3-a758-4000-8583-c27dd790bf23 None admin] RBAC: Authorizing `identity:get_project(project_id=9a98b1e4431940ab8d1fc95df2858a6c)` {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 15:47:13.704010 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c95a36a3-a758-4000-8583-c27dd790bf23 None admin] RBAC: Authorization granted {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 15:47:13.705689 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 221/453] 10.222.0.15 () {56 vars in 1186 bytes} [Mon Aug 22 15:47:13 2022] GET /identity/v3/projects/9a98b1e4431940ab8d1fc95df2858a6c => generated 344 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 15:47:13.711529 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cad7a94d-617d-4a00-937c-fda7eba2a3c8 None admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 15:47:13.715229 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=100 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 15:47:13.818807 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cad7a94d-617d-4a00-937c-fda7eba2a3c8 None admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 15:47:13.853587 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6121f135-b13b-4491-8b08-370b38e4470f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 15:47:13.854225 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6121f135-b13b-4491-8b08-370b38e4470f None admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:47:13.854493 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6121f135-b13b-4491-8b08-370b38e4470f None admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:47:13.855503 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6121f135-b13b-4491-8b08-370b38e4470f None admin] PATH_INFO: `/v3/groups/f064c547078a42a0b477e6b3f08a751d` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:47:13.856569 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6121f135-b13b-4491-8b08-370b38e4470f None admin] RBAC: Authorizing `identity:get_group(group_id=f064c547078a42a0b477e6b3f08a751d)` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 15:47:13.858015 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6121f135-b13b-4491-8b08-370b38e4470f None admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 15:47:13.859972 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 233/454] 10.222.0.15 () {56 vars in 1180 bytes} [Mon Aug 22 15:47:13 2022] GET /identity/v3/groups/f064c547078a42a0b477e6b3f08a751d => generated 229 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 15:47:13.865626 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c95a36a3-a758-4000-8583-c27dd790bf23 None admin] Authenticating user token {{(pid=81767) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 15:47:13.941420 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG dbcounter [-] [81767] Writing DB stats keystone:SELECT=100 {{(pid=81767) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 15:47:13.968464 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c95a36a3-a758-4000-8583-c27dd790bf23 None admin] Validating token access rules against request {{(pid=81767) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 15:47:14.002068 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3008a075-2489-4637-b580-b7d6ea3e1e1e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81767) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 15:47:14.002661 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3008a075-2489-4637-b580-b7d6ea3e1e1e None admin] REQUEST_METHOD: `GET` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:47:14.002939 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3008a075-2489-4637-b580-b7d6ea3e1e1e None admin] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:47:14.003212 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3008a075-2489-4637-b580-b7d6ea3e1e1e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:47:14.003958 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3008a075-2489-4637-b580-b7d6ea3e1e1e None admin] RBAC: Adding query filter params (group.id=f064c547078a42a0b477e6b3f08a751d, role.id=9d9848cfd22e4b089db443c350e56670, scope.project.id=9a98b1e4431940ab8d1fc95df2858a6c) {{(pid=81767) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 15:47:14.004299 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3008a075-2489-4637-b580-b7d6ea3e1e1e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 15:47:14.005729 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3008a075-2489-4637-b580-b7d6ea3e1e1e None admin] RBAC: Authorization granted {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 15:47:14.011846 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 222/455] 10.222.0.15 () {56 vars in 1376 bytes} [Mon Aug 22 15:47:13 2022] GET /identity/v3/role_assignments?group.id=f064c547078a42a0b477e6b3f08a751d&scope.project.id=9a98b1e4431940ab8d1fc95df2858a6c&role.id=9d9848cfd22e4b089db443c350e56670 => generated 611 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 15:47:16.275315 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fe357bc-876a-4a87-9c3b-9c03bd662c8c None None] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:47:16.275905 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fe357bc-876a-4a87-9c3b-9c03bd662c8c None None] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:47:16.275905 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fe357bc-876a-4a87-9c3b-9c03bd662c8c None None] PATH_INFO: `/` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:47:16.277596 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 234/456] 10.222.0.15 () {54 vars in 927 bytes} [Mon Aug 22 15:47:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 15:47:16.283580 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3661c6fb-baa1-4ab5-b9d7-395c7a89b229 None None] REQUEST_METHOD: `POST` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:47:16.284033 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3661c6fb-baa1-4ab5-b9d7-395c7a89b229 None None] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:47:16.284143 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3661c6fb-baa1-4ab5-b9d7-395c7a89b229 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:47:16.307770 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.auth.core [None req-3661c6fb-baa1-4ab5-b9d7-395c7a89b229 None None] MFA Rules not processed for user `6b2199ae0fb0480aa917ada391f5ec54`. Rule list: `[]` (Enabled: `True`). {{(pid=81767) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 15:47:16.309052 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.fernet_utils [None req-3661c6fb-baa1-4ab5-b9d7-395c7a89b229 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81767) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 15:47:16.400946 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 223/457] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Aug 22 15:47:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 22 15:47:16.487126 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e8a49f5-bfce-4e3a-ae8f-056ccbfbd5b2 None None] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:47:16.487787 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e8a49f5-bfce-4e3a-ae8f-056ccbfbd5b2 None None] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:47:16.488320 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e8a49f5-bfce-4e3a-ae8f-056ccbfbd5b2 None None] PATH_INFO: `/` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:47:16.490203 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 235/458] 10.222.0.15 () {54 vars in 927 bytes} [Mon Aug 22 15:47:16 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 15:47:16.501045 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c29eea4-0f81-4cea-88bc-39b061a0850b None None] REQUEST_METHOD: `POST` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:47:16.501326 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c29eea4-0f81-4cea-88bc-39b061a0850b None None] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:47:16.501604 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c29eea4-0f81-4cea-88bc-39b061a0850b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:47:16.524670 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.auth.core [None req-5c29eea4-0f81-4cea-88bc-39b061a0850b None None] MFA Rules not processed for user `6b2199ae0fb0480aa917ada391f5ec54`. Rule list: `[]` (Enabled: `True`). {{(pid=81767) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 15:47:16.525979 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.fernet_utils [None req-5c29eea4-0f81-4cea-88bc-39b061a0850b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81767) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 15:47:16.612909 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG dbcounter [-] [81767] Writing DB stats keystone:SELECT=100 {{(pid=81767) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 15:47:16.618954 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b1db2bf-16dd-4a1b-8820-040a7135a919 None None] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:47:16.619580 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b1db2bf-16dd-4a1b-8820-040a7135a919 None None] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:47:16.620101 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b1db2bf-16dd-4a1b-8820-040a7135a919 None None] PATH_INFO: `/` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:47:16.621654 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 224/459] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Aug 22 15:47:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 22 15:47:16.622025 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 236/460] 10.222.0.15 () {54 vars in 927 bytes} [Mon Aug 22 15:47:16 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 15:47:16.627309 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b2429ea-b72e-4a39-be33-877a9cb969f6 None None] REQUEST_METHOD: `POST` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:47:16.640219 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b2429ea-b72e-4a39-be33-877a9cb969f6 None None] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:47:16.640219 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b2429ea-b72e-4a39-be33-877a9cb969f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:47:16.656398 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.auth.core [None req-7b2429ea-b72e-4a39-be33-877a9cb969f6 None None] MFA Rules not processed for user `6b2199ae0fb0480aa917ada391f5ec54`. Rule list: `[]` (Enabled: `True`). {{(pid=81767) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 15:47:16.657848 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.fernet_utils [None req-7b2429ea-b72e-4a39-be33-877a9cb969f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81767) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 15:47:16.749862 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b1db2bf-16dd-4a1b-8820-040a7135a919 None None] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 15:47:16.752687 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 225/461] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Aug 22 15:47:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 22 15:47:16.890979 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=100 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 15:47:16.936072 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b1db2bf-16dd-4a1b-8820-040a7135a919 None None] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 15:47:16.986723 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ded40b03-04d7-4ca5-8074-515bfa642fb6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 15:47:16.987392 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ded40b03-04d7-4ca5-8074-515bfa642fb6 None admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:47:16.987676 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ded40b03-04d7-4ca5-8074-515bfa642fb6 None admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:47:16.987947 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ded40b03-04d7-4ca5-8074-515bfa642fb6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:47:17.005239 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ded40b03-04d7-4ca5-8074-515bfa642fb6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 15:47:17.006796 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ded40b03-04d7-4ca5-8074-515bfa642fb6 None admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 15:47:17.027203 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application [None req-ded40b03-04d7-4ca5-8074-515bfa642fb6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 22 15:47:17.027203 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 15:47:17.027203 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 15:47:17.027203 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 15:47:17.027203 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 15:47:17.027203 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 15:47:17.027203 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 15:47:17.027203 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 15:47:17.027203 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 15:47:17.027203 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 15:47:17.027203 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 15:47:17.027203 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 15:47:17.027203 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 22 15:47:17.027203 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 22 15:47:17.027203 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 22 15:47:17.027203 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 22 15:47:17.028149 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 15:47:17.028149 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 15:47:17.028149 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 15:47:17.028149 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 15:47:17.028149 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 15:47:17.028149 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 15:47:17.028149 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 15:47:17.028149 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application with Lock( Aug 22 15:47:17.028149 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 15:47:17.028149 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self._enter() Aug 22 15:47:17.028149 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 15:47:17.028149 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 15:47:17.028149 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 15:47:17.028149 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self.creator() Aug 22 15:47:17.028149 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 15:47:17.028149 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application created_value = creator( Aug 22 15:47:17.028149 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 22 15:47:17.028962 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 22 15:47:17.028962 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 22 15:47:17.028962 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application Aug 22 15:47:17.028962 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 237/462] 10.222.0.15 () {56 vars in 1108 bytes} [Mon Aug 22 15:47:16 2022] GET /identity/v3/domains/Default => generated 87 bytes in 279 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 22 15:47:17.034356 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b2429ea-b72e-4a39-be33-877a9cb969f6 None None] Authenticating user token {{(pid=81767) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 15:47:17.079720 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ded40b03-04d7-4ca5-8074-515bfa642fb6 None admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 15:47:17.156642 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b2429ea-b72e-4a39-be33-877a9cb969f6 None None] Validating token access rules against request {{(pid=81767) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 15:47:17.186777 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG dbcounter [-] [81767] Writing DB stats keystone:SELECT=100 {{(pid=81767) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 15:47:17.195012 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ded40b03-04d7-4ca5-8074-515bfa642fb6 None admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 15:47:17.195882 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-977f4e26-d051-41c4-9be0-280f2b9cf226 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81767) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 15:47:17.196831 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-977f4e26-d051-41c4-9be0-280f2b9cf226 None admin] REQUEST_METHOD: `GET` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:47:17.197359 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-977f4e26-d051-41c4-9be0-280f2b9cf226 None admin] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:47:17.204610 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-977f4e26-d051-41c4-9be0-280f2b9cf226 None admin] PATH_INFO: `/v3/domains` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:47:17.204610 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-977f4e26-d051-41c4-9be0-280f2b9cf226 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81767) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 15:47:17.204610 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-977f4e26-d051-41c4-9be0-280f2b9cf226 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 15:47:17.204610 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-977f4e26-d051-41c4-9be0-280f2b9cf226 None admin] RBAC: Authorization granted {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 15:47:17.219101 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 226/463] 10.222.0.15 () {56 vars in 1109 bytes} [Mon Aug 22 15:47:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 15:47:17.220486 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-977f4e26-d051-41c4-9be0-280f2b9cf226 None admin] Authenticating user token {{(pid=81767) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 15:47:17.221867 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=100 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 15:47:17.233923 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c3488f7-0bfa-4271-9bd3-091962de9566 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 15:47:17.234522 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c3488f7-0bfa-4271-9bd3-091962de9566 None admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:47:17.234806 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c3488f7-0bfa-4271-9bd3-091962de9566 None admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:47:17.235076 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c3488f7-0bfa-4271-9bd3-091962de9566 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:47:17.246479 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c3488f7-0bfa-4271-9bd3-091962de9566 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 15:47:17.247995 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c3488f7-0bfa-4271-9bd3-091962de9566 None admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 15:47:17.264813 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application [None req-6c3488f7-0bfa-4271-9bd3-091962de9566 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 22 15:47:17.264813 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 15:47:17.264813 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 15:47:17.264813 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 15:47:17.264813 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 15:47:17.264813 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 15:47:17.264813 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 15:47:17.264813 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 15:47:17.264813 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 15:47:17.264813 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 15:47:17.264813 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 15:47:17.264813 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 15:47:17.264813 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 22 15:47:17.264813 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 22 15:47:17.264813 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 22 15:47:17.264813 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 22 15:47:17.272373 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 15:47:17.272373 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 15:47:17.272373 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 15:47:17.272373 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 15:47:17.272373 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 15:47:17.272373 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 15:47:17.272373 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 15:47:17.272373 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application with Lock( Aug 22 15:47:17.272373 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 15:47:17.272373 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self._enter() Aug 22 15:47:17.272373 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 15:47:17.272373 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 15:47:17.272373 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 15:47:17.272373 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self.creator() Aug 22 15:47:17.272373 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 15:47:17.272373 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application created_value = creator( Aug 22 15:47:17.272373 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 22 15:47:17.273224 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 22 15:47:17.273224 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 22 15:47:17.273224 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application Aug 22 15:47:17.273224 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 238/464] 10.222.0.15 () {56 vars in 1108 bytes} [Mon Aug 22 15:47:17 2022] GET /identity/v3/domains/Default => generated 87 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 22 15:47:17.273224 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c3488f7-0bfa-4271-9bd3-091962de9566 None admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 15:47:17.334429 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-977f4e26-d051-41c4-9be0-280f2b9cf226 None admin] Validating token access rules against request {{(pid=81767) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 15:47:17.376939 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8024ba5-a9d1-48b7-8254-8c5afb747a0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81767) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 15:47:17.377672 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8024ba5-a9d1-48b7-8254-8c5afb747a0a None admin] REQUEST_METHOD: `GET` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:47:17.378015 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8024ba5-a9d1-48b7-8254-8c5afb747a0a None admin] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:47:17.378193 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c3488f7-0bfa-4271-9bd3-091962de9566 None admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 15:47:17.378614 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8024ba5-a9d1-48b7-8254-8c5afb747a0a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:47:17.394107 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8024ba5-a9d1-48b7-8254-8c5afb747a0a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 15:47:17.396194 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8024ba5-a9d1-48b7-8254-8c5afb747a0a None admin] RBAC: Authorization granted {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 15:47:17.414250 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application [None req-b8024ba5-a9d1-48b7-8254-8c5afb747a0a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 22 15:47:17.414250 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 15:47:17.414250 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 15:47:17.414250 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 15:47:17.414250 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 15:47:17.414250 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 15:47:17.414250 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 15:47:17.414250 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 15:47:17.414250 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 15:47:17.414250 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 15:47:17.414250 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 15:47:17.414250 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 15:47:17.414250 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 22 15:47:17.414250 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 22 15:47:17.414250 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 22 15:47:17.414250 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 22 15:47:17.415347 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 15:47:17.415347 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 15:47:17.415347 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 15:47:17.415347 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 15:47:17.415347 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 15:47:17.415347 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 15:47:17.415347 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 15:47:17.415347 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application with Lock( Aug 22 15:47:17.415347 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 15:47:17.415347 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return self._enter() Aug 22 15:47:17.415347 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 15:47:17.415347 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 15:47:17.415347 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 15:47:17.415347 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return self.creator() Aug 22 15:47:17.415347 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 15:47:17.415347 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application created_value = creator( Aug 22 15:47:17.415347 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 22 15:47:17.416348 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 22 15:47:17.416348 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 22 15:47:17.416348 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application Aug 22 15:47:17.416551 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2595bd96-e45e-47e3-a5f4-d278f8414667 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 15:47:17.416551 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2595bd96-e45e-47e3-a5f4-d278f8414667 None admin] REQUEST_METHOD: `POST` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:47:17.416728 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 227/465] 10.222.0.15 () {56 vars in 1108 bytes} [Mon Aug 22 15:47:17 2022] GET /identity/v3/domains/Default => generated 87 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 15:47:17.416806 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2595bd96-e45e-47e3-a5f4-d278f8414667 None admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:47:17.416806 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2595bd96-e45e-47e3-a5f4-d278f8414667 None admin] PATH_INFO: `/v3/users` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:47:17.416934 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8024ba5-a9d1-48b7-8254-8c5afb747a0a None admin] Authenticating user token {{(pid=81767) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 15:47:17.417012 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2595bd96-e45e-47e3-a5f4-d278f8414667 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 15:47:17.422326 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2595bd96-e45e-47e3-a5f4-d278f8414667 None admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 15:47:17.465212 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG dbcounter [-] [81767] Writing DB stats keystone:SELECT=100 {{(pid=81767) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 15:47:17.522015 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8024ba5-a9d1-48b7-8254-8c5afb747a0a None admin] Validating token access rules against request {{(pid=81767) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 15:47:17.556119 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac820734-b375-4b38-ade3-668e8fdc7d54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81767) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 15:47:17.556958 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac820734-b375-4b38-ade3-668e8fdc7d54 None admin] REQUEST_METHOD: `GET` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:47:17.557236 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac820734-b375-4b38-ade3-668e8fdc7d54 None admin] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:47:17.557505 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac820734-b375-4b38-ade3-668e8fdc7d54 None admin] PATH_INFO: `/v3/domains` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:47:17.558195 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac820734-b375-4b38-ade3-668e8fdc7d54 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81767) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 15:47:17.558496 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac820734-b375-4b38-ade3-668e8fdc7d54 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 15:47:17.559883 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac820734-b375-4b38-ade3-668e8fdc7d54 None admin] RBAC: Authorization granted {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 15:47:17.571299 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 228/466] 10.222.0.15 () {56 vars in 1109 bytes} [Mon Aug 22 15:47:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 15:47:17.572793 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74051487-801f-4b87-ba3b-bc290ed5a873 None None] REQUEST_METHOD: `GET` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:47:17.572793 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74051487-801f-4b87-ba3b-bc290ed5a873 None None] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:47:17.573256 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74051487-801f-4b87-ba3b-bc290ed5a873 None None] PATH_INFO: `/` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:47:17.574613 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 229/467] 10.222.0.15 () {54 vars in 927 bytes} [Mon Aug 22 15:47:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 15:47:17.575308 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74051487-801f-4b87-ba3b-bc290ed5a873 None None] Authenticating user token {{(pid=81767) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 15:47:17.647735 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 239/468] 10.222.0.15 () {60 vars in 1132 bytes} [Mon Aug 22 15:47:17 2022] POST /identity/v3/users => generated 246 bytes in 381 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 22 15:47:17.649198 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d6ad4d3-85e6-43f6-9717-dc8c2be1a812 None None] REQUEST_METHOD: `POST` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:47:17.649540 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d6ad4d3-85e6-43f6-9717-dc8c2be1a812 None None] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:47:17.649820 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d6ad4d3-85e6-43f6-9717-dc8c2be1a812 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:47:17.684682 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74051487-801f-4b87-ba3b-bc290ed5a873 None None] Validating token access rules against request {{(pid=81767) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 15:47:17.694556 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG dbcounter [-] [81767] Writing DB stats keystone:SELECT=100 {{(pid=81767) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 15:47:17.695958 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.auth.core [None req-1d6ad4d3-85e6-43f6-9717-dc8c2be1a812 None None] MFA Rules not processed for user `6b2199ae0fb0480aa917ada391f5ec54`. Rule list: `[]` (Enabled: `True`). {{(pid=81768) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 15:47:17.697263 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-1d6ad4d3-85e6-43f6-9717-dc8c2be1a812 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 15:47:17.739729 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 15:47:17.740692 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27fa69a6-548c-4962-ab4f-1f432eadc755 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81767) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 15:47:17.741390 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27fa69a6-548c-4962-ab4f-1f432eadc755 None admin] REQUEST_METHOD: `GET` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:47:17.741670 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27fa69a6-548c-4962-ab4f-1f432eadc755 None admin] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:47:17.742387 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27fa69a6-548c-4962-ab4f-1f432eadc755 None admin] PATH_INFO: `/v3/domains` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:47:17.742621 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27fa69a6-548c-4962-ab4f-1f432eadc755 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81767) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 15:47:17.742925 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27fa69a6-548c-4962-ab4f-1f432eadc755 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 15:47:17.744443 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27fa69a6-548c-4962-ab4f-1f432eadc755 None admin] RBAC: Authorization granted {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 15:47:17.759856 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 230/469] 10.222.0.15 () {56 vars in 1109 bytes} [Mon Aug 22 15:47:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 15:47:17.760772 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27fa69a6-548c-4962-ab4f-1f432eadc755 None admin] Authenticating user token {{(pid=81767) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 15:47:17.801893 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 240/470] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Aug 22 15:47:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 22 15:47:17.803073 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d6ad4d3-85e6-43f6-9717-dc8c2be1a812 None None] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 15:47:17.895648 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27fa69a6-548c-4962-ab4f-1f432eadc755 None admin] Validating token access rules against request {{(pid=81767) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 15:47:17.938769 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d6ad4d3-85e6-43f6-9717-dc8c2be1a812 None None] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 15:47:17.945836 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e5afc91-ee99-48c9-afdc-95225f0729c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81767) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 15:47:17.946463 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e5afc91-ee99-48c9-afdc-95225f0729c9 None admin] REQUEST_METHOD: `POST` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:47:17.947193 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e5afc91-ee99-48c9-afdc-95225f0729c9 None admin] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:47:17.948071 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e5afc91-ee99-48c9-afdc-95225f0729c9 None admin] PATH_INFO: `/v3/users` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:47:17.949259 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e5afc91-ee99-48c9-afdc-95225f0729c9 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 15:47:17.951235 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e5afc91-ee99-48c9-afdc-95225f0729c9 None admin] RBAC: Authorization granted {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 15:47:17.982708 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ecf38ab-c286-4c05-aa11-96f0afb0d45c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 15:47:17.983371 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ecf38ab-c286-4c05-aa11-96f0afb0d45c None admin] REQUEST_METHOD: `POST` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:47:17.983651 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ecf38ab-c286-4c05-aa11-96f0afb0d45c None admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:47:17.983918 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ecf38ab-c286-4c05-aa11-96f0afb0d45c None admin] PATH_INFO: `/v3/users` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:47:17.984615 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ecf38ab-c286-4c05-aa11-96f0afb0d45c None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 15:47:17.986191 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ecf38ab-c286-4c05-aa11-96f0afb0d45c None admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 15:47:18.227463 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 241/471] 10.222.0.15 () {60 vars in 1132 bytes} [Mon Aug 22 15:47:17 2022] POST /identity/v3/users => generated 248 bytes in 425 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 22 15:47:18.228618 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1566cbb-865a-4046-870b-402542481dbd None None] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:47:18.228897 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1566cbb-865a-4046-870b-402542481dbd None None] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:47:18.229247 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1566cbb-865a-4046-870b-402542481dbd None None] PATH_INFO: `/` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:47:18.230512 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 242/472] 10.222.0.15 () {54 vars in 927 bytes} [Mon Aug 22 15:47:18 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 15:47:18.231086 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1566cbb-865a-4046-870b-402542481dbd None None] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 15:47:18.254995 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 15:47:18.293922 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 231/473] 10.222.0.15 () {60 vars in 1132 bytes} [Mon Aug 22 15:47:17 2022] POST /identity/v3/users => generated 248 bytes in 533 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 22 15:47:18.295174 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-402c9fc1-e150-45b9-86f9-e6c5cf65f297 None None] REQUEST_METHOD: `POST` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:47:18.295546 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-402c9fc1-e150-45b9-86f9-e6c5cf65f297 None None] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:47:18.295819 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-402c9fc1-e150-45b9-86f9-e6c5cf65f297 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:47:18.333193 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.auth.core [None req-402c9fc1-e150-45b9-86f9-e6c5cf65f297 None None] MFA Rules not processed for user `6b2199ae0fb0480aa917ada391f5ec54`. Rule list: `[]` (Enabled: `True`). {{(pid=81767) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 15:47:18.334724 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.fernet_utils [None req-402c9fc1-e150-45b9-86f9-e6c5cf65f297 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81767) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 15:47:18.349912 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG dbcounter [-] [81767] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81767) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 15:47:18.400707 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1566cbb-865a-4046-870b-402542481dbd None None] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 15:47:18.455624 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 232/474] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Aug 22 15:47:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 22 15:47:18.457887 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d39a0d9-27e4-4e1b-97af-f005d6675181 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 15:47:18.458725 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d39a0d9-27e4-4e1b-97af-f005d6675181 None admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:47:18.459155 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d39a0d9-27e4-4e1b-97af-f005d6675181 None admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:47:18.459676 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d39a0d9-27e4-4e1b-97af-f005d6675181 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:47:18.472911 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d39a0d9-27e4-4e1b-97af-f005d6675181 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 15:47:18.474649 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d39a0d9-27e4-4e1b-97af-f005d6675181 None admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 15:47:18.491105 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application [None req-1d39a0d9-27e4-4e1b-97af-f005d6675181 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 22 15:47:18.491105 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 15:47:18.491105 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 15:47:18.491105 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 15:47:18.491105 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 15:47:18.491105 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 15:47:18.491105 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 15:47:18.491105 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 15:47:18.491105 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 15:47:18.491105 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 15:47:18.491105 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 15:47:18.491105 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 15:47:18.491105 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 22 15:47:18.491105 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 22 15:47:18.491105 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 22 15:47:18.491105 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 22 15:47:18.492105 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 15:47:18.492105 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 15:47:18.492105 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 15:47:18.492105 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 15:47:18.492105 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 15:47:18.492105 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 15:47:18.492105 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 15:47:18.492105 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application with Lock( Aug 22 15:47:18.492105 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 15:47:18.492105 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self._enter() Aug 22 15:47:18.492105 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 15:47:18.492105 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 15:47:18.492105 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 15:47:18.492105 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self.creator() Aug 22 15:47:18.492105 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 15:47:18.492105 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application created_value = creator( Aug 22 15:47:18.492105 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 22 15:47:18.492977 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 22 15:47:18.492977 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 22 15:47:18.492977 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application Aug 22 15:47:18.492977 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 243/475] 10.222.0.15 () {56 vars in 1108 bytes} [Mon Aug 22 15:47:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 261 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 15:47:18.500878 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-402c9fc1-e150-45b9-86f9-e6c5cf65f297 None None] Authenticating user token {{(pid=81767) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 15:47:18.622701 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-402c9fc1-e150-45b9-86f9-e6c5cf65f297 None None] Validating token access rules against request {{(pid=81767) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 15:47:18.661176 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7ce2cd2-0035-4b25-a87d-9923dbc2952d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81767) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 15:47:18.661851 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7ce2cd2-0035-4b25-a87d-9923dbc2952d None admin] REQUEST_METHOD: `GET` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:47:18.662126 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7ce2cd2-0035-4b25-a87d-9923dbc2952d None admin] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:47:18.662399 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7ce2cd2-0035-4b25-a87d-9923dbc2952d None admin] PATH_INFO: `/v3/domains` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:47:18.663047 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7ce2cd2-0035-4b25-a87d-9923dbc2952d None admin] RBAC: Adding query filter params (name=Default) {{(pid=81767) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 15:47:18.663374 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7ce2cd2-0035-4b25-a87d-9923dbc2952d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 15:47:18.664744 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7ce2cd2-0035-4b25-a87d-9923dbc2952d None admin] RBAC: Authorization granted {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 15:47:18.677429 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 233/476] 10.222.0.15 () {56 vars in 1109 bytes} [Mon Aug 22 15:47:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 15:47:18.682946 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d39a0d9-27e4-4e1b-97af-f005d6675181 None admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 15:47:18.773915 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7ce2cd2-0035-4b25-a87d-9923dbc2952d None admin] Authenticating user token {{(pid=81767) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 15:47:18.778099 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=100 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 15:47:18.791156 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG dbcounter [-] [81767] Writing DB stats keystone:SELECT=100 {{(pid=81767) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 15:47:18.810458 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d39a0d9-27e4-4e1b-97af-f005d6675181 None admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 15:47:18.846661 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5ec2b6b-a80a-419d-9527-4eedc2fa4500 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 15:47:18.847291 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5ec2b6b-a80a-419d-9527-4eedc2fa4500 None admin] REQUEST_METHOD: `POST` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:47:18.847570 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5ec2b6b-a80a-419d-9527-4eedc2fa4500 None admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:47:18.847842 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5ec2b6b-a80a-419d-9527-4eedc2fa4500 None admin] PATH_INFO: `/v3/users` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:47:18.848500 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5ec2b6b-a80a-419d-9527-4eedc2fa4500 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 15:47:18.850019 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5ec2b6b-a80a-419d-9527-4eedc2fa4500 None admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 15:47:18.925165 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7ce2cd2-0035-4b25-a87d-9923dbc2952d None admin] Validating token access rules against request {{(pid=81767) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 15:47:18.961002 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cbe8daf-384a-41c3-9466-293c0e6dd57e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81767) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 15:47:18.961608 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cbe8daf-384a-41c3-9466-293c0e6dd57e None admin] REQUEST_METHOD: `POST` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:47:18.961887 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cbe8daf-384a-41c3-9466-293c0e6dd57e None admin] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:47:18.962146 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cbe8daf-384a-41c3-9466-293c0e6dd57e None admin] PATH_INFO: `/v3/roles` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:47:18.962819 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cbe8daf-384a-41c3-9466-293c0e6dd57e None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 15:47:18.964269 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cbe8daf-384a-41c3-9466-293c0e6dd57e None admin] RBAC: Authorization granted {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 15:47:19.006135 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.sql.core [None req-5cbe8daf-384a-41c3-9466-293c0e6dd57e None admin] Conflict role: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Aug 22 15:47:19.006135 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Aug 22 15:47:19.006135 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [parameters: {'id': 'e8e559cea51a4e7e813cbc575bdb81e9', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Aug 22 15:47:19.006135 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=81767) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Aug 22 15:47:19.023684 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application [None req-5cbe8daf-384a-41c3-9466-293c0e6dd57e None admin] Conflict occurred attempting to store role - Duplicate entry found with name anotherrole.: keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Aug 22 15:47:19.023684 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 15:47:19.023684 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Aug 22 15:47:19.023684 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 22 15:47:19.023684 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 22 15:47:19.023684 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 22 15:47:19.023684 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 22 15:47:19.023684 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application result = self._query(query) Aug 22 15:47:19.023684 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 22 15:47:19.023684 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application conn.query(q) Aug 22 15:47:19.023684 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 22 15:47:19.023684 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 22 15:47:19.023684 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 22 15:47:19.023684 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application result.read() Aug 22 15:47:19.023684 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 22 15:47:19.024692 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 22 15:47:19.024692 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 22 15:47:19.024692 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 22 15:47:19.024692 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 22 15:47:19.024692 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 22 15:47:19.024692 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 22 15:47:19.024692 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 22 15:47:19.024692 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Aug 22 15:47:19.024692 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application Aug 22 15:47:19.024692 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Aug 22 15:47:19.024692 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application Aug 22 15:47:19.024692 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 15:47:19.024692 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Aug 22 15:47:19.024692 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return method(*args, **kwargs) Aug 22 15:47:19.024692 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 33, in create_role Aug 22 15:47:19.024692 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return ref.to_dict() Aug 22 15:47:19.024692 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 22 15:47:19.024692 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application next(self.gen) Aug 22 15:47:19.025844 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Aug 22 15:47:19.025844 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application yield resource Aug 22 15:47:19.025844 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 22 15:47:19.025844 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application next(self.gen) Aug 22 15:47:19.025844 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Aug 22 15:47:19.025844 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application self.session.rollback() Aug 22 15:47:19.025844 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Aug 22 15:47:19.025844 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application self.force_reraise() Aug 22 15:47:19.025844 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Aug 22 15:47:19.025844 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application raise self.value Aug 22 15:47:19.025844 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Aug 22 15:47:19.025844 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Aug 22 15:47:19.025844 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Aug 22 15:47:19.025844 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application session.commit() Aug 22 15:47:19.025844 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Aug 22 15:47:19.025844 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Aug 22 15:47:19.025844 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Aug 22 15:47:19.026910 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application self._prepare_impl() Aug 22 15:47:19.026910 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Aug 22 15:47:19.026910 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application self.session.flush() Aug 22 15:47:19.026910 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Aug 22 15:47:19.026910 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application self._flush(objects) Aug 22 15:47:19.026910 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Aug 22 15:47:19.026910 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Aug 22 15:47:19.026910 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Aug 22 15:47:19.026910 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application compat.raise_( Aug 22 15:47:19.026910 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 22 15:47:19.026910 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application raise exception Aug 22 15:47:19.026910 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Aug 22 15:47:19.026910 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application flush_context.execute() Aug 22 15:47:19.026910 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Aug 22 15:47:19.026910 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application rec.execute(self) Aug 22 15:47:19.026910 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Aug 22 15:47:19.026910 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Aug 22 15:47:19.027903 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Aug 22 15:47:19.027903 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application _emit_insert_statements( Aug 22 15:47:19.027903 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Aug 22 15:47:19.027903 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application c = connection._execute_20( Aug 22 15:47:19.027903 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Aug 22 15:47:19.027903 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Aug 22 15:47:19.027903 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Aug 22 15:47:19.027903 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Aug 22 15:47:19.027903 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Aug 22 15:47:19.027903 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application ret = self._execute_context( Aug 22 15:47:19.027903 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Aug 22 15:47:19.027903 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Aug 22 15:47:19.027903 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Aug 22 15:47:19.027903 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Aug 22 15:47:19.027903 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 22 15:47:19.027903 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application raise exception Aug 22 15:47:19.028725 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Aug 22 15:47:19.028725 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 22 15:47:19.028725 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 22 15:47:19.028725 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 22 15:47:19.028725 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 22 15:47:19.028725 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application result = self._query(query) Aug 22 15:47:19.028725 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 22 15:47:19.028725 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application conn.query(q) Aug 22 15:47:19.028725 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 22 15:47:19.028725 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 22 15:47:19.028725 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 22 15:47:19.028725 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application result.read() Aug 22 15:47:19.028725 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 22 15:47:19.028725 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 22 15:47:19.028725 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 22 15:47:19.028725 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 22 15:47:19.029590 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 22 15:47:19.029590 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 22 15:47:19.029590 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 22 15:47:19.029590 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 22 15:47:19.029590 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Aug 22 15:47:19.029590 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Aug 22 15:47:19.029590 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application [parameters: {'id': 'e8e559cea51a4e7e813cbc575bdb81e9', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Aug 22 15:47:19.029590 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Aug 22 15:47:19.029590 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application Aug 22 15:47:19.029590 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 22 15:47:19.029590 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application Aug 22 15:47:19.029590 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 15:47:19.029590 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 15:47:19.029590 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 15:47:19.029590 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 15:47:19.029590 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 15:47:19.032254 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 15:47:19.032254 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 15:47:19.032254 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 15:47:19.032254 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 15:47:19.032254 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 15:47:19.032254 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 15:47:19.032254 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 115, in post Aug 22 15:47:19.032254 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application ref = PROVIDERS.role_api.create_role( Aug 22 15:47:19.032254 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 15:47:19.032254 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 15:47:19.032254 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1298, in create_role Aug 22 15:47:19.032254 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application ret = self.driver.create_role(role_id, role) Aug 22 15:47:19.032254 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Aug 22 15:47:19.032254 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Aug 22 15:47:19.032254 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Aug 22 15:47:19.032254 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application Aug 22 15:47:19.034310 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 234/477] 10.222.0.15 () {60 vars in 1131 bytes} [Mon Aug 22 15:47:18 2022] POST /identity/v3/roles => generated 144 bytes in 252 msecs (HTTP/1.1 409) 5 headers in 183 bytes (2 switches on core 0) Aug 22 15:47:19.034310 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cbe8daf-384a-41c3-9466-293c0e6dd57e None admin] Authenticating user token {{(pid=81767) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 15:47:19.110194 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG dbcounter [-] [81767] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81767) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 15:47:19.143955 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cbe8daf-384a-41c3-9466-293c0e6dd57e None admin] Validating token access rules against request {{(pid=81767) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 15:47:19.178773 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e447386-849b-4785-b93c-bfc2aa3b7c1e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81767) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 15:47:19.179433 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e447386-849b-4785-b93c-bfc2aa3b7c1e None admin] REQUEST_METHOD: `GET` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:47:19.179709 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e447386-849b-4785-b93c-bfc2aa3b7c1e None admin] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:47:19.179972 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e447386-849b-4785-b93c-bfc2aa3b7c1e None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:47:19.192453 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e447386-849b-4785-b93c-bfc2aa3b7c1e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 22 15:47:19.197761 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application [None req-0e447386-849b-4785-b93c-bfc2aa3b7c1e None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 22 15:47:19.197761 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 15:47:19.197761 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 15:47:19.197761 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 15:47:19.197761 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 15:47:19.197761 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 15:47:19.197761 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 15:47:19.197761 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 15:47:19.197761 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 15:47:19.197761 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 15:47:19.197761 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 15:47:19.197761 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 15:47:19.197761 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 22 15:47:19.197761 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 22 15:47:19.197761 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 22 15:47:19.197761 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 22 15:47:19.200595 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 22 15:47:19.200595 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 22 15:47:19.200595 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 22 15:47:19.200595 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 22 15:47:19.200595 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 15:47:19.200595 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 15:47:19.200595 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 15:47:19.200595 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 15:47:19.200595 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 15:47:19.200595 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 15:47:19.200595 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 15:47:19.200595 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application with Lock( Aug 22 15:47:19.200595 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 15:47:19.200595 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return self._enter() Aug 22 15:47:19.200595 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 15:47:19.200595 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 15:47:19.202555 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 15:47:19.202555 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return self.creator() Aug 22 15:47:19.202555 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 15:47:19.202555 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application created_value = creator( Aug 22 15:47:19.202555 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 22 15:47:19.202555 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 22 15:47:19.202555 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 22 15:47:19.202555 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 22 15:47:19.202555 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 22 15:47:19.202555 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 22 15:47:19.202555 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 22 15:47:19.202555 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application Aug 22 15:47:19.202555 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 235/478] 10.222.0.15 () {56 vars in 1114 bytes} [Mon Aug 22 15:47:19 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 22 15:47:19.208513 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 244/479] 10.222.0.15 () {60 vars in 1132 bytes} [Mon Aug 22 15:47:18 2022] POST /identity/v3/users => generated 249 bytes in 523 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 22 15:47:19.208719 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e447386-849b-4785-b93c-bfc2aa3b7c1e None admin] Authenticating user token {{(pid=81767) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 15:47:19.350601 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e447386-849b-4785-b93c-bfc2aa3b7c1e None admin] Validating token access rules against request {{(pid=81767) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 15:47:19.388106 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG dbcounter [-] [81767] Writing DB stats keystone:SELECT=100 {{(pid=81767) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 15:47:19.395580 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8eaefc7-d086-4c05-977a-f241d1e48ab0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81767) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 15:47:19.396288 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8eaefc7-d086-4c05-977a-f241d1e48ab0 None admin] REQUEST_METHOD: `GET` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:47:19.396553 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8eaefc7-d086-4c05-977a-f241d1e48ab0 None admin] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:47:19.396828 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8eaefc7-d086-4c05-977a-f241d1e48ab0 None admin] PATH_INFO: `/v3/roles` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:47:19.397513 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8eaefc7-d086-4c05-977a-f241d1e48ab0 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81767) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 15:47:19.397818 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8eaefc7-d086-4c05-977a-f241d1e48ab0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 15:47:19.399319 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8eaefc7-d086-4c05-977a-f241d1e48ab0 None admin] RBAC: Authorization granted {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 15:47:19.410102 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 236/480] 10.222.0.15 () {56 vars in 1111 bytes} [Mon Aug 22 15:47:19 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 15:47:20.101776 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7837ac12-2b12-4bef-ba0a-7a1b9233cb3d None None] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:47:20.102285 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7837ac12-2b12-4bef-ba0a-7a1b9233cb3d None None] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:47:20.102358 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7837ac12-2b12-4bef-ba0a-7a1b9233cb3d None None] PATH_INFO: `/` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:47:20.104097 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 245/481] 10.222.0.15 () {54 vars in 927 bytes} [Mon Aug 22 15:47:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 15:47:20.110023 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31f9b672-c87e-4934-bc99-f76baba9a057 None None] REQUEST_METHOD: `POST` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:47:20.110309 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31f9b672-c87e-4934-bc99-f76baba9a057 None None] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:47:20.110581 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31f9b672-c87e-4934-bc99-f76baba9a057 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:47:20.136313 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.auth.core [None req-31f9b672-c87e-4934-bc99-f76baba9a057 None None] MFA Rules not processed for user `6b2199ae0fb0480aa917ada391f5ec54`. Rule list: `[]` (Enabled: `True`). {{(pid=81767) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 15:47:20.137512 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.fernet_utils [None req-31f9b672-c87e-4934-bc99-f76baba9a057 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81767) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 15:47:20.211623 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 237/482] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Aug 22 15:47:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 22 15:47:20.581738 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7837ac12-2b12-4bef-ba0a-7a1b9233cb3d None None] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 15:47:20.672255 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e76b4188-ff05-4b6c-aeb3-c5ce1a57abfd None None] REQUEST_METHOD: `GET` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:47:20.672786 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e76b4188-ff05-4b6c-aeb3-c5ce1a57abfd None None] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:47:20.672786 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e76b4188-ff05-4b6c-aeb3-c5ce1a57abfd None None] PATH_INFO: `/` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:47:20.674423 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 238/483] 10.222.0.15 () {54 vars in 927 bytes} [Mon Aug 22 15:47:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 15:47:20.679853 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e30d74bd-14a2-4c36-be26-c63a9f815ed1 None None] REQUEST_METHOD: `POST` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:47:20.680196 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e30d74bd-14a2-4c36-be26-c63a9f815ed1 None None] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:47:20.680472 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e30d74bd-14a2-4c36-be26-c63a9f815ed1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:47:20.709821 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.auth.core [None req-e30d74bd-14a2-4c36-be26-c63a9f815ed1 None None] MFA Rules not processed for user `6b2199ae0fb0480aa917ada391f5ec54`. Rule list: `[]` (Enabled: `True`). {{(pid=81767) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 15:47:20.711015 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.fernet_utils [None req-e30d74bd-14a2-4c36-be26-c63a9f815ed1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81767) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 15:47:20.717508 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7837ac12-2b12-4bef-ba0a-7a1b9233cb3d None None] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 15:47:20.765744 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 15:47:20.785963 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3640087-314c-46ab-8c75-1d5f4f66b983 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 15:47:20.786595 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3640087-314c-46ab-8c75-1d5f4f66b983 None admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:47:20.786870 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3640087-314c-46ab-8c75-1d5f4f66b983 None admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:47:20.787137 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3640087-314c-46ab-8c75-1d5f4f66b983 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:47:20.814669 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3640087-314c-46ab-8c75-1d5f4f66b983 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 22 15:47:20.821249 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 239/484] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Aug 22 15:47:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 22 15:47:20.821585 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application [None req-d3640087-314c-46ab-8c75-1d5f4f66b983 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 22 15:47:20.821585 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 15:47:20.821585 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 15:47:20.821585 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 15:47:20.821585 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 15:47:20.821585 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 15:47:20.821585 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 15:47:20.821585 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 15:47:20.821585 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 15:47:20.821585 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 15:47:20.821585 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 15:47:20.821585 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 15:47:20.821585 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 22 15:47:20.821585 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 22 15:47:20.821585 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 22 15:47:20.821585 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 22 15:47:20.822409 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 22 15:47:20.822409 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 22 15:47:20.822409 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 22 15:47:20.822409 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 22 15:47:20.822409 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 15:47:20.822409 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 15:47:20.822409 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 15:47:20.822409 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 15:47:20.822409 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 15:47:20.822409 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 15:47:20.822409 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 15:47:20.822409 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application with Lock( Aug 22 15:47:20.822409 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 15:47:20.822409 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self._enter() Aug 22 15:47:20.822409 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 15:47:20.822409 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 15:47:20.823210 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 15:47:20.823210 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self.creator() Aug 22 15:47:20.823210 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 15:47:20.823210 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application created_value = creator( Aug 22 15:47:20.823210 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 22 15:47:20.823210 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 22 15:47:20.823210 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 22 15:47:20.823210 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 22 15:47:20.823210 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 22 15:47:20.823210 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 22 15:47:20.823210 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 22 15:47:20.823210 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application Aug 22 15:47:20.823210 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 246/485] 10.222.0.15 () {56 vars in 1102 bytes} [Mon Aug 22 15:47:20 2022] GET /identity/v3/roles/service => generated 85 bytes in 242 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 15:47:20.828854 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e30d74bd-14a2-4c36-be26-c63a9f815ed1 None None] Authenticating user token {{(pid=81767) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 15:47:20.902981 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG dbcounter [-] [81767] Writing DB stats keystone:SELECT=100 {{(pid=81767) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 15:47:20.949531 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa43864-ba16-4d92-a583-e3f87a4e2a6f None None] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:47:20.949876 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa43864-ba16-4d92-a583-e3f87a4e2a6f None None] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:47:20.950192 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa43864-ba16-4d92-a583-e3f87a4e2a6f None None] PATH_INFO: `/` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:47:20.951886 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 247/486] 10.222.0.15 () {54 vars in 927 bytes} [Mon Aug 22 15:47:20 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 15:47:20.964895 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aa3b18c-b660-4ef0-967a-ca515067c17c None None] REQUEST_METHOD: `POST` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:47:20.965262 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aa3b18c-b660-4ef0-967a-ca515067c17c None None] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:47:20.965533 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aa3b18c-b660-4ef0-967a-ca515067c17c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:47:20.987036 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e30d74bd-14a2-4c36-be26-c63a9f815ed1 None None] Validating token access rules against request {{(pid=81767) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 15:47:20.993327 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.auth.core [None req-8aa3b18c-b660-4ef0-967a-ca515067c17c None None] MFA Rules not processed for user `6b2199ae0fb0480aa917ada391f5ec54`. Rule list: `[]` (Enabled: `True`). {{(pid=81768) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 15:47:20.994634 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-8aa3b18c-b660-4ef0-967a-ca515067c17c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 15:47:21.073520 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b529912e-e7cb-426a-ba9a-0c847cb19338 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81767) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 15:47:21.074125 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b529912e-e7cb-426a-ba9a-0c847cb19338 None admin] REQUEST_METHOD: `GET` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:47:21.074405 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b529912e-e7cb-426a-ba9a-0c847cb19338 None admin] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:47:21.074676 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b529912e-e7cb-426a-ba9a-0c847cb19338 None admin] PATH_INFO: `/v3/roles` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:47:21.075364 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b529912e-e7cb-426a-ba9a-0c847cb19338 None admin] RBAC: Adding query filter params (name=service) {{(pid=81767) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 15:47:21.075661 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b529912e-e7cb-426a-ba9a-0c847cb19338 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 15:47:21.077030 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b529912e-e7cb-426a-ba9a-0c847cb19338 None admin] RBAC: Authorization granted {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 15:47:21.077195 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 248/487] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Aug 22 15:47:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 22 15:47:21.094314 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 240/488] 10.222.0.15 () {56 vars in 1103 bytes} [Mon Aug 22 15:47:20 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 266 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 22 15:47:21.099746 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aa3b18c-b660-4ef0-967a-ca515067c17c None None] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 15:47:21.163058 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b529912e-e7cb-426a-ba9a-0c847cb19338 None admin] Authenticating user token {{(pid=81767) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 15:47:21.289958 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aa3b18c-b660-4ef0-967a-ca515067c17c None None] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 15:47:21.301285 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b529912e-e7cb-426a-ba9a-0c847cb19338 None admin] Validating token access rules against request {{(pid=81767) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 15:47:21.320226 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=100 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 15:47:21.324296 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG dbcounter [-] [81767] Writing DB stats keystone:SELECT=100 {{(pid=81767) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 15:47:21.352598 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-443af021-d60b-4429-89e3-59732a01b3ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 15:47:21.353219 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-443af021-d60b-4429-89e3-59732a01b3ba None admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:47:21.353489 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-443af021-d60b-4429-89e3-59732a01b3ba None admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:47:21.353748 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-443af021-d60b-4429-89e3-59732a01b3ba None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:47:21.363958 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-840e795f-cc17-4c76-9261-70eb0e400e1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81767) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 15:47:21.364673 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-840e795f-cc17-4c76-9261-70eb0e400e1a None admin] REQUEST_METHOD: `GET` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:47:21.364946 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-840e795f-cc17-4c76-9261-70eb0e400e1a None admin] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:47:21.365215 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-840e795f-cc17-4c76-9261-70eb0e400e1a None admin] PATH_INFO: `/v3/roles/service` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:47:21.373244 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-443af021-d60b-4429-89e3-59732a01b3ba None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 15:47:21.377335 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-443af021-d60b-4429-89e3-59732a01b3ba None admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 15:47:21.379954 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-840e795f-cc17-4c76-9261-70eb0e400e1a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 22 15:47:21.385102 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application [None req-840e795f-cc17-4c76-9261-70eb0e400e1a None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 22 15:47:21.385102 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 15:47:21.385102 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 15:47:21.385102 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 15:47:21.385102 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 15:47:21.385102 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 15:47:21.385102 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 15:47:21.385102 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 15:47:21.385102 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 15:47:21.385102 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 15:47:21.385102 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 15:47:21.385102 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 15:47:21.385102 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 22 15:47:21.385102 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 22 15:47:21.385102 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 22 15:47:21.385102 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 22 15:47:21.393435 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 22 15:47:21.393435 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 22 15:47:21.393435 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 22 15:47:21.393435 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 22 15:47:21.393435 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 15:47:21.393435 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 15:47:21.393435 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 15:47:21.393435 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 15:47:21.393435 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 15:47:21.393435 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 15:47:21.393435 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 15:47:21.393435 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application with Lock( Aug 22 15:47:21.393435 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 15:47:21.393435 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return self._enter() Aug 22 15:47:21.393435 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 15:47:21.393435 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 15:47:21.394228 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 15:47:21.394228 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return self.creator() Aug 22 15:47:21.394228 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 15:47:21.394228 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application created_value = creator( Aug 22 15:47:21.394228 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 22 15:47:21.394228 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 22 15:47:21.394228 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 22 15:47:21.394228 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 22 15:47:21.394228 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 22 15:47:21.394228 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 22 15:47:21.394228 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 22 15:47:21.394228 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application Aug 22 15:47:21.394228 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 241/489] 10.222.0.15 () {56 vars in 1102 bytes} [Mon Aug 22 15:47:21 2022] GET /identity/v3/roles/service => generated 85 bytes in 225 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 22 15:47:21.394228 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-840e795f-cc17-4c76-9261-70eb0e400e1a None admin] Authenticating user token {{(pid=81767) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 15:47:21.403364 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application [None req-443af021-d60b-4429-89e3-59732a01b3ba None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 22 15:47:21.403364 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 15:47:21.403364 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 15:47:21.403364 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 15:47:21.403364 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 15:47:21.403364 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 15:47:21.403364 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 15:47:21.403364 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 15:47:21.403364 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 15:47:21.403364 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 15:47:21.403364 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 15:47:21.403364 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 15:47:21.403364 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 22 15:47:21.403364 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 22 15:47:21.403364 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 22 15:47:21.403364 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 22 15:47:21.404269 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 15:47:21.404269 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 15:47:21.404269 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 15:47:21.404269 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 15:47:21.404269 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 15:47:21.404269 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 15:47:21.404269 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 15:47:21.404269 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application with Lock( Aug 22 15:47:21.404269 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 15:47:21.404269 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self._enter() Aug 22 15:47:21.404269 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 15:47:21.404269 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 15:47:21.404269 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 15:47:21.404269 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self.creator() Aug 22 15:47:21.404269 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 15:47:21.404269 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application created_value = creator( Aug 22 15:47:21.404269 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 22 15:47:21.405107 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 22 15:47:21.405107 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 22 15:47:21.405107 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application Aug 22 15:47:21.407208 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 249/490] 10.222.0.15 () {56 vars in 1108 bytes} [Mon Aug 22 15:47:21 2022] GET /identity/v3/domains/Default => generated 87 bytes in 308 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 15:47:21.408374 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-443af021-d60b-4429-89e3-59732a01b3ba None admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 15:47:21.522182 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-840e795f-cc17-4c76-9261-70eb0e400e1a None admin] Validating token access rules against request {{(pid=81767) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 15:47:21.550464 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-443af021-d60b-4429-89e3-59732a01b3ba None admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 15:47:21.569849 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d27b366d-51d2-4905-b0c6-e8b772728c43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81767) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 15:47:21.570864 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d27b366d-51d2-4905-b0c6-e8b772728c43 None admin] REQUEST_METHOD: `GET` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:47:21.571142 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d27b366d-51d2-4905-b0c6-e8b772728c43 None admin] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:47:21.571443 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d27b366d-51d2-4905-b0c6-e8b772728c43 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:47:21.584559 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d27b366d-51d2-4905-b0c6-e8b772728c43 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 22 15:47:21.590746 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application [None req-d27b366d-51d2-4905-b0c6-e8b772728c43 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 22 15:47:21.590746 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 15:47:21.590746 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 15:47:21.590746 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 15:47:21.590746 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 15:47:21.590746 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 15:47:21.590746 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 15:47:21.590746 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 15:47:21.590746 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 15:47:21.590746 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 15:47:21.590746 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 15:47:21.590746 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 15:47:21.590746 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 22 15:47:21.590746 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 22 15:47:21.590746 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 22 15:47:21.590746 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 22 15:47:21.591823 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 22 15:47:21.591823 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 22 15:47:21.591823 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 22 15:47:21.591823 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 22 15:47:21.591823 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 15:47:21.591823 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 15:47:21.591823 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 15:47:21.591823 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 15:47:21.591823 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 15:47:21.591823 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 15:47:21.591823 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 15:47:21.591823 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application with Lock( Aug 22 15:47:21.591823 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 15:47:21.591823 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return self._enter() Aug 22 15:47:21.591823 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 15:47:21.591823 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 15:47:21.592629 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 15:47:21.592629 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return self.creator() Aug 22 15:47:21.592629 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 15:47:21.592629 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application created_value = creator( Aug 22 15:47:21.592629 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 22 15:47:21.592629 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 22 15:47:21.592629 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 22 15:47:21.592629 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 22 15:47:21.592629 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 22 15:47:21.592629 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 22 15:47:21.592629 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 22 15:47:21.592629 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application Aug 22 15:47:21.594901 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f12117ea-f10c-49d1-9dca-4b6fbb5157a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 15:47:21.599425 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f12117ea-f10c-49d1-9dca-4b6fbb5157a6 None admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:47:21.599425 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f12117ea-f10c-49d1-9dca-4b6fbb5157a6 None admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:47:21.599425 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f12117ea-f10c-49d1-9dca-4b6fbb5157a6 None admin] PATH_INFO: `/v3/roles` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:47:21.599425 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f12117ea-f10c-49d1-9dca-4b6fbb5157a6 None admin] RBAC: Adding query filter params (name=service) {{(pid=81768) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 15:47:21.599425 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f12117ea-f10c-49d1-9dca-4b6fbb5157a6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 15:47:21.599425 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f12117ea-f10c-49d1-9dca-4b6fbb5157a6 None admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 15:47:21.600381 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 242/491] 10.222.0.15 () {56 vars in 1102 bytes} [Mon Aug 22 15:47:21 2022] GET /identity/v3/roles/service => generated 85 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 15:47:21.601465 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d27b366d-51d2-4905-b0c6-e8b772728c43 None admin] Authenticating user token {{(pid=81767) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 15:47:21.614630 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 250/492] 10.222.0.15 () {56 vars in 1103 bytes} [Mon Aug 22 15:47:21 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 15:47:21.614630 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f12117ea-f10c-49d1-9dca-4b6fbb5157a6 None admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 15:47:21.647218 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG dbcounter [-] [81767] Writing DB stats keystone:SELECT=100 {{(pid=81767) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 15:47:21.653100 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=100 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 15:47:21.718923 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d27b366d-51d2-4905-b0c6-e8b772728c43 None admin] Validating token access rules against request {{(pid=81767) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 15:47:21.740550 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f12117ea-f10c-49d1-9dca-4b6fbb5157a6 None admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 15:47:21.760175 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e01e4e8c-53b2-498e-ac83-4098a4426110 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81767) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 15:47:21.761295 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e01e4e8c-53b2-498e-ac83-4098a4426110 None admin] REQUEST_METHOD: `GET` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:47:21.761575 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e01e4e8c-53b2-498e-ac83-4098a4426110 None admin] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:47:21.761843 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e01e4e8c-53b2-498e-ac83-4098a4426110 None admin] PATH_INFO: `/v3/domains` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:47:21.762488 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e01e4e8c-53b2-498e-ac83-4098a4426110 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81767) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 15:47:21.762781 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e01e4e8c-53b2-498e-ac83-4098a4426110 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 15:47:21.764770 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e01e4e8c-53b2-498e-ac83-4098a4426110 None admin] RBAC: Authorization granted {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 15:47:21.778647 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaca9451-f701-4c74-850a-04f7a8855d14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 15:47:21.779643 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaca9451-f701-4c74-850a-04f7a8855d14 None admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:47:21.779751 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 243/493] 10.222.0.15 () {56 vars in 1109 bytes} [Mon Aug 22 15:47:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 15:47:21.780396 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaca9451-f701-4c74-850a-04f7a8855d14 None admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:47:21.780868 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaca9451-f701-4c74-850a-04f7a8855d14 None admin] PATH_INFO: `/v3/roles` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:47:21.781758 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaca9451-f701-4c74-850a-04f7a8855d14 None admin] RBAC: Adding query filter params (name=service) {{(pid=81768) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 15:47:21.782255 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaca9451-f701-4c74-850a-04f7a8855d14 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 15:47:21.783884 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaca9451-f701-4c74-850a-04f7a8855d14 None admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 15:47:21.785058 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e01e4e8c-53b2-498e-ac83-4098a4426110 None admin] Authenticating user token {{(pid=81767) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 15:47:21.801157 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 251/494] 10.222.0.15 () {56 vars in 1103 bytes} [Mon Aug 22 15:47:21 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 15:47:21.802511 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaca9451-f701-4c74-850a-04f7a8855d14 None admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 15:47:21.898480 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG dbcounter [-] [81767] Writing DB stats keystone:SELECT=100 {{(pid=81767) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 15:47:21.902911 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=100 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 15:47:21.907584 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e01e4e8c-53b2-498e-ac83-4098a4426110 None admin] Validating token access rules against request {{(pid=81767) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 15:47:21.913502 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaca9451-f701-4c74-850a-04f7a8855d14 None admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 15:47:21.945071 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e48accf-d409-46bf-9614-de090e2f1e03 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81767) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 15:47:21.945708 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e48accf-d409-46bf-9614-de090e2f1e03 None admin] REQUEST_METHOD: `GET` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:47:21.945987 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e48accf-d409-46bf-9614-de090e2f1e03 None admin] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:47:21.946259 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e48accf-d409-46bf-9614-de090e2f1e03 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:47:21.949549 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49434d0a-bfb6-4061-8761-29657a67aa7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 15:47:21.950175 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49434d0a-bfb6-4061-8761-29657a67aa7f None admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:47:21.950449 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49434d0a-bfb6-4061-8761-29657a67aa7f None admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:47:21.950716 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49434d0a-bfb6-4061-8761-29657a67aa7f None admin] PATH_INFO: `/v3/users/nova` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:47:21.957376 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e48accf-d409-46bf-9614-de090e2f1e03 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 15:47:21.958625 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49434d0a-bfb6-4061-8761-29657a67aa7f None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 15:47:21.958897 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e48accf-d409-46bf-9614-de090e2f1e03 None admin] RBAC: Authorization granted {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 15:47:21.960081 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49434d0a-bfb6-4061-8761-29657a67aa7f None admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 15:47:21.972935 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application [None req-49434d0a-bfb6-4061-8761-29657a67aa7f None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 22 15:47:21.972935 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 15:47:21.972935 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 15:47:21.972935 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 15:47:21.972935 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 15:47:21.972935 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 15:47:21.972935 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 15:47:21.972935 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 15:47:21.972935 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 15:47:21.972935 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 15:47:21.972935 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 15:47:21.972935 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 15:47:21.972935 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 22 15:47:21.972935 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 22 15:47:21.972935 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 22 15:47:21.972935 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 22 15:47:21.974048 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 15:47:21.974048 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 15:47:21.974048 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 22 15:47:21.974048 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 22 15:47:21.974048 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 22 15:47:21.974048 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 22 15:47:21.974048 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 15:47:21.974048 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 15:47:21.974048 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 15:47:21.974048 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 15:47:21.974048 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 15:47:21.974048 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application with Lock( Aug 22 15:47:21.974048 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 15:47:21.974048 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self._enter() Aug 22 15:47:21.974048 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 15:47:21.974048 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 15:47:21.974048 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 15:47:21.974048 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self.creator() Aug 22 15:47:21.974980 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 15:47:21.974980 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application created_value = creator( Aug 22 15:47:21.974980 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 22 15:47:21.974980 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 22 15:47:21.974980 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 22 15:47:21.974980 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 22 15:47:21.974980 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 22 15:47:21.974980 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 22 15:47:21.974980 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 22 15:47:21.974980 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application Aug 22 15:47:21.974980 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 252/495] 10.222.0.15 () {56 vars in 1093 bytes} [Mon Aug 22 15:47:21 2022] GET /identity/v3/users/nova => generated 82 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 15:47:21.980468 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application [None req-3e48accf-d409-46bf-9614-de090e2f1e03 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 22 15:47:21.980468 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 15:47:21.980468 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 15:47:21.980468 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 15:47:21.980468 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 15:47:21.980468 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 15:47:21.980468 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 15:47:21.980468 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 15:47:21.980468 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 15:47:21.980468 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 15:47:21.980468 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 15:47:21.980468 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 15:47:21.980468 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 22 15:47:21.980468 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 22 15:47:21.980468 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 22 15:47:21.980468 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 22 15:47:21.981343 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 15:47:21.981343 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 15:47:21.981343 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 15:47:21.981343 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 15:47:21.981343 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 15:47:21.981343 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 15:47:21.981343 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 15:47:21.981343 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application with Lock( Aug 22 15:47:21.981343 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 15:47:21.981343 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return self._enter() Aug 22 15:47:21.981343 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 15:47:21.981343 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 15:47:21.981343 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 15:47:21.981343 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return self.creator() Aug 22 15:47:21.981343 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 15:47:21.981343 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application created_value = creator( Aug 22 15:47:21.981343 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 22 15:47:21.982170 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 22 15:47:21.982170 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 22 15:47:21.982170 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application Aug 22 15:47:21.982340 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49434d0a-bfb6-4061-8761-29657a67aa7f None admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 15:47:21.982407 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 244/496] 10.222.0.15 () {56 vars in 1108 bytes} [Mon Aug 22 15:47:21 2022] GET /identity/v3/domains/Default => generated 87 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 15:47:21.982407 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31ebee37-ae90-4338-89f1-bd5298e8728a None None] REQUEST_METHOD: `GET` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:47:21.982407 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31ebee37-ae90-4338-89f1-bd5298e8728a None None] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:47:21.982407 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31ebee37-ae90-4338-89f1-bd5298e8728a None None] PATH_INFO: `/` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:47:21.982407 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 245/497] 10.222.0.15 () {54 vars in 927 bytes} [Mon Aug 22 15:47:21 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 15:47:21.982407 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e0c838c-331b-4dfb-a584-9e1a625b0de5 None None] REQUEST_METHOD: `GET` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:47:21.982407 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e0c838c-331b-4dfb-a584-9e1a625b0de5 None None] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:47:21.982775 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e0c838c-331b-4dfb-a584-9e1a625b0de5 None None] PATH_INFO: `/` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:47:21.982775 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 246/498] 10.222.0.15 () {54 vars in 927 bytes} [Mon Aug 22 15:47:21 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 15:47:21.982775 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e0c838c-331b-4dfb-a584-9e1a625b0de5 None None] Authenticating user token {{(pid=81767) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 15:47:22.085583 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49434d0a-bfb6-4061-8761-29657a67aa7f None admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 15:47:22.089264 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e0c838c-331b-4dfb-a584-9e1a625b0de5 None None] Validating token access rules against request {{(pid=81767) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 15:47:22.119471 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b412880b-f41e-4bfe-a939-ed18046ef518 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 15:47:22.120089 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b412880b-f41e-4bfe-a939-ed18046ef518 None admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:47:22.120900 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b412880b-f41e-4bfe-a939-ed18046ef518 None admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:47:22.121160 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b412880b-f41e-4bfe-a939-ed18046ef518 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:47:22.125829 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4594e1fe-eb9b-4def-8120-555022be62dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81767) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 15:47:22.126585 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4594e1fe-eb9b-4def-8120-555022be62dc None admin] REQUEST_METHOD: `GET` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:47:22.126927 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4594e1fe-eb9b-4def-8120-555022be62dc None admin] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:47:22.127263 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4594e1fe-eb9b-4def-8120-555022be62dc None admin] PATH_INFO: `/v3/users` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:47:22.128141 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4594e1fe-eb9b-4def-8120-555022be62dc None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81767) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 15:47:22.128516 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4594e1fe-eb9b-4def-8120-555022be62dc None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 15:47:22.130011 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4594e1fe-eb9b-4def-8120-555022be62dc None admin] RBAC: Authorization granted {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 15:47:22.131645 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b412880b-f41e-4bfe-a939-ed18046ef518 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 15:47:22.133105 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b412880b-f41e-4bfe-a939-ed18046ef518 None admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 15:47:22.147703 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application [None req-b412880b-f41e-4bfe-a939-ed18046ef518 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 22 15:47:22.147703 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 15:47:22.147703 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 15:47:22.147703 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 15:47:22.147703 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 15:47:22.147703 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 15:47:22.147703 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 15:47:22.147703 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 15:47:22.147703 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 15:47:22.147703 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 15:47:22.147703 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 15:47:22.147703 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 15:47:22.147703 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 22 15:47:22.147703 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 22 15:47:22.147703 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 22 15:47:22.147703 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 22 15:47:22.148767 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 15:47:22.148767 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 15:47:22.148767 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 15:47:22.148767 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 15:47:22.148767 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 15:47:22.148767 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 15:47:22.148767 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 15:47:22.148767 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application with Lock( Aug 22 15:47:22.148767 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 15:47:22.148767 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self._enter() Aug 22 15:47:22.148767 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 15:47:22.148767 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 15:47:22.148767 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 15:47:22.148767 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self.creator() Aug 22 15:47:22.148767 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 15:47:22.148767 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application created_value = creator( Aug 22 15:47:22.148767 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 22 15:47:22.149800 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 22 15:47:22.149800 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 22 15:47:22.149800 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application Aug 22 15:47:22.149800 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 253/499] 10.222.0.15 () {56 vars in 1108 bytes} [Mon Aug 22 15:47:21 2022] GET /identity/v3/domains/Default => generated 87 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 15:47:22.149800 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b412880b-f41e-4bfe-a939-ed18046ef518 None admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 15:47:22.155488 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=100 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 15:47:22.172209 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 247/500] 10.222.0.15 () {56 vars in 1133 bytes} [Mon Aug 22 15:47:21 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 15:47:22.173444 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-643f469c-4595-44bb-9c4b-00388267ad39 None None] REQUEST_METHOD: `POST` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:47:22.173730 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-643f469c-4595-44bb-9c4b-00388267ad39 None None] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:47:22.174030 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-643f469c-4595-44bb-9c4b-00388267ad39 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:47:22.182099 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG dbcounter [-] [81767] Writing DB stats keystone:SELECT=100 {{(pid=81767) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 15:47:22.193486 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.auth.core [None req-643f469c-4595-44bb-9c4b-00388267ad39 None None] MFA Rules not processed for user `6b2199ae0fb0480aa917ada391f5ec54`. Rule list: `[]` (Enabled: `True`). {{(pid=81767) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 15:47:22.194866 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.fernet_utils [None req-643f469c-4595-44bb-9c4b-00388267ad39 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81767) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 15:47:22.254360 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b412880b-f41e-4bfe-a939-ed18046ef518 None admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 15:47:22.264491 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 248/501] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Aug 22 15:47:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 22 15:47:22.265810 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-769c7b59-1bf4-4ec6-af53-9ebab5c14031 None None] REQUEST_METHOD: `POST` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:47:22.266103 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-769c7b59-1bf4-4ec6-af53-9ebab5c14031 None None] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:47:22.266376 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-769c7b59-1bf4-4ec6-af53-9ebab5c14031 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:47:22.293739 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dff4925-0661-4518-8a62-b1f9040c8eeb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 15:47:22.294437 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dff4925-0661-4518-8a62-b1f9040c8eeb None admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:47:22.294715 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dff4925-0661-4518-8a62-b1f9040c8eeb None admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:47:22.295080 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dff4925-0661-4518-8a62-b1f9040c8eeb None admin] PATH_INFO: `/v3/domains` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:47:22.295248 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.auth.core [None req-769c7b59-1bf4-4ec6-af53-9ebab5c14031 None None] MFA Rules not processed for user `6b2199ae0fb0480aa917ada391f5ec54`. Rule list: `[]` (Enabled: `True`). {{(pid=81767) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 15:47:22.295752 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dff4925-0661-4518-8a62-b1f9040c8eeb None admin] RBAC: Adding query filter params (name=Default) {{(pid=81768) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 15:47:22.296141 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dff4925-0661-4518-8a62-b1f9040c8eeb None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 15:47:22.296518 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.fernet_utils [None req-769c7b59-1bf4-4ec6-af53-9ebab5c14031 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81767) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 15:47:22.297493 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dff4925-0661-4518-8a62-b1f9040c8eeb None admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 15:47:22.315700 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 254/502] 10.222.0.15 () {56 vars in 1109 bytes} [Mon Aug 22 15:47:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 15:47:22.315700 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dff4925-0661-4518-8a62-b1f9040c8eeb None admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 15:47:22.387320 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 249/503] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Aug 22 15:47:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 22 15:47:22.387320 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-769c7b59-1bf4-4ec6-af53-9ebab5c14031 None None] Authenticating user token {{(pid=81767) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 15:47:22.396584 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=100 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 15:47:22.455556 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dff4925-0661-4518-8a62-b1f9040c8eeb None admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 15:47:22.460196 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG dbcounter [-] [81767] Writing DB stats keystone:SELECT=100 {{(pid=81767) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 15:47:22.491017 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e687f5ac-5333-4f15-8aaf-886634c6c779 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 15:47:22.491902 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e687f5ac-5333-4f15-8aaf-886634c6c779 None admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:47:22.492351 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e687f5ac-5333-4f15-8aaf-886634c6c779 None admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:47:22.492753 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e687f5ac-5333-4f15-8aaf-886634c6c779 None admin] PATH_INFO: `/v3/domains` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:47:22.493574 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e687f5ac-5333-4f15-8aaf-886634c6c779 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81768) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 15:47:22.494058 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e687f5ac-5333-4f15-8aaf-886634c6c779 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 15:47:22.497281 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e687f5ac-5333-4f15-8aaf-886634c6c779 None admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 15:47:22.503552 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-769c7b59-1bf4-4ec6-af53-9ebab5c14031 None None] Validating token access rules against request {{(pid=81767) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 15:47:22.514498 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 255/504] 10.222.0.15 () {56 vars in 1109 bytes} [Mon Aug 22 15:47:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 15:47:22.515891 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e687f5ac-5333-4f15-8aaf-886634c6c779 None admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 15:47:22.539772 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0c7ac02-2687-47be-adaa-80f91f12c4f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81767) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 15:47:22.540481 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c7ac02-2687-47be-adaa-80f91f12c4f1 None admin] REQUEST_METHOD: `GET` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:47:22.540831 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c7ac02-2687-47be-adaa-80f91f12c4f1 None admin] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:47:22.541200 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c7ac02-2687-47be-adaa-80f91f12c4f1 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:47:22.617456 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0c7ac02-2687-47be-adaa-80f91f12c4f1 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 15:47:22.619187 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0c7ac02-2687-47be-adaa-80f91f12c4f1 None admin] RBAC: Authorization granted {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 15:47:22.625899 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e687f5ac-5333-4f15-8aaf-886634c6c779 None admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 15:47:22.652453 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=100 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 15:47:22.665886 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fddd54b-9e79-41ff-8a87-3803453a706c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 15:47:22.668004 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fddd54b-9e79-41ff-8a87-3803453a706c None admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:47:22.668004 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fddd54b-9e79-41ff-8a87-3803453a706c None admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:47:22.668004 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fddd54b-9e79-41ff-8a87-3803453a706c None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:47:22.675253 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fddd54b-9e79-41ff-8a87-3803453a706c None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 15:47:22.676734 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fddd54b-9e79-41ff-8a87-3803453a706c None admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 15:47:22.684533 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG dbcounter [-] [81767] Writing DB stats keystone:SELECT=100 {{(pid=81767) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 15:47:22.689165 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application [None req-1fddd54b-9e79-41ff-8a87-3803453a706c None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Aug 22 15:47:22.689165 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 15:47:22.689165 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 15:47:22.689165 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 15:47:22.689165 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 15:47:22.689165 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 15:47:22.689165 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 15:47:22.689165 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 15:47:22.689165 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 15:47:22.689165 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 15:47:22.689165 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 15:47:22.689165 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 15:47:22.689165 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 22 15:47:22.689165 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 22 15:47:22.689165 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 22 15:47:22.689165 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 22 15:47:22.690136 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 15:47:22.690136 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 15:47:22.690136 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 22 15:47:22.690136 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 22 15:47:22.690136 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 22 15:47:22.690136 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 22 15:47:22.690136 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 15:47:22.690136 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 15:47:22.690136 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 15:47:22.690136 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 15:47:22.690136 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 15:47:22.690136 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application with Lock( Aug 22 15:47:22.690136 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 15:47:22.690136 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self._enter() Aug 22 15:47:22.690136 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 15:47:22.690136 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 15:47:22.690136 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 15:47:22.690136 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self.creator() Aug 22 15:47:22.691015 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 15:47:22.691015 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application created_value = creator( Aug 22 15:47:22.691015 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 22 15:47:22.691015 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 22 15:47:22.691015 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 22 15:47:22.691015 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 22 15:47:22.691015 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 22 15:47:22.691015 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 22 15:47:22.691015 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Aug 22 15:47:22.691015 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application Aug 22 15:47:22.691015 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 256/505] 10.222.0.15 () {56 vars in 1099 bytes} [Mon Aug 22 15:47:22 2022] GET /identity/v3/users/cinder => generated 84 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 15:47:22.692106 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fddd54b-9e79-41ff-8a87-3803453a706c None admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 15:47:22.726130 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 250/506] 10.222.0.15 () {58 vars in 1282 bytes} [Mon Aug 22 15:47:22 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 341 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 22 15:47:22.726964 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0c7ac02-2687-47be-adaa-80f91f12c4f1 None admin] Authenticating user token {{(pid=81767) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 15:47:22.798219 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fddd54b-9e79-41ff-8a87-3803453a706c None admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 15:47:22.831864 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c4d5c08-abac-41ad-8b04-9a5a81f71bfc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 15:47:22.832445 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0c7ac02-2687-47be-adaa-80f91f12c4f1 None admin] Validating token access rules against request {{(pid=81767) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 15:47:22.832584 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c4d5c08-abac-41ad-8b04-9a5a81f71bfc None admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:47:22.832745 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c4d5c08-abac-41ad-8b04-9a5a81f71bfc None admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:47:22.833011 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c4d5c08-abac-41ad-8b04-9a5a81f71bfc None admin] PATH_INFO: `/v3/users/glance` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:47:22.840870 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c4d5c08-abac-41ad-8b04-9a5a81f71bfc None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 15:47:22.842306 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c4d5c08-abac-41ad-8b04-9a5a81f71bfc None admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 15:47:22.853247 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application [None req-1c4d5c08-abac-41ad-8b04-9a5a81f71bfc None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Aug 22 15:47:22.853247 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 15:47:22.853247 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 15:47:22.853247 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 15:47:22.853247 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 15:47:22.853247 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 15:47:22.853247 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 15:47:22.853247 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 15:47:22.853247 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 15:47:22.853247 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 15:47:22.853247 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 15:47:22.853247 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 15:47:22.853247 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 22 15:47:22.853247 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 22 15:47:22.853247 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 22 15:47:22.853247 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 22 15:47:22.854135 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 15:47:22.854135 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 15:47:22.854135 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 22 15:47:22.854135 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 22 15:47:22.854135 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 22 15:47:22.854135 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 22 15:47:22.854135 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 15:47:22.854135 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 15:47:22.854135 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 15:47:22.854135 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 15:47:22.854135 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 15:47:22.854135 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application with Lock( Aug 22 15:47:22.854135 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 15:47:22.854135 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self._enter() Aug 22 15:47:22.854135 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 15:47:22.854135 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 15:47:22.854135 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 15:47:22.854135 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self.creator() Aug 22 15:47:22.855665 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 15:47:22.855665 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application created_value = creator( Aug 22 15:47:22.855665 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 22 15:47:22.855665 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 22 15:47:22.855665 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 22 15:47:22.855665 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 22 15:47:22.855665 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 22 15:47:22.855665 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 22 15:47:22.855665 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Aug 22 15:47:22.855665 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application Aug 22 15:47:22.855665 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 257/507] 10.222.0.15 () {56 vars in 1099 bytes} [Mon Aug 22 15:47:22 2022] GET /identity/v3/users/glance => generated 84 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 15:47:22.855665 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c4d5c08-abac-41ad-8b04-9a5a81f71bfc None admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 15:47:22.871445 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16378373-0864-4e03-a260-76afb35289c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81767) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 15:47:22.871996 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16378373-0864-4e03-a260-76afb35289c0 None admin] REQUEST_METHOD: `GET` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:47:22.872273 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16378373-0864-4e03-a260-76afb35289c0 None admin] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:47:22.872538 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16378373-0864-4e03-a260-76afb35289c0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:47:22.882946 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16378373-0864-4e03-a260-76afb35289c0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 15:47:22.884407 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16378373-0864-4e03-a260-76afb35289c0 None admin] RBAC: Authorization granted {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 15:47:22.898948 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application [None req-16378373-0864-4e03-a260-76afb35289c0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 22 15:47:22.898948 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 15:47:22.898948 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 15:47:22.898948 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 15:47:22.898948 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 15:47:22.898948 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 15:47:22.898948 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 15:47:22.898948 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 15:47:22.898948 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 15:47:22.898948 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 15:47:22.898948 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 15:47:22.898948 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 15:47:22.898948 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 22 15:47:22.898948 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 22 15:47:22.898948 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 22 15:47:22.898948 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 22 15:47:22.900099 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 15:47:22.900099 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 15:47:22.900099 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 15:47:22.900099 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 15:47:22.900099 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 15:47:22.900099 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 15:47:22.900099 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 15:47:22.900099 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application with Lock( Aug 22 15:47:22.900099 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 15:47:22.900099 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return self._enter() Aug 22 15:47:22.900099 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 15:47:22.900099 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 15:47:22.900099 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 15:47:22.900099 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return self.creator() Aug 22 15:47:22.900099 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 15:47:22.900099 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application created_value = creator( Aug 22 15:47:22.900099 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 22 15:47:22.900999 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 22 15:47:22.900999 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 22 15:47:22.900999 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application Aug 22 15:47:22.900999 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 251/508] 10.222.0.15 () {56 vars in 1108 bytes} [Mon Aug 22 15:47:22 2022] GET /identity/v3/domains/Default => generated 87 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 15:47:22.901889 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16378373-0864-4e03-a260-76afb35289c0 None admin] Authenticating user token {{(pid=81767) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 15:47:22.905237 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=100 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 15:47:22.921377 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG dbcounter [-] [81767] Writing DB stats keystone:SELECT=100 {{(pid=81767) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 15:47:22.963264 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c4d5c08-abac-41ad-8b04-9a5a81f71bfc None admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 15:47:23.001040 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0b2d83c-e6a7-476e-9dbb-9517b2be2392 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 15:47:23.001654 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0b2d83c-e6a7-476e-9dbb-9517b2be2392 None admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:47:23.001931 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0b2d83c-e6a7-476e-9dbb-9517b2be2392 None admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:47:23.002204 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0b2d83c-e6a7-476e-9dbb-9517b2be2392 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:47:23.009520 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16378373-0864-4e03-a260-76afb35289c0 None admin] Validating token access rules against request {{(pid=81767) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 15:47:23.013037 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0b2d83c-e6a7-476e-9dbb-9517b2be2392 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 22 15:47:23.018291 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application [None req-b0b2d83c-e6a7-476e-9dbb-9517b2be2392 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 22 15:47:23.018291 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 15:47:23.018291 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 15:47:23.018291 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 15:47:23.018291 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 15:47:23.018291 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 15:47:23.018291 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 15:47:23.018291 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 15:47:23.018291 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 15:47:23.018291 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 15:47:23.018291 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 15:47:23.018291 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 15:47:23.018291 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 22 15:47:23.018291 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 22 15:47:23.018291 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 22 15:47:23.018291 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 22 15:47:23.019224 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 22 15:47:23.019224 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 22 15:47:23.019224 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 22 15:47:23.019224 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 22 15:47:23.019224 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 15:47:23.019224 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 15:47:23.019224 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 15:47:23.019224 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 15:47:23.019224 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 15:47:23.019224 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 15:47:23.019224 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 15:47:23.019224 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application with Lock( Aug 22 15:47:23.019224 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 15:47:23.019224 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self._enter() Aug 22 15:47:23.019224 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 15:47:23.019224 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 15:47:23.020119 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 15:47:23.020119 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self.creator() Aug 22 15:47:23.020119 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 15:47:23.020119 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application created_value = creator( Aug 22 15:47:23.020119 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 22 15:47:23.020119 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 22 15:47:23.020119 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 22 15:47:23.020119 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 22 15:47:23.020119 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 22 15:47:23.020119 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 22 15:47:23.020119 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 22 15:47:23.020119 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application Aug 22 15:47:23.020119 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 258/509] 10.222.0.15 () {56 vars in 1102 bytes} [Mon Aug 22 15:47:22 2022] GET /identity/v3/roles/service => generated 85 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 15:47:23.020824 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0b2d83c-e6a7-476e-9dbb-9517b2be2392 None admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 15:47:23.048442 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bca67c48-404c-4bae-9aa0-afe8fb094d92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81767) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 15:47:23.049100 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca67c48-404c-4bae-9aa0-afe8fb094d92 None admin] REQUEST_METHOD: `GET` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:47:23.049371 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca67c48-404c-4bae-9aa0-afe8fb094d92 None admin] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:47:23.049638 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca67c48-404c-4bae-9aa0-afe8fb094d92 None admin] PATH_INFO: `/v3/users` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:47:23.050367 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bca67c48-404c-4bae-9aa0-afe8fb094d92 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=81767) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 15:47:23.050670 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bca67c48-404c-4bae-9aa0-afe8fb094d92 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 15:47:23.052120 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bca67c48-404c-4bae-9aa0-afe8fb094d92 None admin] RBAC: Authorization granted {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 15:47:23.071512 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 252/510] 10.222.0.15 () {56 vars in 1137 bytes} [Mon Aug 22 15:47:22 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 15:47:23.072427 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bca67c48-404c-4bae-9aa0-afe8fb094d92 None admin] Authenticating user token {{(pid=81767) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 15:47:23.140852 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0b2d83c-e6a7-476e-9dbb-9517b2be2392 None admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 15:47:23.143963 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=100 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 15:47:23.161157 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG dbcounter [-] [81767] Writing DB stats keystone:SELECT=100 {{(pid=81767) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 15:47:23.189102 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bca67c48-404c-4bae-9aa0-afe8fb094d92 None admin] Validating token access rules against request {{(pid=81767) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 15:47:23.189511 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d554516-7335-478b-90f4-6de1291f3a5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 15:47:23.190142 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d554516-7335-478b-90f4-6de1291f3a5e None admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:47:23.190421 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d554516-7335-478b-90f4-6de1291f3a5e None admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:47:23.190697 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d554516-7335-478b-90f4-6de1291f3a5e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:47:23.200867 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d554516-7335-478b-90f4-6de1291f3a5e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 15:47:23.202346 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d554516-7335-478b-90f4-6de1291f3a5e None admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 15:47:23.216975 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application [None req-4d554516-7335-478b-90f4-6de1291f3a5e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 22 15:47:23.216975 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 15:47:23.216975 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 15:47:23.216975 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 15:47:23.216975 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 15:47:23.216975 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 15:47:23.216975 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 15:47:23.216975 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 15:47:23.216975 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 15:47:23.216975 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 15:47:23.216975 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 15:47:23.216975 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 15:47:23.216975 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 22 15:47:23.216975 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 22 15:47:23.216975 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 22 15:47:23.216975 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 22 15:47:23.218076 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 15:47:23.218076 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 15:47:23.218076 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 15:47:23.218076 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 15:47:23.218076 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 15:47:23.218076 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 15:47:23.218076 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 15:47:23.218076 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application with Lock( Aug 22 15:47:23.218076 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 15:47:23.218076 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self._enter() Aug 22 15:47:23.218076 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 15:47:23.218076 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 15:47:23.218076 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 15:47:23.218076 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self.creator() Aug 22 15:47:23.218076 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 15:47:23.218076 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application created_value = creator( Aug 22 15:47:23.218076 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 22 15:47:23.219060 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 22 15:47:23.219060 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 22 15:47:23.219060 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application Aug 22 15:47:23.219060 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 259/511] 10.222.0.15 () {56 vars in 1108 bytes} [Mon Aug 22 15:47:23 2022] GET /identity/v3/domains/Default => generated 87 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 15:47:23.219060 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d554516-7335-478b-90f4-6de1291f3a5e None admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 15:47:23.228356 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bd073d9-27f8-4bc9-9364-62ad0fac75f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81767) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 15:47:23.228962 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bd073d9-27f8-4bc9-9364-62ad0fac75f1 None admin] REQUEST_METHOD: `GET` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:47:23.229239 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bd073d9-27f8-4bc9-9364-62ad0fac75f1 None admin] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:47:23.230017 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bd073d9-27f8-4bc9-9364-62ad0fac75f1 None admin] PATH_INFO: `/v3/users` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:47:23.230751 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bd073d9-27f8-4bc9-9364-62ad0fac75f1 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=81767) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 15:47:23.231052 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bd073d9-27f8-4bc9-9364-62ad0fac75f1 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 15:47:23.232511 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bd073d9-27f8-4bc9-9364-62ad0fac75f1 None admin] RBAC: Authorization granted {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 15:47:23.251321 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 253/512] 10.222.0.15 () {56 vars in 1137 bytes} [Mon Aug 22 15:47:23 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 15:47:23.252147 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bd073d9-27f8-4bc9-9364-62ad0fac75f1 None admin] Authenticating user token {{(pid=81767) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 15:47:23.326198 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d554516-7335-478b-90f4-6de1291f3a5e None admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 15:47:23.360102 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bd073d9-27f8-4bc9-9364-62ad0fac75f1 None admin] Validating token access rules against request {{(pid=81767) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 15:47:23.365844 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52e0c34b-42ed-4346-9808-231fd7853507 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 15:47:23.366442 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52e0c34b-42ed-4346-9808-231fd7853507 None admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:47:23.366716 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52e0c34b-42ed-4346-9808-231fd7853507 None admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:47:23.366984 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52e0c34b-42ed-4346-9808-231fd7853507 None admin] PATH_INFO: `/v3/domains` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:47:23.367652 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52e0c34b-42ed-4346-9808-231fd7853507 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81768) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 15:47:23.367958 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52e0c34b-42ed-4346-9808-231fd7853507 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 15:47:23.369410 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52e0c34b-42ed-4346-9808-231fd7853507 None admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 15:47:23.382281 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 260/513] 10.222.0.15 () {56 vars in 1109 bytes} [Mon Aug 22 15:47:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 15:47:23.383098 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52e0c34b-42ed-4346-9808-231fd7853507 None admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 15:47:23.395908 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG dbcounter [-] [81767] Writing DB stats keystone:SELECT=100 {{(pid=81767) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 15:47:23.400689 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=100 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 15:47:23.401117 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09e0c0ef-202a-4584-87c2-4e8dd63a67fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81767) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 15:47:23.402234 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09e0c0ef-202a-4584-87c2-4e8dd63a67fa None admin] REQUEST_METHOD: `GET` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:47:23.402511 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09e0c0ef-202a-4584-87c2-4e8dd63a67fa None admin] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:47:23.402783 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09e0c0ef-202a-4584-87c2-4e8dd63a67fa None admin] PATH_INFO: `/v3/roles` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:47:23.403479 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09e0c0ef-202a-4584-87c2-4e8dd63a67fa None admin] RBAC: Adding query filter params (name=service) {{(pid=81767) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 15:47:23.403782 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09e0c0ef-202a-4584-87c2-4e8dd63a67fa None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 15:47:23.405159 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09e0c0ef-202a-4584-87c2-4e8dd63a67fa None admin] RBAC: Authorization granted {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 15:47:23.413234 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 254/514] 10.222.0.15 () {56 vars in 1103 bytes} [Mon Aug 22 15:47:23 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 15:47:23.418979 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09e0c0ef-202a-4584-87c2-4e8dd63a67fa None admin] Authenticating user token {{(pid=81767) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 15:47:23.486568 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52e0c34b-42ed-4346-9808-231fd7853507 None admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 15:47:23.517539 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09e0c0ef-202a-4584-87c2-4e8dd63a67fa None admin] Validating token access rules against request {{(pid=81767) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 15:47:23.521227 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d544f80f-6758-49ac-8872-173f58433fe9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 15:47:23.521824 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d544f80f-6758-49ac-8872-173f58433fe9 None admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:47:23.522104 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d544f80f-6758-49ac-8872-173f58433fe9 None admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:47:23.522368 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d544f80f-6758-49ac-8872-173f58433fe9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:47:23.552080 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dea6ba1d-3aae-4b6f-9ee3-747738cf67fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81767) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 15:47:23.552715 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dea6ba1d-3aae-4b6f-9ee3-747738cf67fb None admin] REQUEST_METHOD: `GET` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:47:23.552940 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dea6ba1d-3aae-4b6f-9ee3-747738cf67fb None admin] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:47:23.553262 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dea6ba1d-3aae-4b6f-9ee3-747738cf67fb None admin] PATH_INFO: `/v3/domains` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:47:23.553986 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dea6ba1d-3aae-4b6f-9ee3-747738cf67fb None admin] RBAC: Adding query filter params (name=Default) {{(pid=81767) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 15:47:23.554283 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dea6ba1d-3aae-4b6f-9ee3-747738cf67fb None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 15:47:23.555763 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dea6ba1d-3aae-4b6f-9ee3-747738cf67fb None admin] RBAC: Authorization granted {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 15:47:23.568140 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 255/515] 10.222.0.15 () {56 vars in 1109 bytes} [Mon Aug 22 15:47:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 15:47:23.569114 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dea6ba1d-3aae-4b6f-9ee3-747738cf67fb None admin] Authenticating user token {{(pid=81767) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 15:47:23.597577 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d544f80f-6758-49ac-8872-173f58433fe9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 15:47:23.599063 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d544f80f-6758-49ac-8872-173f58433fe9 None admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 15:47:23.617996 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=100 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 15:47:23.629734 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG dbcounter [-] [81767] Writing DB stats keystone:SELECT=100 {{(pid=81767) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 15:47:23.673337 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dea6ba1d-3aae-4b6f-9ee3-747738cf67fb None admin] Validating token access rules against request {{(pid=81767) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 15:47:23.699592 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 261/516] 10.222.0.15 () {58 vars in 1282 bytes} [Mon Aug 22 15:47:23 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 317 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 22 15:47:23.700411 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d544f80f-6758-49ac-8872-173f58433fe9 None admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 15:47:23.711089 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e5099cb-2da4-48c9-b521-ffa299abd1a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81767) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 15:47:23.711725 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e5099cb-2da4-48c9-b521-ffa299abd1a2 None admin] REQUEST_METHOD: `GET` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:47:23.711997 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e5099cb-2da4-48c9-b521-ffa299abd1a2 None admin] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:47:23.712258 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e5099cb-2da4-48c9-b521-ffa299abd1a2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:47:23.782614 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e5099cb-2da4-48c9-b521-ffa299abd1a2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 15:47:23.784182 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e5099cb-2da4-48c9-b521-ffa299abd1a2 None admin] RBAC: Authorization granted {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 15:47:23.802186 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d544f80f-6758-49ac-8872-173f58433fe9 None admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 15:47:23.819234 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=100 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 15:47:23.837413 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00ff0880-18c4-4046-bb55-e88efa49cb32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 15:47:23.838009 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00ff0880-18c4-4046-bb55-e88efa49cb32 None admin] REQUEST_METHOD: `POST` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:47:23.838332 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00ff0880-18c4-4046-bb55-e88efa49cb32 None admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:47:23.838602 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00ff0880-18c4-4046-bb55-e88efa49cb32 None admin] PATH_INFO: `/v3/users` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:47:23.839821 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00ff0880-18c4-4046-bb55-e88efa49cb32 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 15:47:23.841261 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00ff0880-18c4-4046-bb55-e88efa49cb32 None admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 15:47:23.842415 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG dbcounter [-] [81767] Writing DB stats keystone:SELECT=100 {{(pid=81767) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 15:47:23.888390 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 256/517] 10.222.0.15 () {58 vars in 1282 bytes} [Mon Aug 22 15:47:23 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 320 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 22 15:47:23.889254 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e5099cb-2da4-48c9-b521-ffa299abd1a2 None admin] Authenticating user token {{(pid=81767) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 15:47:23.986056 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e5099cb-2da4-48c9-b521-ffa299abd1a2 None admin] Validating token access rules against request {{(pid=81767) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 15:47:23.988402 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 262/518] 10.222.0.15 () {60 vars in 1132 bytes} [Mon Aug 22 15:47:23 2022] POST /identity/v3/users => generated 247 bytes in 288 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 22 15:47:23.989293 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00ff0880-18c4-4046-bb55-e88efa49cb32 None admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 15:47:24.029883 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9c59239-dd0e-45d9-a54b-7f321b7e296a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81767) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 15:47:24.030788 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9c59239-dd0e-45d9-a54b-7f321b7e296a None admin] REQUEST_METHOD: `GET` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:47:24.031070 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9c59239-dd0e-45d9-a54b-7f321b7e296a None admin] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:47:24.031373 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9c59239-dd0e-45d9-a54b-7f321b7e296a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:47:24.043229 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9c59239-dd0e-45d9-a54b-7f321b7e296a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 15:47:24.044848 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9c59239-dd0e-45d9-a54b-7f321b7e296a None admin] RBAC: Authorization granted {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 15:47:24.062884 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application [None req-e9c59239-dd0e-45d9-a54b-7f321b7e296a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 22 15:47:24.062884 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 15:47:24.062884 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 15:47:24.062884 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 15:47:24.062884 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 15:47:24.062884 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 15:47:24.062884 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 15:47:24.062884 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 15:47:24.062884 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 15:47:24.062884 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 15:47:24.062884 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 15:47:24.062884 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 15:47:24.062884 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 22 15:47:24.062884 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 22 15:47:24.062884 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 22 15:47:24.062884 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 22 15:47:24.064197 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 15:47:24.064197 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 15:47:24.064197 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 15:47:24.064197 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 15:47:24.064197 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 15:47:24.064197 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 15:47:24.064197 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 15:47:24.064197 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application with Lock( Aug 22 15:47:24.064197 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 15:47:24.064197 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return self._enter() Aug 22 15:47:24.064197 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 15:47:24.064197 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 15:47:24.064197 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 15:47:24.064197 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return self.creator() Aug 22 15:47:24.064197 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 15:47:24.064197 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application created_value = creator( Aug 22 15:47:24.064197 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 22 15:47:24.065088 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 22 15:47:24.065088 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 22 15:47:24.065088 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application Aug 22 15:47:24.065088 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 257/519] 10.222.0.15 () {56 vars in 1108 bytes} [Mon Aug 22 15:47:23 2022] GET /identity/v3/domains/Default => generated 87 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 15:47:24.065088 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9c59239-dd0e-45d9-a54b-7f321b7e296a None admin] Authenticating user token {{(pid=81767) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 15:47:24.079579 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG dbcounter [-] [81767] Writing DB stats keystone:SELECT=100 {{(pid=81767) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 15:47:24.121008 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00ff0880-18c4-4046-bb55-e88efa49cb32 None admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 15:47:24.159203 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-978f052e-5e7e-41e8-a2f1-688333873198 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 15:47:24.159859 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-978f052e-5e7e-41e8-a2f1-688333873198 None admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:47:24.160138 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-978f052e-5e7e-41e8-a2f1-688333873198 None admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:47:24.161008 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-978f052e-5e7e-41e8-a2f1-688333873198 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:47:24.168665 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-978f052e-5e7e-41e8-a2f1-688333873198 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 15:47:24.170148 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-978f052e-5e7e-41e8-a2f1-688333873198 None admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 15:47:24.187510 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application [None req-978f052e-5e7e-41e8-a2f1-688333873198 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 22 15:47:24.187510 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 15:47:24.187510 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 15:47:24.187510 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 15:47:24.187510 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 15:47:24.187510 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 15:47:24.187510 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 15:47:24.187510 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 15:47:24.187510 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 15:47:24.187510 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 15:47:24.187510 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 15:47:24.187510 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 15:47:24.187510 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 22 15:47:24.187510 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 22 15:47:24.187510 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 22 15:47:24.188390 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 22 15:47:24.188390 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 15:47:24.188390 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 15:47:24.188390 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 15:47:24.188390 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 15:47:24.188390 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 15:47:24.188390 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 15:47:24.188390 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 15:47:24.188390 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application with Lock( Aug 22 15:47:24.188390 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 15:47:24.188390 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self._enter() Aug 22 15:47:24.188390 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 15:47:24.188390 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 15:47:24.188390 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 15:47:24.188390 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self.creator() Aug 22 15:47:24.188390 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 15:47:24.188390 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application created_value = creator( Aug 22 15:47:24.189214 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 22 15:47:24.189214 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 22 15:47:24.189214 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 22 15:47:24.189214 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 22 15:47:24.189214 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 22 15:47:24.189214 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 22 15:47:24.189214 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 22 15:47:24.189214 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application Aug 22 15:47:24.189214 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 263/520] 10.222.0.15 () {56 vars in 1111 bytes} [Mon Aug 22 15:47:23 2022] GET /identity/v3/projects/service => generated 88 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 15:47:24.189789 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-978f052e-5e7e-41e8-a2f1-688333873198 None admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 15:47:24.202331 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9c59239-dd0e-45d9-a54b-7f321b7e296a None admin] Validating token access rules against request {{(pid=81767) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 15:47:24.224277 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 15:47:24.243593 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0322d7b-6413-4ebe-a0a6-2dc5e6e69e56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81767) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 15:47:24.244243 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0322d7b-6413-4ebe-a0a6-2dc5e6e69e56 None admin] REQUEST_METHOD: `GET` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:47:24.244648 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0322d7b-6413-4ebe-a0a6-2dc5e6e69e56 None admin] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:47:24.244976 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0322d7b-6413-4ebe-a0a6-2dc5e6e69e56 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:47:24.257197 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0322d7b-6413-4ebe-a0a6-2dc5e6e69e56 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 15:47:24.258808 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0322d7b-6413-4ebe-a0a6-2dc5e6e69e56 None admin] RBAC: Authorization granted {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 15:47:24.275420 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application [None req-f0322d7b-6413-4ebe-a0a6-2dc5e6e69e56 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 22 15:47:24.275420 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 15:47:24.275420 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 15:47:24.275420 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 15:47:24.275420 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 15:47:24.275420 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 15:47:24.275420 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 15:47:24.275420 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 15:47:24.275420 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 15:47:24.275420 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 15:47:24.275420 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 15:47:24.275420 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 15:47:24.275420 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 22 15:47:24.275420 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 22 15:47:24.275420 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 22 15:47:24.275420 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 22 15:47:24.276599 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 15:47:24.276599 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 15:47:24.276599 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 15:47:24.276599 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 15:47:24.276599 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 15:47:24.276599 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 15:47:24.276599 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 15:47:24.276599 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application with Lock( Aug 22 15:47:24.276599 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 15:47:24.276599 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return self._enter() Aug 22 15:47:24.276599 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 15:47:24.276599 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 15:47:24.276599 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 15:47:24.276599 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return self.creator() Aug 22 15:47:24.276599 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 15:47:24.276599 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application created_value = creator( Aug 22 15:47:24.276599 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 22 15:47:24.277785 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 22 15:47:24.277785 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 22 15:47:24.277785 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application Aug 22 15:47:24.278240 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 258/521] 10.222.0.15 () {56 vars in 1108 bytes} [Mon Aug 22 15:47:24 2022] GET /identity/v3/domains/Default => generated 87 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 15:47:24.279159 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0322d7b-6413-4ebe-a0a6-2dc5e6e69e56 None admin] Authenticating user token {{(pid=81767) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 15:47:24.308248 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-978f052e-5e7e-41e8-a2f1-688333873198 None admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 15:47:24.343849 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG dbcounter [-] [81767] Writing DB stats keystone:SELECT=100 {{(pid=81767) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 15:47:24.349123 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1833fe6-b4e3-4dda-9cdf-b4959c9c4c0f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 15:47:24.349757 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1833fe6-b4e3-4dda-9cdf-b4959c9c4c0f None admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:47:24.350034 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1833fe6-b4e3-4dda-9cdf-b4959c9c4c0f None admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:47:24.350302 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1833fe6-b4e3-4dda-9cdf-b4959c9c4c0f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:47:24.362033 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1833fe6-b4e3-4dda-9cdf-b4959c9c4c0f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 15:47:24.363540 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1833fe6-b4e3-4dda-9cdf-b4959c9c4c0f None admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 15:47:24.380204 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application [None req-a1833fe6-b4e3-4dda-9cdf-b4959c9c4c0f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 22 15:47:24.380204 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 15:47:24.380204 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 15:47:24.380204 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 15:47:24.380204 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 15:47:24.380204 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 15:47:24.380204 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 15:47:24.380204 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 15:47:24.380204 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 15:47:24.380204 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 15:47:24.380204 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 15:47:24.380204 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 15:47:24.380204 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 22 15:47:24.380204 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 22 15:47:24.380204 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 22 15:47:24.380204 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 22 15:47:24.381257 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 15:47:24.381257 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 15:47:24.381257 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 15:47:24.381257 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 15:47:24.381257 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 15:47:24.381257 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 15:47:24.381257 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 15:47:24.381257 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application with Lock( Aug 22 15:47:24.381257 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 15:47:24.381257 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self._enter() Aug 22 15:47:24.381257 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 15:47:24.381257 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 15:47:24.381257 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 15:47:24.381257 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self.creator() Aug 22 15:47:24.381257 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 15:47:24.381257 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application created_value = creator( Aug 22 15:47:24.381257 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 22 15:47:24.382107 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 22 15:47:24.382107 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 22 15:47:24.382107 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application Aug 22 15:47:24.382107 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 264/522] 10.222.0.15 () {56 vars in 1108 bytes} [Mon Aug 22 15:47:24 2022] GET /identity/v3/domains/Default => generated 87 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 15:47:24.382332 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1833fe6-b4e3-4dda-9cdf-b4959c9c4c0f None admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 15:47:24.402704 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0322d7b-6413-4ebe-a0a6-2dc5e6e69e56 None admin] Validating token access rules against request {{(pid=81767) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 15:47:24.440629 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0477960-4ff7-4ce6-ae1c-30e7007f9f11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81767) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 15:47:24.441573 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0477960-4ff7-4ce6-ae1c-30e7007f9f11 None admin] REQUEST_METHOD: `GET` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:47:24.441999 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0477960-4ff7-4ce6-ae1c-30e7007f9f11 None admin] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:47:24.442409 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0477960-4ff7-4ce6-ae1c-30e7007f9f11 None admin] PATH_INFO: `/v3/domains` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:47:24.443506 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0477960-4ff7-4ce6-ae1c-30e7007f9f11 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81767) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 15:47:24.444068 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0477960-4ff7-4ce6-ae1c-30e7007f9f11 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 15:47:24.445674 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0477960-4ff7-4ce6-ae1c-30e7007f9f11 None admin] RBAC: Authorization granted {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 15:47:24.460207 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 259/523] 10.222.0.15 () {56 vars in 1109 bytes} [Mon Aug 22 15:47:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 15:47:24.461280 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0477960-4ff7-4ce6-ae1c-30e7007f9f11 None admin] Authenticating user token {{(pid=81767) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 15:47:24.476495 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=100 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 15:47:24.494680 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1833fe6-b4e3-4dda-9cdf-b4959c9c4c0f None admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 15:47:24.532153 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d1e4c9e-247d-4fed-a696-0868aedfff35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 15:47:24.532900 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d1e4c9e-247d-4fed-a696-0868aedfff35 None admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:47:24.533272 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d1e4c9e-247d-4fed-a696-0868aedfff35 None admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:47:24.533642 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d1e4c9e-247d-4fed-a696-0868aedfff35 None admin] PATH_INFO: `/v3/projects` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:47:24.534329 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d1e4c9e-247d-4fed-a696-0868aedfff35 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81768) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 15:47:24.534632 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d1e4c9e-247d-4fed-a696-0868aedfff35 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 15:47:24.536090 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d1e4c9e-247d-4fed-a696-0868aedfff35 None admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 15:47:24.557058 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 265/524] 10.222.0.15 () {56 vars in 1148 bytes} [Mon Aug 22 15:47:24 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 15:47:24.559078 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d1e4c9e-247d-4fed-a696-0868aedfff35 None admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 15:47:24.579361 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0477960-4ff7-4ce6-ae1c-30e7007f9f11 None admin] Validating token access rules against request {{(pid=81767) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 15:47:24.598324 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG dbcounter [-] [81767] Writing DB stats keystone:SELECT=100 {{(pid=81767) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 15:47:24.615515 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa148864-fed1-4801-8bf6-5561dfe5a905 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81767) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 15:47:24.616287 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa148864-fed1-4801-8bf6-5561dfe5a905 None admin] REQUEST_METHOD: `GET` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:47:24.616682 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa148864-fed1-4801-8bf6-5561dfe5a905 None admin] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:47:24.617041 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa148864-fed1-4801-8bf6-5561dfe5a905 None admin] PATH_INFO: `/v3/domains` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:47:24.617805 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa148864-fed1-4801-8bf6-5561dfe5a905 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81767) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 15:47:24.618309 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa148864-fed1-4801-8bf6-5561dfe5a905 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 15:47:24.619906 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa148864-fed1-4801-8bf6-5561dfe5a905 None admin] RBAC: Authorization granted {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 15:47:24.632577 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 260/525] 10.222.0.15 () {56 vars in 1109 bytes} [Mon Aug 22 15:47:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 15:47:24.633965 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa148864-fed1-4801-8bf6-5561dfe5a905 None admin] Authenticating user token {{(pid=81767) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 15:47:24.672476 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d1e4c9e-247d-4fed-a696-0868aedfff35 None admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 15:47:24.710473 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c618584c-9b32-48b8-b8a1-a68c79f0c204 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 15:47:24.711204 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c618584c-9b32-48b8-b8a1-a68c79f0c204 None admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:47:24.711513 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c618584c-9b32-48b8-b8a1-a68c79f0c204 None admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:47:24.711942 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c618584c-9b32-48b8-b8a1-a68c79f0c204 None admin] PATH_INFO: `/v3/domains` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:47:24.713419 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c618584c-9b32-48b8-b8a1-a68c79f0c204 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81768) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 15:47:24.713732 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c618584c-9b32-48b8-b8a1-a68c79f0c204 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 15:47:24.715246 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c618584c-9b32-48b8-b8a1-a68c79f0c204 None admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 15:47:24.727194 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 266/526] 10.222.0.15 () {56 vars in 1109 bytes} [Mon Aug 22 15:47:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 15:47:24.728051 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c618584c-9b32-48b8-b8a1-a68c79f0c204 None admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 15:47:24.734840 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=100 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 15:47:24.751085 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa148864-fed1-4801-8bf6-5561dfe5a905 None admin] Validating token access rules against request {{(pid=81767) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 15:47:24.790500 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5adeb9a-98cc-4f07-9f7b-84267602c374 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81767) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 15:47:24.791295 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5adeb9a-98cc-4f07-9f7b-84267602c374 None admin] REQUEST_METHOD: `GET` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:47:24.791660 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5adeb9a-98cc-4f07-9f7b-84267602c374 None admin] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:47:24.792513 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5adeb9a-98cc-4f07-9f7b-84267602c374 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:47:24.800839 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5adeb9a-98cc-4f07-9f7b-84267602c374 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 15:47:24.802251 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5adeb9a-98cc-4f07-9f7b-84267602c374 None admin] RBAC: Authorization granted {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 15:47:24.815591 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application [None req-f5adeb9a-98cc-4f07-9f7b-84267602c374 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Aug 22 15:47:24.815591 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 15:47:24.815591 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 15:47:24.815591 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 15:47:24.815591 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 15:47:24.815591 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 15:47:24.815591 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 15:47:24.815591 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 15:47:24.815591 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 15:47:24.815591 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 15:47:24.815591 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 15:47:24.815591 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 15:47:24.815591 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 22 15:47:24.815591 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 22 15:47:24.815591 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 22 15:47:24.815591 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 22 15:47:24.816756 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 15:47:24.816756 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 15:47:24.816756 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 22 15:47:24.816756 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 22 15:47:24.816756 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 22 15:47:24.816756 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 22 15:47:24.816756 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 15:47:24.816756 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 15:47:24.816756 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 15:47:24.816756 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 15:47:24.816756 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 15:47:24.816756 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application with Lock( Aug 22 15:47:24.816756 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 15:47:24.816756 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return self._enter() Aug 22 15:47:24.816756 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 15:47:24.816756 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 15:47:24.816756 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 15:47:24.816756 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return self.creator() Aug 22 15:47:24.817960 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 15:47:24.817960 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application created_value = creator( Aug 22 15:47:24.817960 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 22 15:47:24.817960 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 22 15:47:24.817960 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 22 15:47:24.817960 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 22 15:47:24.817960 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 22 15:47:24.817960 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 22 15:47:24.817960 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Aug 22 15:47:24.817960 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application Aug 22 15:47:24.817960 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 261/527] 10.222.0.15 () {56 vars in 1102 bytes} [Mon Aug 22 15:47:24 2022] GET /identity/v3/users/neutron => generated 85 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 15:47:24.820473 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5adeb9a-98cc-4f07-9f7b-84267602c374 None admin] Authenticating user token {{(pid=81767) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 15:47:24.844350 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c618584c-9b32-48b8-b8a1-a68c79f0c204 None admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 15:47:24.863513 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG dbcounter [-] [81767] Writing DB stats keystone:SELECT=100 {{(pid=81767) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 15:47:24.882826 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9894bdca-591b-48a8-8469-fe0265c8bcbe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 15:47:24.883457 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9894bdca-591b-48a8-8469-fe0265c8bcbe None admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:47:24.883735 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9894bdca-591b-48a8-8469-fe0265c8bcbe None admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:47:24.884068 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9894bdca-591b-48a8-8469-fe0265c8bcbe None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:47:24.884792 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9894bdca-591b-48a8-8469-fe0265c8bcbe None admin] RBAC: Adding query filter params (role.id=a5fbba1dc615490a96cde27e3a102eb6, scope.project.id=c4ee0413f9ef4d4fb1f95a660babe090, user.id=e4ef569e41fb46d0b721bb1c7663616f) {{(pid=81768) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 15:47:24.885093 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9894bdca-591b-48a8-8469-fe0265c8bcbe None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 15:47:24.886566 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9894bdca-591b-48a8-8469-fe0265c8bcbe None admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 15:47:24.892174 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 267/528] 10.222.0.15 () {56 vars in 1374 bytes} [Mon Aug 22 15:47:24 2022] GET /identity/v3/role_assignments?user.id=e4ef569e41fb46d0b721bb1c7663616f&scope.project.id=c4ee0413f9ef4d4fb1f95a660babe090&role.id=a5fbba1dc615490a96cde27e3a102eb6 => generated 259 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 15:47:24.892961 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9894bdca-591b-48a8-8469-fe0265c8bcbe None admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 15:47:24.965783 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5adeb9a-98cc-4f07-9f7b-84267602c374 None admin] Validating token access rules against request {{(pid=81767) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 15:47:24.988311 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=100 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 15:47:25.019789 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e24d9ae2-77e2-43b6-816d-090480c1fe55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81767) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 15:47:25.020711 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e24d9ae2-77e2-43b6-816d-090480c1fe55 None admin] REQUEST_METHOD: `GET` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:47:25.021363 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e24d9ae2-77e2-43b6-816d-090480c1fe55 None admin] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:47:25.023123 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e24d9ae2-77e2-43b6-816d-090480c1fe55 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:47:25.035257 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e24d9ae2-77e2-43b6-816d-090480c1fe55 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 15:47:25.037230 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e24d9ae2-77e2-43b6-816d-090480c1fe55 None admin] RBAC: Authorization granted {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 15:47:25.037819 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9894bdca-591b-48a8-8469-fe0265c8bcbe None admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 15:47:25.054536 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application [None req-e24d9ae2-77e2-43b6-816d-090480c1fe55 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 22 15:47:25.054536 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 15:47:25.054536 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 15:47:25.054536 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 15:47:25.054536 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 15:47:25.054536 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 15:47:25.054536 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 15:47:25.054536 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 15:47:25.054536 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 15:47:25.054536 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 15:47:25.054536 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 15:47:25.054536 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 15:47:25.054536 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 22 15:47:25.054536 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 22 15:47:25.054536 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 22 15:47:25.055524 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 22 15:47:25.055524 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 15:47:25.055524 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 15:47:25.055524 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 15:47:25.055524 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 15:47:25.055524 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 15:47:25.055524 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 15:47:25.055524 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 15:47:25.055524 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application with Lock( Aug 22 15:47:25.055524 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 15:47:25.055524 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return self._enter() Aug 22 15:47:25.055524 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 15:47:25.055524 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 15:47:25.055524 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 15:47:25.055524 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return self.creator() Aug 22 15:47:25.055524 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 15:47:25.055524 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application created_value = creator( Aug 22 15:47:25.056355 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 22 15:47:25.056355 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 22 15:47:25.056355 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 22 15:47:25.056355 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 22 15:47:25.056355 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 22 15:47:25.056355 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 22 15:47:25.056355 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 22 15:47:25.056355 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application Aug 22 15:47:25.062466 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 262/529] 10.222.0.15 () {56 vars in 1111 bytes} [Mon Aug 22 15:47:24 2022] GET /identity/v3/projects/service => generated 88 bytes in 238 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 15:47:25.062466 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e24d9ae2-77e2-43b6-816d-090480c1fe55 None admin] Authenticating user token {{(pid=81767) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 15:47:25.079249 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff3afc1a-d048-4218-ba85-bcb7b6b10267 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 15:47:25.080246 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff3afc1a-d048-4218-ba85-bcb7b6b10267 None admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:47:25.080738 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff3afc1a-d048-4218-ba85-bcb7b6b10267 None admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:47:25.081248 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff3afc1a-d048-4218-ba85-bcb7b6b10267 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:47:25.088569 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff3afc1a-d048-4218-ba85-bcb7b6b10267 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 15:47:25.090988 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff3afc1a-d048-4218-ba85-bcb7b6b10267 None admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 15:47:25.103615 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application [None req-ff3afc1a-d048-4218-ba85-bcb7b6b10267 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 22 15:47:25.103615 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 15:47:25.103615 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 15:47:25.103615 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 15:47:25.103615 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 15:47:25.103615 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 15:47:25.103615 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 15:47:25.103615 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 15:47:25.103615 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 15:47:25.103615 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 15:47:25.103615 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 15:47:25.103615 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 15:47:25.103615 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 22 15:47:25.103615 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 22 15:47:25.103615 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 22 15:47:25.105097 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 22 15:47:25.105097 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 15:47:25.105097 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 15:47:25.105097 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 15:47:25.105097 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 15:47:25.105097 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 15:47:25.105097 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 15:47:25.105097 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 15:47:25.105097 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application with Lock( Aug 22 15:47:25.105097 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 15:47:25.105097 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self._enter() Aug 22 15:47:25.105097 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 15:47:25.105097 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 15:47:25.105097 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 15:47:25.105097 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self.creator() Aug 22 15:47:25.105097 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 15:47:25.105097 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application created_value = creator( Aug 22 15:47:25.105904 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 22 15:47:25.105904 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 22 15:47:25.105904 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 22 15:47:25.105904 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 22 15:47:25.105904 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 22 15:47:25.105904 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 22 15:47:25.105904 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 22 15:47:25.105904 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application Aug 22 15:47:25.112985 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 268/530] 10.222.0.15 () {56 vars in 1111 bytes} [Mon Aug 22 15:47:24 2022] GET /identity/v3/projects/service => generated 88 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 15:47:25.114159 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff3afc1a-d048-4218-ba85-bcb7b6b10267 None admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 15:47:25.178013 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG dbcounter [-] [81767] Writing DB stats keystone:SELECT=100 {{(pid=81767) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 15:47:25.182347 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e24d9ae2-77e2-43b6-816d-090480c1fe55 None admin] Validating token access rules against request {{(pid=81767) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 15:47:25.222151 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aed7a1c0-01c0-4163-aa9b-e4380b8ad1ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81767) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 15:47:25.223145 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aed7a1c0-01c0-4163-aa9b-e4380b8ad1ca None admin] REQUEST_METHOD: `GET` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:47:25.223692 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aed7a1c0-01c0-4163-aa9b-e4380b8ad1ca None admin] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:47:25.224150 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aed7a1c0-01c0-4163-aa9b-e4380b8ad1ca None admin] PATH_INFO: `/v3/users` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:47:25.225070 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aed7a1c0-01c0-4163-aa9b-e4380b8ad1ca None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=81767) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 15:47:25.225605 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aed7a1c0-01c0-4163-aa9b-e4380b8ad1ca None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 15:47:25.227411 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aed7a1c0-01c0-4163-aa9b-e4380b8ad1ca None admin] RBAC: Authorization granted {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 15:47:25.246855 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff3afc1a-d048-4218-ba85-bcb7b6b10267 None admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 15:47:25.247624 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 263/531] 10.222.0.15 () {56 vars in 1139 bytes} [Mon Aug 22 15:47:25 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 15:47:25.248816 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aed7a1c0-01c0-4163-aa9b-e4380b8ad1ca None admin] Authenticating user token {{(pid=81767) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 15:47:25.283130 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=100 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 15:47:25.294475 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea36b7e4-51cd-41fa-b5e6-634a39d285b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 15:47:25.295096 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea36b7e4-51cd-41fa-b5e6-634a39d285b8 None admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:47:25.295391 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea36b7e4-51cd-41fa-b5e6-634a39d285b8 None admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:47:25.295663 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea36b7e4-51cd-41fa-b5e6-634a39d285b8 None admin] PATH_INFO: `/v3/projects` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:47:25.296357 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea36b7e4-51cd-41fa-b5e6-634a39d285b8 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81768) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 15:47:25.296660 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea36b7e4-51cd-41fa-b5e6-634a39d285b8 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 15:47:25.298666 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea36b7e4-51cd-41fa-b5e6-634a39d285b8 None admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 15:47:25.319471 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 269/532] 10.222.0.15 () {56 vars in 1148 bytes} [Mon Aug 22 15:47:25 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 15:47:25.319471 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea36b7e4-51cd-41fa-b5e6-634a39d285b8 None admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 15:47:25.379165 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aed7a1c0-01c0-4163-aa9b-e4380b8ad1ca None admin] Validating token access rules against request {{(pid=81767) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 15:47:25.414940 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66515507-fd37-4a13-9bb1-a695a37478b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81767) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 15:47:25.416060 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66515507-fd37-4a13-9bb1-a695a37478b2 None admin] REQUEST_METHOD: `GET` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:47:25.416060 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66515507-fd37-4a13-9bb1-a695a37478b2 None admin] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:47:25.416438 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66515507-fd37-4a13-9bb1-a695a37478b2 None admin] PATH_INFO: `/v3/projects` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:47:25.417824 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66515507-fd37-4a13-9bb1-a695a37478b2 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81767) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 15:47:25.418277 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66515507-fd37-4a13-9bb1-a695a37478b2 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 15:47:25.419965 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66515507-fd37-4a13-9bb1-a695a37478b2 None admin] RBAC: Authorization granted {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 15:47:25.440092 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 264/533] 10.222.0.15 () {56 vars in 1148 bytes} [Mon Aug 22 15:47:25 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 15:47:25.441328 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66515507-fd37-4a13-9bb1-a695a37478b2 None admin] Authenticating user token {{(pid=81767) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 15:47:25.451072 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea36b7e4-51cd-41fa-b5e6-634a39d285b8 None admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 15:47:25.468941 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG dbcounter [-] [81767] Writing DB stats keystone:SELECT=100 {{(pid=81767) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 15:47:25.491489 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fffdbe29-700d-4660-81de-59a6ab311516 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 15:47:25.492326 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fffdbe29-700d-4660-81de-59a6ab311516 None admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:47:25.492749 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fffdbe29-700d-4660-81de-59a6ab311516 None admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:47:25.493229 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fffdbe29-700d-4660-81de-59a6ab311516 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:47:25.556598 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=100 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 15:47:25.560607 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66515507-fd37-4a13-9bb1-a695a37478b2 None admin] Validating token access rules against request {{(pid=81767) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 15:47:25.571898 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fffdbe29-700d-4660-81de-59a6ab311516 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 15:47:25.574297 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fffdbe29-700d-4660-81de-59a6ab311516 None admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 15:47:25.596817 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54fc7826-fbb5-4b36-b6e4-3ec3e58ddf01 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81767) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 15:47:25.597540 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54fc7826-fbb5-4b36-b6e4-3ec3e58ddf01 None admin] REQUEST_METHOD: `GET` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:47:25.597876 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54fc7826-fbb5-4b36-b6e4-3ec3e58ddf01 None admin] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:47:25.598219 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54fc7826-fbb5-4b36-b6e4-3ec3e58ddf01 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:47:25.599074 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54fc7826-fbb5-4b36-b6e4-3ec3e58ddf01 None admin] RBAC: Adding query filter params (role.id=a5fbba1dc615490a96cde27e3a102eb6, scope.project.id=c4ee0413f9ef4d4fb1f95a660babe090, user.id=79706d480f864998b9f8594be407ceab) {{(pid=81767) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 15:47:25.599461 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54fc7826-fbb5-4b36-b6e4-3ec3e58ddf01 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 15:47:25.600930 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54fc7826-fbb5-4b36-b6e4-3ec3e58ddf01 None admin] RBAC: Authorization granted {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 15:47:25.608504 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 265/534] 10.222.0.15 () {56 vars in 1374 bytes} [Mon Aug 22 15:47:25 2022] GET /identity/v3/role_assignments?user.id=79706d480f864998b9f8594be407ceab&scope.project.id=c4ee0413f9ef4d4fb1f95a660babe090&role.id=a5fbba1dc615490a96cde27e3a102eb6 => generated 259 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 15:47:25.609513 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54fc7826-fbb5-4b36-b6e4-3ec3e58ddf01 None admin] Authenticating user token {{(pid=81767) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 15:47:25.693632 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 270/535] 10.222.0.15 () {58 vars in 1282 bytes} [Mon Aug 22 15:47:25 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 376 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 22 15:47:25.701577 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fffdbe29-700d-4660-81de-59a6ab311516 None admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 15:47:25.717888 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG dbcounter [-] [81767] Writing DB stats keystone:SELECT=100 {{(pid=81767) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 15:47:25.735339 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54fc7826-fbb5-4b36-b6e4-3ec3e58ddf01 None admin] Validating token access rules against request {{(pid=81767) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 15:47:25.780522 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf0db6b5-759c-4aeb-a098-c88ec286894c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81767) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 15:47:25.781144 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf0db6b5-759c-4aeb-a098-c88ec286894c None admin] REQUEST_METHOD: `GET` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:47:25.781416 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf0db6b5-759c-4aeb-a098-c88ec286894c None admin] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:47:25.781682 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf0db6b5-759c-4aeb-a098-c88ec286894c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:47:25.782478 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf0db6b5-759c-4aeb-a098-c88ec286894c None admin] RBAC: Adding query filter params (role.id=a5fbba1dc615490a96cde27e3a102eb6, scope.project.id=c4ee0413f9ef4d4fb1f95a660babe090, user.id=91f565473c824a15930e2b12863c1ed3) {{(pid=81767) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 15:47:25.782783 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf0db6b5-759c-4aeb-a098-c88ec286894c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 15:47:25.786375 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf0db6b5-759c-4aeb-a098-c88ec286894c None admin] RBAC: Authorization granted {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 15:47:25.798261 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 266/536] 10.222.0.15 () {56 vars in 1374 bytes} [Mon Aug 22 15:47:25 2022] GET /identity/v3/role_assignments?user.id=91f565473c824a15930e2b12863c1ed3&scope.project.id=c4ee0413f9ef4d4fb1f95a660babe090&role.id=a5fbba1dc615490a96cde27e3a102eb6 => generated 259 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 15:47:25.877456 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=100 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 15:47:25.888150 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fffdbe29-700d-4660-81de-59a6ab311516 None admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 15:47:25.931299 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0635175-a9f7-46c3-b229-e26ee2c1e5cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 15:47:25.932203 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0635175-a9f7-46c3-b229-e26ee2c1e5cc None admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:47:25.932980 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0635175-a9f7-46c3-b229-e26ee2c1e5cc None admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:47:25.933254 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0635175-a9f7-46c3-b229-e26ee2c1e5cc None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:47:25.951473 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0635175-a9f7-46c3-b229-e26ee2c1e5cc None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 15:47:25.953034 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0635175-a9f7-46c3-b229-e26ee2c1e5cc None admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 15:47:25.972312 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application [None req-c0635175-a9f7-46c3-b229-e26ee2c1e5cc None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 22 15:47:25.972312 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 15:47:25.972312 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 15:47:25.972312 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 15:47:25.972312 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 15:47:25.972312 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 15:47:25.972312 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 15:47:25.972312 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 15:47:25.972312 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 15:47:25.972312 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 15:47:25.972312 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 15:47:25.972312 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 15:47:25.972312 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 22 15:47:25.972312 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 22 15:47:25.972312 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 22 15:47:25.972312 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 22 15:47:25.973461 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 15:47:25.973461 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 15:47:25.973461 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 15:47:25.973461 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 15:47:25.973461 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 15:47:25.973461 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 15:47:25.973461 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 15:47:25.973461 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application with Lock( Aug 22 15:47:25.973461 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 15:47:25.973461 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self._enter() Aug 22 15:47:25.973461 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 15:47:25.973461 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 15:47:25.973461 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 15:47:25.973461 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self.creator() Aug 22 15:47:25.973461 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 15:47:25.973461 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application created_value = creator( Aug 22 15:47:25.973461 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 22 15:47:25.974761 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 22 15:47:25.974761 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 22 15:47:25.974761 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application Aug 22 15:47:25.974761 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 271/537] 10.222.0.15 () {56 vars in 1108 bytes} [Mon Aug 22 15:47:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 272 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 15:47:25.978978 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf0db6b5-759c-4aeb-a098-c88ec286894c None admin] Authenticating user token {{(pid=81767) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 15:47:26.049152 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d589a8a8-68ce-46da-8ab0-a8d5ff533b7a None None] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:47:26.049524 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d589a8a8-68ce-46da-8ab0-a8d5ff533b7a None None] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:47:26.049889 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d589a8a8-68ce-46da-8ab0-a8d5ff533b7a None None] PATH_INFO: `/` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:47:26.051448 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 272/538] 10.222.0.15 () {54 vars in 927 bytes} [Mon Aug 22 15:47:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 22 15:47:26.057798 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d78fc24d-d18a-4ade-861c-ed33195ff13c None None] REQUEST_METHOD: `POST` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:47:26.058324 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d78fc24d-d18a-4ade-861c-ed33195ff13c None None] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:47:26.058776 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d78fc24d-d18a-4ade-861c-ed33195ff13c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:47:26.078744 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.auth.core [None req-d78fc24d-d18a-4ade-861c-ed33195ff13c None None] MFA Rules not processed for user `6b2199ae0fb0480aa917ada391f5ec54`. Rule list: `[]` (Enabled: `True`). {{(pid=81768) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 15:47:26.080110 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-d78fc24d-d18a-4ade-861c-ed33195ff13c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 15:47:26.118256 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf0db6b5-759c-4aeb-a098-c88ec286894c None admin] Validating token access rules against request {{(pid=81767) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 15:47:26.181404 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 273/539] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Aug 22 15:47:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 22 15:47:26.192708 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10feb4c4-92d6-4ff8-81fa-46951013ad96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81767) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 15:47:26.193641 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10feb4c4-92d6-4ff8-81fa-46951013ad96 None admin] REQUEST_METHOD: `GET` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:47:26.194093 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10feb4c4-92d6-4ff8-81fa-46951013ad96 None admin] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:47:26.194668 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10feb4c4-92d6-4ff8-81fa-46951013ad96 None admin] PATH_INFO: `/v3/domains` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:47:26.195553 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10feb4c4-92d6-4ff8-81fa-46951013ad96 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81767) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 15:47:26.195999 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10feb4c4-92d6-4ff8-81fa-46951013ad96 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 15:47:26.197637 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10feb4c4-92d6-4ff8-81fa-46951013ad96 None admin] RBAC: Authorization granted {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 15:47:26.217133 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 267/540] 10.222.0.15 () {56 vars in 1109 bytes} [Mon Aug 22 15:47:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 239 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 22 15:47:26.222314 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d78fc24d-d18a-4ade-861c-ed33195ff13c None None] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 15:47:26.307387 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=100 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 15:47:26.340742 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d78fc24d-d18a-4ade-861c-ed33195ff13c None None] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 15:47:26.376338 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5a889e5-f891-479f-8ac6-7d0f1aacc18f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 15:47:26.376969 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5a889e5-f891-479f-8ac6-7d0f1aacc18f None admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:47:26.377247 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5a889e5-f891-479f-8ac6-7d0f1aacc18f None admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:47:26.377515 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5a889e5-f891-479f-8ac6-7d0f1aacc18f None admin] PATH_INFO: `/v3/projects/service` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:47:26.384477 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5a889e5-f891-479f-8ac6-7d0f1aacc18f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 15:47:26.386451 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5a889e5-f891-479f-8ac6-7d0f1aacc18f None admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 15:47:26.398134 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application [None req-e5a889e5-f891-479f-8ac6-7d0f1aacc18f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 22 15:47:26.398134 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 15:47:26.398134 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 15:47:26.398134 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 15:47:26.398134 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 15:47:26.398134 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 15:47:26.398134 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 15:47:26.398134 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 15:47:26.398134 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 15:47:26.398134 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 15:47:26.398134 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 15:47:26.398134 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 15:47:26.398134 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 22 15:47:26.398134 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 22 15:47:26.398134 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 22 15:47:26.399064 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 22 15:47:26.399064 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 15:47:26.399064 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 15:47:26.399064 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 15:47:26.399064 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 15:47:26.399064 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 15:47:26.399064 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 15:47:26.399064 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 15:47:26.399064 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application with Lock( Aug 22 15:47:26.399064 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 15:47:26.399064 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self._enter() Aug 22 15:47:26.399064 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 15:47:26.399064 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 15:47:26.399064 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 15:47:26.399064 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self.creator() Aug 22 15:47:26.399064 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 15:47:26.399064 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application created_value = creator( Aug 22 15:47:26.399930 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 22 15:47:26.399930 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 22 15:47:26.399930 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 22 15:47:26.399930 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 22 15:47:26.399930 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 22 15:47:26.399930 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 22 15:47:26.399930 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 22 15:47:26.399930 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application Aug 22 15:47:26.399930 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 274/541] 10.222.0.15 () {56 vars in 1111 bytes} [Mon Aug 22 15:47:26 2022] GET /identity/v3/projects/service => generated 88 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 15:47:26.405564 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10feb4c4-92d6-4ff8-81fa-46951013ad96 None admin] Authenticating user token {{(pid=81767) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 15:47:26.415664 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG dbcounter [-] [81767] Writing DB stats keystone:SELECT=100 {{(pid=81767) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 15:47:26.570216 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10feb4c4-92d6-4ff8-81fa-46951013ad96 None admin] Validating token access rules against request {{(pid=81767) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 15:47:26.615179 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de57a26a-3f42-4291-b2af-dd13afdf7837 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81767) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 15:47:26.616064 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de57a26a-3f42-4291-b2af-dd13afdf7837 None admin] REQUEST_METHOD: `GET` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:47:26.616659 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de57a26a-3f42-4291-b2af-dd13afdf7837 None admin] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:47:26.617088 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de57a26a-3f42-4291-b2af-dd13afdf7837 None admin] PATH_INFO: `/v3/projects` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:47:26.617920 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de57a26a-3f42-4291-b2af-dd13afdf7837 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81767) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 15:47:26.618365 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de57a26a-3f42-4291-b2af-dd13afdf7837 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 15:47:26.620522 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de57a26a-3f42-4291-b2af-dd13afdf7837 None admin] RBAC: Authorization granted {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 15:47:26.638671 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 268/542] 10.222.0.15 () {56 vars in 1148 bytes} [Mon Aug 22 15:47:26 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 233 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 15:47:26.648350 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5a889e5-f891-479f-8ac6-7d0f1aacc18f None admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 15:47:26.726132 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de57a26a-3f42-4291-b2af-dd13afdf7837 None admin] Authenticating user token {{(pid=81767) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 15:47:26.754732 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5a889e5-f891-479f-8ac6-7d0f1aacc18f None admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 15:47:26.809736 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=100 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 15:47:26.815586 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcbcb7e2-ce8f-48a9-a300-88d3b5f47d6e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 15:47:26.815586 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcbcb7e2-ce8f-48a9-a300-88d3b5f47d6e None admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:47:26.815586 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcbcb7e2-ce8f-48a9-a300-88d3b5f47d6e None admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:47:26.815586 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcbcb7e2-ce8f-48a9-a300-88d3b5f47d6e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:47:26.815986 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcbcb7e2-ce8f-48a9-a300-88d3b5f47d6e None admin] RBAC: Adding query filter params (role.id=a5fbba1dc615490a96cde27e3a102eb6, scope.project.id=c4ee0413f9ef4d4fb1f95a660babe090, user.id=234687adad594f1aa45775a565ffbdc7) {{(pid=81768) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 15:47:26.815986 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcbcb7e2-ce8f-48a9-a300-88d3b5f47d6e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 15:47:26.825499 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcbcb7e2-ce8f-48a9-a300-88d3b5f47d6e None admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 15:47:26.832074 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 275/543] 10.222.0.15 () {56 vars in 1374 bytes} [Mon Aug 22 15:47:26 2022] GET /identity/v3/role_assignments?user.id=234687adad594f1aa45775a565ffbdc7&scope.project.id=c4ee0413f9ef4d4fb1f95a660babe090&role.id=a5fbba1dc615490a96cde27e3a102eb6 => generated 259 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 15:47:26.855388 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG dbcounter [-] [81767] Writing DB stats keystone:SELECT=100 {{(pid=81767) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 15:47:26.911917 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de57a26a-3f42-4291-b2af-dd13afdf7837 None admin] Validating token access rules against request {{(pid=81767) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 15:47:26.957601 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc7a355e-45b1-47e5-bdb5-f97da6382742 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81767) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 15:47:26.958404 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc7a355e-45b1-47e5-bdb5-f97da6382742 None admin] REQUEST_METHOD: `GET` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:47:26.958795 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc7a355e-45b1-47e5-bdb5-f97da6382742 None admin] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:47:26.959171 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc7a355e-45b1-47e5-bdb5-f97da6382742 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:47:26.971155 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc7a355e-45b1-47e5-bdb5-f97da6382742 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 22 15:47:26.976577 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application [None req-fc7a355e-45b1-47e5-bdb5-f97da6382742 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 22 15:47:26.976577 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 15:47:26.976577 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 15:47:26.976577 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 15:47:26.976577 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 15:47:26.976577 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 15:47:26.976577 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 15:47:26.976577 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 15:47:26.976577 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 15:47:26.976577 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 15:47:26.976577 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 15:47:26.976577 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 15:47:26.976577 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 22 15:47:26.976577 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 22 15:47:26.976577 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 22 15:47:26.976577 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 22 15:47:26.977684 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 22 15:47:26.977684 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 22 15:47:26.977684 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 22 15:47:26.977684 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 22 15:47:26.977684 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 15:47:26.977684 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 15:47:26.977684 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 15:47:26.977684 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 15:47:26.977684 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 15:47:26.977684 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 15:47:26.977684 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 15:47:26.977684 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application with Lock( Aug 22 15:47:26.977684 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 15:47:26.977684 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return self._enter() Aug 22 15:47:26.977684 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 15:47:26.977684 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 15:47:26.978639 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 15:47:26.978639 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return self.creator() Aug 22 15:47:26.978639 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 15:47:26.978639 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application created_value = creator( Aug 22 15:47:26.978639 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 22 15:47:26.978639 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 22 15:47:26.978639 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 22 15:47:26.978639 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 22 15:47:26.978639 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 22 15:47:26.978639 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 22 15:47:26.978639 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 22 15:47:26.978639 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application Aug 22 15:47:26.983314 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 269/544] 10.222.0.15 () {56 vars in 1102 bytes} [Mon Aug 22 15:47:26 2022] GET /identity/v3/roles/service => generated 85 bytes in 255 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 22 15:47:26.993509 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcbcb7e2-ce8f-48a9-a300-88d3b5f47d6e None admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 15:47:27.116407 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcbcb7e2-ce8f-48a9-a300-88d3b5f47d6e None admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 15:47:27.156192 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-296e4e86-d144-405a-94c9-6a34187bee17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 15:47:27.156960 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-296e4e86-d144-405a-94c9-6a34187bee17 None admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:47:27.157338 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-296e4e86-d144-405a-94c9-6a34187bee17 None admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:47:27.157778 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-296e4e86-d144-405a-94c9-6a34187bee17 None admin] PATH_INFO: `/v3/roles` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:47:27.158544 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-296e4e86-d144-405a-94c9-6a34187bee17 None admin] RBAC: Adding query filter params (name=service) {{(pid=81768) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 15:47:27.158989 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-296e4e86-d144-405a-94c9-6a34187bee17 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 15:47:27.160509 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-296e4e86-d144-405a-94c9-6a34187bee17 None admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 15:47:27.169829 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 276/545] 10.222.0.15 () {56 vars in 1103 bytes} [Mon Aug 22 15:47:26 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 15:47:27.175155 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc7a355e-45b1-47e5-bdb5-f97da6382742 None admin] Authenticating user token {{(pid=81767) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 15:47:27.277685 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9ab2112-3044-4e13-8160-adcc9020636e None None] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:47:27.277966 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9ab2112-3044-4e13-8160-adcc9020636e None None] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:47:27.278239 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9ab2112-3044-4e13-8160-adcc9020636e None None] PATH_INFO: `/` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:47:27.279856 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 277/546] 10.222.0.15 () {54 vars in 927 bytes} [Mon Aug 22 15:47:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 22 15:47:27.285903 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80fea2e8-1231-4f30-b18e-c7b1e8236109 None None] REQUEST_METHOD: `POST` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:47:27.286181 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80fea2e8-1231-4f30-b18e-c7b1e8236109 None None] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:47:27.286452 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80fea2e8-1231-4f30-b18e-c7b1e8236109 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:47:27.315911 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.auth.core [None req-80fea2e8-1231-4f30-b18e-c7b1e8236109 None None] MFA Rules not processed for user `6b2199ae0fb0480aa917ada391f5ec54`. Rule list: `[]` (Enabled: `True`). {{(pid=81768) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 15:47:27.317926 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-80fea2e8-1231-4f30-b18e-c7b1e8236109 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 15:47:27.333770 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc7a355e-45b1-47e5-bdb5-f97da6382742 None admin] Validating token access rules against request {{(pid=81767) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 15:47:27.381892 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG dbcounter [-] [81767] Writing DB stats keystone:SELECT=100 {{(pid=81767) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 15:47:27.404583 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0f611dd-95d7-480e-82a8-08ad92ee7439 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81767) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 15:47:27.405232 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0f611dd-95d7-480e-82a8-08ad92ee7439 None admin] REQUEST_METHOD: `GET` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:47:27.405507 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0f611dd-95d7-480e-82a8-08ad92ee7439 None admin] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:47:27.405776 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0f611dd-95d7-480e-82a8-08ad92ee7439 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:47:27.425915 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=100 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 15:47:27.435299 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0f611dd-95d7-480e-82a8-08ad92ee7439 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 15:47:27.435299 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0f611dd-95d7-480e-82a8-08ad92ee7439 None admin] RBAC: Authorization granted {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 15:47:27.462916 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application [None req-b0f611dd-95d7-480e-82a8-08ad92ee7439 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 22 15:47:27.462916 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 15:47:27.462916 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 15:47:27.462916 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 15:47:27.462916 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 15:47:27.462916 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 15:47:27.462916 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 15:47:27.462916 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 15:47:27.462916 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 15:47:27.462916 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 15:47:27.462916 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 15:47:27.462916 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 15:47:27.462916 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 22 15:47:27.462916 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 22 15:47:27.462916 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 22 15:47:27.462916 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 22 15:47:27.464600 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 15:47:27.464600 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 15:47:27.464600 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 15:47:27.464600 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 15:47:27.464600 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 15:47:27.464600 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 15:47:27.464600 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 15:47:27.464600 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application with Lock( Aug 22 15:47:27.464600 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 15:47:27.464600 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return self._enter() Aug 22 15:47:27.464600 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 15:47:27.464600 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 15:47:27.464600 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 15:47:27.464600 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return self.creator() Aug 22 15:47:27.464600 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 15:47:27.464600 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application created_value = creator( Aug 22 15:47:27.464600 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 22 15:47:27.465924 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 22 15:47:27.465924 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 22 15:47:27.465924 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application Aug 22 15:47:27.467883 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 270/547] 10.222.0.15 () {56 vars in 1108 bytes} [Mon Aug 22 15:47:27 2022] GET /identity/v3/domains/Default => generated 87 bytes in 293 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 15:47:27.469995 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 278/548] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Aug 22 15:47:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 22 15:47:27.476325 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0f611dd-95d7-480e-82a8-08ad92ee7439 None admin] Authenticating user token {{(pid=81767) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 15:47:27.631491 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0f611dd-95d7-480e-82a8-08ad92ee7439 None admin] Validating token access rules against request {{(pid=81767) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 15:47:27.684756 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbf3d09d-5ab8-41f1-96bf-194dfd9b5fdc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81767) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 15:47:27.685284 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf3d09d-5ab8-41f1-96bf-194dfd9b5fdc None admin] REQUEST_METHOD: `GET` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:47:27.685566 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf3d09d-5ab8-41f1-96bf-194dfd9b5fdc None admin] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:47:27.685833 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf3d09d-5ab8-41f1-96bf-194dfd9b5fdc None admin] PATH_INFO: `/v3/domains` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:47:27.686488 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbf3d09d-5ab8-41f1-96bf-194dfd9b5fdc None admin] RBAC: Adding query filter params (name=Default) {{(pid=81767) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 15:47:27.686788 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbf3d09d-5ab8-41f1-96bf-194dfd9b5fdc None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 15:47:27.688193 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbf3d09d-5ab8-41f1-96bf-194dfd9b5fdc None admin] RBAC: Authorization granted {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 15:47:27.705799 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 271/549] 10.222.0.15 () {56 vars in 1109 bytes} [Mon Aug 22 15:47:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 15:47:27.711000 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80fea2e8-1231-4f30-b18e-c7b1e8236109 None None] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 15:47:27.799429 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbf3d09d-5ab8-41f1-96bf-194dfd9b5fdc None admin] Authenticating user token {{(pid=81767) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 15:47:27.839943 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG dbcounter [-] [81767] Writing DB stats keystone:SELECT=100 {{(pid=81767) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 15:47:27.915368 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbf3d09d-5ab8-41f1-96bf-194dfd9b5fdc None admin] Validating token access rules against request {{(pid=81767) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 15:47:27.931553 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80fea2e8-1231-4f30-b18e-c7b1e8236109 None None] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 15:47:27.955609 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42dd410e-200b-4d96-a21f-63e4e81c7466 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81767) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 15:47:27.956161 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42dd410e-200b-4d96-a21f-63e4e81c7466 None admin] REQUEST_METHOD: `GET` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:47:27.956432 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42dd410e-200b-4d96-a21f-63e4e81c7466 None admin] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:47:27.956699 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42dd410e-200b-4d96-a21f-63e4e81c7466 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:47:27.968454 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42dd410e-200b-4d96-a21f-63e4e81c7466 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 22 15:47:27.973606 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application [None req-42dd410e-200b-4d96-a21f-63e4e81c7466 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 22 15:47:27.973606 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 15:47:27.973606 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 15:47:27.973606 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 15:47:27.973606 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 15:47:27.973606 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 15:47:27.973606 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 15:47:27.973606 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 15:47:27.973606 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 15:47:27.973606 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 15:47:27.973606 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 15:47:27.973606 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 15:47:27.973606 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 22 15:47:27.973606 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 22 15:47:27.973606 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 22 15:47:27.973606 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 22 15:47:27.974491 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 22 15:47:27.974491 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 22 15:47:27.974491 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 22 15:47:27.974491 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 22 15:47:27.974491 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 15:47:27.974491 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 15:47:27.974491 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 15:47:27.974491 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 15:47:27.974491 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 15:47:27.974491 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 15:47:27.974491 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 15:47:27.974491 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application with Lock( Aug 22 15:47:27.974491 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 15:47:27.974491 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return self._enter() Aug 22 15:47:27.974491 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 15:47:27.974491 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 15:47:27.975322 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 15:47:27.975322 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return self.creator() Aug 22 15:47:27.975322 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 15:47:27.975322 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application created_value = creator( Aug 22 15:47:27.975322 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 22 15:47:27.975322 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 22 15:47:27.975322 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 22 15:47:27.975322 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 22 15:47:27.975322 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 22 15:47:27.975322 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 22 15:47:27.975322 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 22 15:47:27.975322 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application Aug 22 15:47:27.975322 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 272/550] 10.222.0.15 () {56 vars in 1102 bytes} [Mon Aug 22 15:47:27 2022] GET /identity/v3/roles/service => generated 85 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 22 15:47:27.980766 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42dd410e-200b-4d96-a21f-63e4e81c7466 None admin] Authenticating user token {{(pid=81767) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 15:47:28.048029 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-686236a8-b489-41b5-8d77-d735d5f776b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 15:47:28.048683 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-686236a8-b489-41b5-8d77-d735d5f776b6 None admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:47:28.048961 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-686236a8-b489-41b5-8d77-d735d5f776b6 None admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:47:28.049226 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-686236a8-b489-41b5-8d77-d735d5f776b6 None admin] PATH_INFO: `/v3/users/swift` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:47:28.063229 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-686236a8-b489-41b5-8d77-d735d5f776b6 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 15:47:28.064842 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-686236a8-b489-41b5-8d77-d735d5f776b6 None admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 15:47:28.087750 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG dbcounter [-] [81767] Writing DB stats keystone:SELECT=100 {{(pid=81767) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 15:47:28.094186 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application [None req-686236a8-b489-41b5-8d77-d735d5f776b6 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 22 15:47:28.094186 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 15:47:28.094186 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 15:47:28.094186 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 15:47:28.094186 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 15:47:28.094186 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 15:47:28.094186 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 15:47:28.094186 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 15:47:28.094186 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 15:47:28.094186 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 15:47:28.094186 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 15:47:28.094186 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 15:47:28.094186 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 22 15:47:28.094186 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 22 15:47:28.094186 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 22 15:47:28.094186 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 22 15:47:28.096311 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 15:47:28.096311 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 15:47:28.096311 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 22 15:47:28.096311 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 22 15:47:28.096311 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 22 15:47:28.096311 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 22 15:47:28.096311 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 15:47:28.096311 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 15:47:28.096311 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 15:47:28.096311 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 15:47:28.096311 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 15:47:28.096311 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application with Lock( Aug 22 15:47:28.096311 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 15:47:28.096311 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self._enter() Aug 22 15:47:28.096311 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 15:47:28.096311 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 15:47:28.096311 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 15:47:28.096311 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self.creator() Aug 22 15:47:28.098520 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 15:47:28.098520 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application created_value = creator( Aug 22 15:47:28.098520 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 22 15:47:28.098520 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 22 15:47:28.098520 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 22 15:47:28.098520 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 22 15:47:28.098520 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 22 15:47:28.098520 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 22 15:47:28.098520 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 22 15:47:28.098520 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application Aug 22 15:47:28.099762 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42dd410e-200b-4d96-a21f-63e4e81c7466 None admin] Validating token access rules against request {{(pid=81767) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 15:47:28.099921 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 279/551] 10.222.0.15 () {56 vars in 1096 bytes} [Mon Aug 22 15:47:27 2022] GET /identity/v3/users/swift => generated 83 bytes in 385 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 15:47:28.103365 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-686236a8-b489-41b5-8d77-d735d5f776b6 None admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 15:47:28.142729 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aefe955b-80df-478f-bddc-057c06a1a64a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81767) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 15:47:28.143480 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aefe955b-80df-478f-bddc-057c06a1a64a None admin] REQUEST_METHOD: `GET` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:47:28.143860 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aefe955b-80df-478f-bddc-057c06a1a64a None admin] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:47:28.144135 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aefe955b-80df-478f-bddc-057c06a1a64a None admin] PATH_INFO: `/v3/roles` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:47:28.144808 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aefe955b-80df-478f-bddc-057c06a1a64a None admin] RBAC: Adding query filter params (name=service) {{(pid=81767) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 15:47:28.145109 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aefe955b-80df-478f-bddc-057c06a1a64a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 15:47:28.146493 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aefe955b-80df-478f-bddc-057c06a1a64a None admin] RBAC: Authorization granted {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 15:47:28.156487 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=100 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 15:47:28.157624 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 273/552] 10.222.0.15 () {56 vars in 1103 bytes} [Mon Aug 22 15:47:27 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 22 15:47:28.159034 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55433e5f-8097-44d5-b3f5-16840e3fe483 None None] REQUEST_METHOD: `GET` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:47:28.159356 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55433e5f-8097-44d5-b3f5-16840e3fe483 None None] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:47:28.159632 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55433e5f-8097-44d5-b3f5-16840e3fe483 None None] PATH_INFO: `/` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:47:28.164996 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 274/553] 10.222.0.15 () {54 vars in 927 bytes} [Mon Aug 22 15:47:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 15:47:28.165797 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55433e5f-8097-44d5-b3f5-16840e3fe483 None None] Authenticating user token {{(pid=81767) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 15:47:28.215473 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-686236a8-b489-41b5-8d77-d735d5f776b6 None admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 15:47:28.250644 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0984a953-8871-4cec-8861-dd5a53810c7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 15:47:28.251242 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0984a953-8871-4cec-8861-dd5a53810c7f None admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:47:28.251527 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0984a953-8871-4cec-8861-dd5a53810c7f None admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:47:28.251798 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0984a953-8871-4cec-8861-dd5a53810c7f None admin] PATH_INFO: `/v3/users` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:47:28.252657 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0984a953-8871-4cec-8861-dd5a53810c7f None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81768) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 15:47:28.252959 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0984a953-8871-4cec-8861-dd5a53810c7f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 15:47:28.254378 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0984a953-8871-4cec-8861-dd5a53810c7f None admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 15:47:28.281469 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55433e5f-8097-44d5-b3f5-16840e3fe483 None None] Validating token access rules against request {{(pid=81767) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 15:47:28.295577 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 280/554] 10.222.0.15 () {56 vars in 1135 bytes} [Mon Aug 22 15:47:28 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 15:47:28.296710 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef4dd5a6-3e8a-4048-a616-6dd0df945edf None None] REQUEST_METHOD: `POST` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:47:28.296986 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef4dd5a6-3e8a-4048-a616-6dd0df945edf None None] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:47:28.297259 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef4dd5a6-3e8a-4048-a616-6dd0df945edf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:47:28.317307 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.auth.core [None req-ef4dd5a6-3e8a-4048-a616-6dd0df945edf None None] MFA Rules not processed for user `6b2199ae0fb0480aa917ada391f5ec54`. Rule list: `[]` (Enabled: `True`). {{(pid=81768) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 15:47:28.318513 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-ef4dd5a6-3e8a-4048-a616-6dd0df945edf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 15:47:28.330569 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9be2871-45c4-45cb-8068-4f529f35bf89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81767) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 15:47:28.331580 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9be2871-45c4-45cb-8068-4f529f35bf89 None admin] REQUEST_METHOD: `GET` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:47:28.335572 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9be2871-45c4-45cb-8068-4f529f35bf89 None admin] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:47:28.335867 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9be2871-45c4-45cb-8068-4f529f35bf89 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:47:28.347788 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9be2871-45c4-45cb-8068-4f529f35bf89 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 15:47:28.349246 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9be2871-45c4-45cb-8068-4f529f35bf89 None admin] RBAC: Authorization granted {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 15:47:28.364056 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application [None req-d9be2871-45c4-45cb-8068-4f529f35bf89 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 22 15:47:28.364056 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 15:47:28.364056 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 15:47:28.364056 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 15:47:28.364056 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 15:47:28.364056 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 15:47:28.364056 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 15:47:28.364056 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 15:47:28.364056 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 15:47:28.364056 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 15:47:28.364056 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 15:47:28.364056 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 15:47:28.364056 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 22 15:47:28.364056 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 22 15:47:28.364056 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 22 15:47:28.364056 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 22 15:47:28.365021 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 15:47:28.365021 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 15:47:28.365021 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 15:47:28.365021 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 15:47:28.365021 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 15:47:28.365021 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 15:47:28.365021 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 15:47:28.365021 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application with Lock( Aug 22 15:47:28.365021 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 15:47:28.365021 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return self._enter() Aug 22 15:47:28.365021 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 15:47:28.365021 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 15:47:28.365021 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 15:47:28.365021 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return self.creator() Aug 22 15:47:28.365021 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 15:47:28.365021 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application created_value = creator( Aug 22 15:47:28.365021 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 22 15:47:28.373605 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 22 15:47:28.373605 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 22 15:47:28.373605 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application Aug 22 15:47:28.373605 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 275/555] 10.222.0.15 () {56 vars in 1108 bytes} [Mon Aug 22 15:47:28 2022] GET /identity/v3/domains/Default => generated 87 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 15:47:28.373605 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daee03c7-1620-4504-9c5a-852cbdd5f355 None None] REQUEST_METHOD: `GET` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:47:28.373605 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daee03c7-1620-4504-9c5a-852cbdd5f355 None None] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:47:28.373605 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daee03c7-1620-4504-9c5a-852cbdd5f355 None None] PATH_INFO: `/` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:47:28.373605 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 276/556] 10.222.0.15 () {54 vars in 927 bytes} [Mon Aug 22 15:47:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 15:47:28.373605 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daee03c7-1620-4504-9c5a-852cbdd5f355 None None] Authenticating user token {{(pid=81767) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 15:47:28.384662 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG dbcounter [-] [81767] Writing DB stats keystone:SELECT=100 {{(pid=81767) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 15:47:28.405900 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 281/557] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Aug 22 15:47:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 22 15:47:28.407020 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef4dd5a6-3e8a-4048-a616-6dd0df945edf None None] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 15:47:28.468562 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=100 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 15:47:28.525273 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daee03c7-1620-4504-9c5a-852cbdd5f355 None None] Validating token access rules against request {{(pid=81767) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 15:47:28.543868 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef4dd5a6-3e8a-4048-a616-6dd0df945edf None None] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 15:47:28.570124 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5c2150d-2ff2-4b0d-ab36-fb243cc22da2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81767) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 15:47:28.571298 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5c2150d-2ff2-4b0d-ab36-fb243cc22da2 None admin] REQUEST_METHOD: `GET` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:47:28.572047 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5c2150d-2ff2-4b0d-ab36-fb243cc22da2 None admin] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:47:28.572636 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5c2150d-2ff2-4b0d-ab36-fb243cc22da2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:47:28.583783 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40599f84-a9c3-4555-87e5-a4bc639c1fa6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 15:47:28.584459 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40599f84-a9c3-4555-87e5-a4bc639c1fa6 None admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:47:28.584835 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40599f84-a9c3-4555-87e5-a4bc639c1fa6 None admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:47:28.585494 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40599f84-a9c3-4555-87e5-a4bc639c1fa6 None admin] PATH_INFO: `/v3/domains` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:47:28.586009 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40599f84-a9c3-4555-87e5-a4bc639c1fa6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81768) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 15:47:28.586388 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40599f84-a9c3-4555-87e5-a4bc639c1fa6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 15:47:28.587916 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40599f84-a9c3-4555-87e5-a4bc639c1fa6 None admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 15:47:28.607341 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 282/558] 10.222.0.15 () {56 vars in 1109 bytes} [Mon Aug 22 15:47:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 15:47:28.608932 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6c5854f-aecd-439e-b2f0-7832bc5a9858 None None] REQUEST_METHOD: `POST` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:47:28.609193 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6c5854f-aecd-439e-b2f0-7832bc5a9858 None None] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:47:28.609542 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6c5854f-aecd-439e-b2f0-7832bc5a9858 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:47:28.627832 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.auth.core [None req-c6c5854f-aecd-439e-b2f0-7832bc5a9858 None None] MFA Rules not processed for user `6b2199ae0fb0480aa917ada391f5ec54`. Rule list: `[]` (Enabled: `True`). {{(pid=81768) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 15:47:28.629128 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-c6c5854f-aecd-439e-b2f0-7832bc5a9858 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 15:47:28.671031 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5c2150d-2ff2-4b0d-ab36-fb243cc22da2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 15:47:28.673163 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5c2150d-2ff2-4b0d-ab36-fb243cc22da2 None admin] RBAC: Authorization granted {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 15:47:28.688933 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG dbcounter [-] [81767] Writing DB stats keystone:SELECT=100 {{(pid=81767) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 15:47:28.706228 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 283/559] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Aug 22 15:47:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 22 15:47:28.708153 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6c5854f-aecd-439e-b2f0-7832bc5a9858 None None] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 15:47:28.742863 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=100 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 15:47:28.801052 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 277/560] 10.222.0.15 () {58 vars in 1282 bytes} [Mon Aug 22 15:47:28 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 432 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 22 15:47:28.802283 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5c2150d-2ff2-4b0d-ab36-fb243cc22da2 None admin] Authenticating user token {{(pid=81767) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 15:47:28.846500 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6c5854f-aecd-439e-b2f0-7832bc5a9858 None None] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 15:47:28.888310 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ce5a99c-5379-4568-bee8-1c810e7afe45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 15:47:28.889134 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ce5a99c-5379-4568-bee8-1c810e7afe45 None admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:47:28.889517 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ce5a99c-5379-4568-bee8-1c810e7afe45 None admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:47:28.889892 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ce5a99c-5379-4568-bee8-1c810e7afe45 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:47:28.898462 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ce5a99c-5379-4568-bee8-1c810e7afe45 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 15:47:28.900244 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ce5a99c-5379-4568-bee8-1c810e7afe45 None admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 15:47:28.915882 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5c2150d-2ff2-4b0d-ab36-fb243cc22da2 None admin] Validating token access rules against request {{(pid=81767) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 15:47:28.916535 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application [None req-1ce5a99c-5379-4568-bee8-1c810e7afe45 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 22 15:47:28.916535 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 15:47:28.916535 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 15:47:28.916535 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 15:47:28.916535 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 15:47:28.916535 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 15:47:28.916535 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 15:47:28.916535 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 15:47:28.916535 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 15:47:28.916535 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 15:47:28.916535 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 15:47:28.916535 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 15:47:28.916535 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 22 15:47:28.916535 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 22 15:47:28.916535 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 22 15:47:28.916535 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 22 15:47:28.918251 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 15:47:28.918251 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 15:47:28.918251 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 22 15:47:28.918251 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 22 15:47:28.918251 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 22 15:47:28.918251 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 22 15:47:28.918251 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 15:47:28.918251 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 15:47:28.918251 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 15:47:28.918251 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 15:47:28.918251 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 15:47:28.918251 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application with Lock( Aug 22 15:47:28.918251 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 15:47:28.918251 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self._enter() Aug 22 15:47:28.918251 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 15:47:28.918251 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 15:47:28.918251 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 15:47:28.918251 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self.creator() Aug 22 15:47:28.919488 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 15:47:28.919488 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application created_value = creator( Aug 22 15:47:28.919488 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 22 15:47:28.919488 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 22 15:47:28.919488 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 22 15:47:28.919488 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 22 15:47:28.919488 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 22 15:47:28.919488 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 22 15:47:28.919488 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 22 15:47:28.919488 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application Aug 22 15:47:28.926217 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 284/561] 10.222.0.15 () {56 vars in 1093 bytes} [Mon Aug 22 15:47:28 2022] GET /identity/v3/users/nova => generated 82 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 15:47:28.927306 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ce5a99c-5379-4568-bee8-1c810e7afe45 None admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 15:47:28.933637 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG dbcounter [-] [81767] Writing DB stats keystone:SELECT=100 {{(pid=81767) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 15:47:28.958637 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43d6f051-5b97-49ce-a6e7-cfb291a2abd4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81767) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 15:47:28.959613 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43d6f051-5b97-49ce-a6e7-cfb291a2abd4 None admin] REQUEST_METHOD: `GET` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:47:28.960194 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43d6f051-5b97-49ce-a6e7-cfb291a2abd4 None admin] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:47:28.960661 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43d6f051-5b97-49ce-a6e7-cfb291a2abd4 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:47:28.972479 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43d6f051-5b97-49ce-a6e7-cfb291a2abd4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 22 15:47:28.977749 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application [None req-43d6f051-5b97-49ce-a6e7-cfb291a2abd4 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 22 15:47:28.977749 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 15:47:28.977749 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 15:47:28.977749 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 15:47:28.977749 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 15:47:28.977749 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 15:47:28.977749 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 15:47:28.977749 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 15:47:28.977749 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 15:47:28.977749 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 15:47:28.977749 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 15:47:28.977749 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 15:47:28.977749 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 22 15:47:28.977749 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 22 15:47:28.977749 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 22 15:47:28.977749 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 22 15:47:28.978879 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 22 15:47:28.978879 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 22 15:47:28.978879 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 22 15:47:28.978879 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 22 15:47:28.978879 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 15:47:28.978879 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 15:47:28.978879 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 15:47:28.978879 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 15:47:28.978879 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 15:47:28.978879 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 15:47:28.978879 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 15:47:28.978879 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application with Lock( Aug 22 15:47:28.978879 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 15:47:28.978879 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return self._enter() Aug 22 15:47:28.978879 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 15:47:28.978879 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 15:47:28.980492 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 15:47:28.980492 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return self.creator() Aug 22 15:47:28.980492 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 15:47:28.980492 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application created_value = creator( Aug 22 15:47:28.980492 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 22 15:47:28.980492 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 22 15:47:28.980492 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 22 15:47:28.980492 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 22 15:47:28.980492 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 22 15:47:28.980492 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 22 15:47:28.980492 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 22 15:47:28.980492 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application Aug 22 15:47:28.982819 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 278/562] 10.222.0.15 () {56 vars in 1102 bytes} [Mon Aug 22 15:47:28 2022] GET /identity/v3/roles/service => generated 85 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 15:47:28.983870 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43d6f051-5b97-49ce-a6e7-cfb291a2abd4 None admin] Authenticating user token {{(pid=81767) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 15:47:29.027216 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=100 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 15:47:29.043019 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ce5a99c-5379-4568-bee8-1c810e7afe45 None admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 15:47:29.079148 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0d23cb0-f3d2-4db9-871f-1fc74aa0e113 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 15:47:29.093639 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d23cb0-f3d2-4db9-871f-1fc74aa0e113 None admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:47:29.093639 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d23cb0-f3d2-4db9-871f-1fc74aa0e113 None admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:47:29.093639 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d23cb0-f3d2-4db9-871f-1fc74aa0e113 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:47:29.093939 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43d6f051-5b97-49ce-a6e7-cfb291a2abd4 None admin] Validating token access rules against request {{(pid=81767) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 15:47:29.094039 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0d23cb0-f3d2-4db9-871f-1fc74aa0e113 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 15:47:29.094039 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0d23cb0-f3d2-4db9-871f-1fc74aa0e113 None admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 15:47:29.109805 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application [None req-d0d23cb0-f3d2-4db9-871f-1fc74aa0e113 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 22 15:47:29.109805 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 15:47:29.109805 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 15:47:29.109805 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 15:47:29.109805 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 15:47:29.109805 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 15:47:29.109805 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 15:47:29.109805 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 15:47:29.109805 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 15:47:29.109805 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 15:47:29.109805 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 15:47:29.109805 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 15:47:29.109805 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 22 15:47:29.109805 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 22 15:47:29.109805 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 22 15:47:29.109805 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 22 15:47:29.115616 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 15:47:29.115616 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 15:47:29.115616 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 15:47:29.115616 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 15:47:29.115616 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 15:47:29.115616 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 15:47:29.115616 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 15:47:29.115616 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application with Lock( Aug 22 15:47:29.115616 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 15:47:29.115616 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self._enter() Aug 22 15:47:29.115616 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 15:47:29.115616 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 15:47:29.115616 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 15:47:29.115616 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self.creator() Aug 22 15:47:29.115616 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 15:47:29.115616 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application created_value = creator( Aug 22 15:47:29.115616 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 22 15:47:29.116752 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 22 15:47:29.116752 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 22 15:47:29.116752 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application Aug 22 15:47:29.116752 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 285/563] 10.222.0.15 () {56 vars in 1108 bytes} [Mon Aug 22 15:47:28 2022] GET /identity/v3/domains/Default => generated 87 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 15:47:29.116752 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0d23cb0-f3d2-4db9-871f-1fc74aa0e113 None admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 15:47:29.138602 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98c6c16e-fabd-4510-acac-6909b69d2a7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81767) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 15:47:29.139559 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98c6c16e-fabd-4510-acac-6909b69d2a7c None admin] REQUEST_METHOD: `GET` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:47:29.139833 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98c6c16e-fabd-4510-acac-6909b69d2a7c None admin] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:47:29.140098 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98c6c16e-fabd-4510-acac-6909b69d2a7c None admin] PATH_INFO: `/v3/users` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:47:29.140836 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98c6c16e-fabd-4510-acac-6909b69d2a7c None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81767) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 15:47:29.141135 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98c6c16e-fabd-4510-acac-6909b69d2a7c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 15:47:29.142573 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98c6c16e-fabd-4510-acac-6909b69d2a7c None admin] RBAC: Authorization granted {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 15:47:29.160921 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 279/564] 10.222.0.15 () {56 vars in 1133 bytes} [Mon Aug 22 15:47:28 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 15:47:29.164209 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98c6c16e-fabd-4510-acac-6909b69d2a7c None admin] Authenticating user token {{(pid=81767) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 15:47:29.192777 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG dbcounter [-] [81767] Writing DB stats keystone:SELECT=100 {{(pid=81767) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 15:47:29.220622 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0d23cb0-f3d2-4db9-871f-1fc74aa0e113 None admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 15:47:29.254532 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dfbac34-5989-4e29-92ff-5bf188731e90 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 15:47:29.255138 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dfbac34-5989-4e29-92ff-5bf188731e90 None admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:47:29.255510 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dfbac34-5989-4e29-92ff-5bf188731e90 None admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:47:29.255770 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dfbac34-5989-4e29-92ff-5bf188731e90 None admin] PATH_INFO: `/v3/roles` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:47:29.256515 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dfbac34-5989-4e29-92ff-5bf188731e90 None admin] RBAC: Adding query filter params (name=service) {{(pid=81768) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 15:47:29.256735 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dfbac34-5989-4e29-92ff-5bf188731e90 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 15:47:29.258132 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dfbac34-5989-4e29-92ff-5bf188731e90 None admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 15:47:29.261149 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=100 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 15:47:29.266546 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98c6c16e-fabd-4510-acac-6909b69d2a7c None admin] Validating token access rules against request {{(pid=81767) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 15:47:29.268360 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 286/565] 10.222.0.15 () {56 vars in 1103 bytes} [Mon Aug 22 15:47:29 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 15:47:29.268983 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dfbac34-5989-4e29-92ff-5bf188731e90 None admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 15:47:29.305982 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2520c728-2c85-4dba-9dce-2925d1fa1491 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81767) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 15:47:29.306607 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2520c728-2c85-4dba-9dce-2925d1fa1491 None admin] REQUEST_METHOD: `GET` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:47:29.307141 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2520c728-2c85-4dba-9dce-2925d1fa1491 None admin] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:47:29.307450 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2520c728-2c85-4dba-9dce-2925d1fa1491 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:47:29.318275 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2520c728-2c85-4dba-9dce-2925d1fa1491 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 22 15:47:29.323856 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application [None req-2520c728-2c85-4dba-9dce-2925d1fa1491 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 22 15:47:29.323856 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 15:47:29.323856 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 15:47:29.323856 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 15:47:29.323856 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 15:47:29.323856 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 15:47:29.323856 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 15:47:29.323856 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 15:47:29.323856 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 15:47:29.323856 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 15:47:29.323856 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 15:47:29.323856 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 15:47:29.323856 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 22 15:47:29.323856 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 22 15:47:29.323856 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 22 15:47:29.323856 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 22 15:47:29.325070 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 22 15:47:29.325070 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 22 15:47:29.325070 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 22 15:47:29.325070 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 22 15:47:29.325070 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 15:47:29.325070 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 15:47:29.325070 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 15:47:29.325070 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 15:47:29.325070 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 15:47:29.325070 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 15:47:29.325070 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 15:47:29.325070 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application with Lock( Aug 22 15:47:29.325070 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 15:47:29.325070 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return self._enter() Aug 22 15:47:29.325070 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 15:47:29.325070 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 15:47:29.326068 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 15:47:29.326068 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return self.creator() Aug 22 15:47:29.326068 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 15:47:29.326068 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application created_value = creator( Aug 22 15:47:29.326068 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 22 15:47:29.326068 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 22 15:47:29.326068 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 22 15:47:29.326068 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 22 15:47:29.326068 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 22 15:47:29.326068 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 22 15:47:29.326068 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 22 15:47:29.326068 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application Aug 22 15:47:29.326068 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 280/566] 10.222.0.15 () {56 vars in 1102 bytes} [Mon Aug 22 15:47:29 2022] GET /identity/v3/roles/service => generated 85 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 15:47:29.326068 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cefb189-62ac-4ff9-a9c8-9cff7bfabedb None None] REQUEST_METHOD: `GET` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:47:29.326975 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cefb189-62ac-4ff9-a9c8-9cff7bfabedb None None] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:47:29.326975 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cefb189-62ac-4ff9-a9c8-9cff7bfabedb None None] PATH_INFO: `/` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:47:29.327977 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 281/567] 10.222.0.15 () {54 vars in 927 bytes} [Mon Aug 22 15:47:29 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 15:47:29.331410 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cefb189-62ac-4ff9-a9c8-9cff7bfabedb None None] Authenticating user token {{(pid=81767) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 15:47:29.374220 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dfbac34-5989-4e29-92ff-5bf188731e90 None admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 15:47:29.414096 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-777e48f9-8213-4703-b74d-7196109de1c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 15:47:29.414997 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-777e48f9-8213-4703-b74d-7196109de1c0 None admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:47:29.415288 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-777e48f9-8213-4703-b74d-7196109de1c0 None admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:47:29.415575 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-777e48f9-8213-4703-b74d-7196109de1c0 None admin] PATH_INFO: `/v3/domains` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:47:29.416214 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-777e48f9-8213-4703-b74d-7196109de1c0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81768) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 15:47:29.416545 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-777e48f9-8213-4703-b74d-7196109de1c0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 15:47:29.417975 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-777e48f9-8213-4703-b74d-7196109de1c0 None admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 15:47:29.421642 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG dbcounter [-] [81767] Writing DB stats keystone:SELECT=100 {{(pid=81767) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 15:47:29.429521 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 287/568] 10.222.0.15 () {56 vars in 1109 bytes} [Mon Aug 22 15:47:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 15:47:29.430336 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-777e48f9-8213-4703-b74d-7196109de1c0 None admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 15:47:29.438001 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cefb189-62ac-4ff9-a9c8-9cff7bfabedb None None] Validating token access rules against request {{(pid=81767) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 15:47:29.471950 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-578f4ed6-3e45-4bbe-94f5-fe5678aaabc2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81767) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 15:47:29.472567 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-578f4ed6-3e45-4bbe-94f5-fe5678aaabc2 None admin] REQUEST_METHOD: `GET` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:47:29.472843 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-578f4ed6-3e45-4bbe-94f5-fe5678aaabc2 None admin] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:47:29.473117 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-578f4ed6-3e45-4bbe-94f5-fe5678aaabc2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:47:29.483034 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-578f4ed6-3e45-4bbe-94f5-fe5678aaabc2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 15:47:29.484605 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-578f4ed6-3e45-4bbe-94f5-fe5678aaabc2 None admin] RBAC: Authorization granted {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 15:47:29.489939 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=100 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 15:47:29.500530 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application [None req-578f4ed6-3e45-4bbe-94f5-fe5678aaabc2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 22 15:47:29.500530 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 15:47:29.500530 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 15:47:29.500530 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 15:47:29.500530 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 15:47:29.500530 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 15:47:29.500530 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 15:47:29.500530 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 15:47:29.500530 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 15:47:29.500530 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 15:47:29.500530 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 15:47:29.500530 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 15:47:29.500530 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 22 15:47:29.500530 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 22 15:47:29.500530 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 22 15:47:29.500530 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 22 15:47:29.501696 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 15:47:29.501696 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 15:47:29.501696 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 15:47:29.501696 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 15:47:29.501696 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 15:47:29.501696 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 15:47:29.501696 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 15:47:29.501696 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application with Lock( Aug 22 15:47:29.501696 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 15:47:29.501696 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return self._enter() Aug 22 15:47:29.501696 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 15:47:29.501696 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 15:47:29.501696 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 15:47:29.501696 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return self.creator() Aug 22 15:47:29.501696 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 15:47:29.501696 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application created_value = creator( Aug 22 15:47:29.501696 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 22 15:47:29.502630 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 22 15:47:29.502630 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 22 15:47:29.502630 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application Aug 22 15:47:29.502630 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 282/569] 10.222.0.15 () {56 vars in 1108 bytes} [Mon Aug 22 15:47:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 15:47:29.502630 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-578f4ed6-3e45-4bbe-94f5-fe5678aaabc2 None admin] Authenticating user token {{(pid=81767) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 15:47:29.536598 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-777e48f9-8213-4703-b74d-7196109de1c0 None admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 15:47:29.569770 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-effb913a-2b6b-4b2f-8c49-419ea4d51a88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 15:47:29.570377 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-effb913a-2b6b-4b2f-8c49-419ea4d51a88 None admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:47:29.570656 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-effb913a-2b6b-4b2f-8c49-419ea4d51a88 None admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:47:29.570923 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-effb913a-2b6b-4b2f-8c49-419ea4d51a88 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:47:29.582395 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-effb913a-2b6b-4b2f-8c49-419ea4d51a88 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 15:47:29.584396 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-effb913a-2b6b-4b2f-8c49-419ea4d51a88 None admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 15:47:29.598844 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application [None req-effb913a-2b6b-4b2f-8c49-419ea4d51a88 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 22 15:47:29.598844 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 15:47:29.598844 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 15:47:29.598844 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 15:47:29.598844 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 15:47:29.598844 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 15:47:29.598844 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 15:47:29.598844 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 15:47:29.598844 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 15:47:29.598844 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 15:47:29.598844 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 15:47:29.598844 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 15:47:29.598844 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 22 15:47:29.598844 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 22 15:47:29.598844 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 22 15:47:29.598844 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 22 15:47:29.599916 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 15:47:29.599916 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 15:47:29.599916 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 15:47:29.599916 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 15:47:29.599916 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 15:47:29.599916 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 15:47:29.599916 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 15:47:29.599916 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application with Lock( Aug 22 15:47:29.599916 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 15:47:29.599916 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self._enter() Aug 22 15:47:29.599916 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 15:47:29.599916 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 15:47:29.599916 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 15:47:29.599916 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self.creator() Aug 22 15:47:29.599916 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 15:47:29.599916 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application created_value = creator( Aug 22 15:47:29.599916 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 22 15:47:29.600780 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 22 15:47:29.600780 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 22 15:47:29.600780 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application Aug 22 15:47:29.600780 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 288/570] 10.222.0.15 () {56 vars in 1108 bytes} [Mon Aug 22 15:47:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 15:47:29.601459 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f531b32-b7c7-4b65-b834-2e62e6c28827 None None] REQUEST_METHOD: `POST` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:47:29.601573 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f531b32-b7c7-4b65-b834-2e62e6c28827 None None] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:47:29.601768 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f531b32-b7c7-4b65-b834-2e62e6c28827 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:47:29.610965 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-578f4ed6-3e45-4bbe-94f5-fe5678aaabc2 None admin] Validating token access rules against request {{(pid=81767) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 15:47:29.620900 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.auth.core [None req-2f531b32-b7c7-4b65-b834-2e62e6c28827 None None] MFA Rules not processed for user `6b2199ae0fb0480aa917ada391f5ec54`. Rule list: `[]` (Enabled: `True`). {{(pid=81768) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 15:47:29.622076 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-2f531b32-b7c7-4b65-b834-2e62e6c28827 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 15:47:29.646879 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ddb946e-010a-4412-a70c-f62e0daa2092 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81767) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 15:47:29.647517 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ddb946e-010a-4412-a70c-f62e0daa2092 None admin] REQUEST_METHOD: `GET` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:47:29.647794 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ddb946e-010a-4412-a70c-f62e0daa2092 None admin] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:47:29.648065 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ddb946e-010a-4412-a70c-f62e0daa2092 None admin] PATH_INFO: `/v3/roles` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:47:29.648768 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ddb946e-010a-4412-a70c-f62e0daa2092 None admin] RBAC: Adding query filter params (name=service) {{(pid=81767) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 15:47:29.649087 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ddb946e-010a-4412-a70c-f62e0daa2092 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 15:47:29.650474 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ddb946e-010a-4412-a70c-f62e0daa2092 None admin] RBAC: Authorization granted {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 15:47:29.658853 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 283/571] 10.222.0.15 () {56 vars in 1103 bytes} [Mon Aug 22 15:47:29 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 15:47:29.659728 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ddb946e-010a-4412-a70c-f62e0daa2092 None admin] Authenticating user token {{(pid=81767) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 15:47:29.662411 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG dbcounter [-] [81767] Writing DB stats keystone:SELECT=100 {{(pid=81767) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 15:47:29.690231 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 289/572] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Aug 22 15:47:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 22 15:47:29.691043 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f531b32-b7c7-4b65-b834-2e62e6c28827 None None] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 15:47:29.738694 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=100 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 15:47:29.767883 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ddb946e-010a-4412-a70c-f62e0daa2092 None admin] Validating token access rules against request {{(pid=81767) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 15:47:29.802271 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f531b32-b7c7-4b65-b834-2e62e6c28827 None None] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 15:47:29.802800 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f6bb8c8-c579-4ef5-9ca6-1288b9713729 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81767) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 15:47:29.803444 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f6bb8c8-c579-4ef5-9ca6-1288b9713729 None admin] REQUEST_METHOD: `GET` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:47:29.803730 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f6bb8c8-c579-4ef5-9ca6-1288b9713729 None admin] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:47:29.804001 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f6bb8c8-c579-4ef5-9ca6-1288b9713729 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:47:29.813518 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f6bb8c8-c579-4ef5-9ca6-1288b9713729 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 15:47:29.815482 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f6bb8c8-c579-4ef5-9ca6-1288b9713729 None admin] RBAC: Authorization granted {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 15:47:29.828171 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application [None req-6f6bb8c8-c579-4ef5-9ca6-1288b9713729 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 22 15:47:29.828171 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 15:47:29.828171 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 15:47:29.828171 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 15:47:29.828171 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 15:47:29.828171 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 15:47:29.828171 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 15:47:29.828171 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 15:47:29.828171 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 15:47:29.828171 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 15:47:29.828171 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 15:47:29.828171 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 15:47:29.828171 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 22 15:47:29.828171 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 22 15:47:29.828171 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 22 15:47:29.829281 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 22 15:47:29.829281 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 15:47:29.829281 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 15:47:29.829281 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 15:47:29.829281 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 15:47:29.829281 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 15:47:29.829281 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 15:47:29.829281 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 15:47:29.829281 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application with Lock( Aug 22 15:47:29.829281 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 15:47:29.829281 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return self._enter() Aug 22 15:47:29.829281 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 15:47:29.829281 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 15:47:29.829281 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 15:47:29.829281 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return self.creator() Aug 22 15:47:29.829281 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 15:47:29.829281 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application created_value = creator( Aug 22 15:47:29.830138 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 22 15:47:29.830138 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 22 15:47:29.830138 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 22 15:47:29.830138 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 22 15:47:29.830138 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 22 15:47:29.830138 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 22 15:47:29.830138 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 22 15:47:29.830138 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application Aug 22 15:47:29.830138 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 284/573] 10.222.0.15 () {56 vars in 1111 bytes} [Mon Aug 22 15:47:29 2022] GET /identity/v3/projects/service => generated 88 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 15:47:29.830138 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f6bb8c8-c579-4ef5-9ca6-1288b9713729 None admin] Authenticating user token {{(pid=81767) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 15:47:29.845409 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0456d6a7-e8fa-4988-aa27-74d05644957a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 15:47:29.846024 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0456d6a7-e8fa-4988-aa27-74d05644957a None admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:47:29.846297 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0456d6a7-e8fa-4988-aa27-74d05644957a None admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:47:29.847095 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0456d6a7-e8fa-4988-aa27-74d05644957a None admin] PATH_INFO: `/v3/domains` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:47:29.847208 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0456d6a7-e8fa-4988-aa27-74d05644957a None admin] RBAC: Adding query filter params (name=Default) {{(pid=81768) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 15:47:29.847689 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0456d6a7-e8fa-4988-aa27-74d05644957a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 15:47:29.848903 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0456d6a7-e8fa-4988-aa27-74d05644957a None admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 15:47:29.861950 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 290/574] 10.222.0.15 () {56 vars in 1109 bytes} [Mon Aug 22 15:47:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 15:47:29.862668 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0456d6a7-e8fa-4988-aa27-74d05644957a None admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 15:47:29.908417 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG dbcounter [-] [81767] Writing DB stats keystone:SELECT=100 {{(pid=81767) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 15:47:29.966933 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=100 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 15:47:29.969882 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0456d6a7-e8fa-4988-aa27-74d05644957a None admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 15:47:30.068414 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f6bb8c8-c579-4ef5-9ca6-1288b9713729 None admin] Validating token access rules against request {{(pid=81767) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 15:47:30.071211 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58e13031-6e66-4e4b-bf61-65ec96bba8f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 15:47:30.071903 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58e13031-6e66-4e4b-bf61-65ec96bba8f0 None admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:47:30.072183 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58e13031-6e66-4e4b-bf61-65ec96bba8f0 None admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:47:30.072475 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58e13031-6e66-4e4b-bf61-65ec96bba8f0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:47:30.084366 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58e13031-6e66-4e4b-bf61-65ec96bba8f0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 15:47:30.085769 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58e13031-6e66-4e4b-bf61-65ec96bba8f0 None admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 15:47:30.100676 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application [None req-58e13031-6e66-4e4b-bf61-65ec96bba8f0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 22 15:47:30.100676 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 15:47:30.100676 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 15:47:30.100676 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 15:47:30.100676 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 15:47:30.100676 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 15:47:30.100676 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 15:47:30.100676 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 15:47:30.100676 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 15:47:30.100676 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 15:47:30.100676 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 15:47:30.100676 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 15:47:30.100676 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 22 15:47:30.100676 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 22 15:47:30.100676 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 22 15:47:30.100676 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 22 15:47:30.101822 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 15:47:30.101822 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 15:47:30.101822 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 15:47:30.101822 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 15:47:30.101822 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 15:47:30.101822 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 15:47:30.101822 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 15:47:30.101822 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application with Lock( Aug 22 15:47:30.101822 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 15:47:30.101822 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self._enter() Aug 22 15:47:30.101822 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 15:47:30.101822 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 15:47:30.101822 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 15:47:30.101822 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self.creator() Aug 22 15:47:30.101822 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 15:47:30.101822 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application created_value = creator( Aug 22 15:47:30.101822 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 22 15:47:30.102865 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 22 15:47:30.102865 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 22 15:47:30.102865 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application Aug 22 15:47:30.102865 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 291/575] 10.222.0.15 () {56 vars in 1108 bytes} [Mon Aug 22 15:47:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 239 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 15:47:30.102865 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58e13031-6e66-4e4b-bf61-65ec96bba8f0 None admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 15:47:30.108368 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29336495-5895-4589-a64f-d84d1fc050a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81767) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 15:47:30.109573 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29336495-5895-4589-a64f-d84d1fc050a3 None admin] REQUEST_METHOD: `GET` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:47:30.109858 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29336495-5895-4589-a64f-d84d1fc050a3 None admin] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:47:30.110330 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29336495-5895-4589-a64f-d84d1fc050a3 None admin] PATH_INFO: `/v3/domains` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:47:30.111493 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29336495-5895-4589-a64f-d84d1fc050a3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81767) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 15:47:30.111802 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29336495-5895-4589-a64f-d84d1fc050a3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 15:47:30.113164 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29336495-5895-4589-a64f-d84d1fc050a3 None admin] RBAC: Authorization granted {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 15:47:30.126694 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 285/576] 10.222.0.15 () {56 vars in 1109 bytes} [Mon Aug 22 15:47:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 297 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 15:47:30.127472 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29336495-5895-4589-a64f-d84d1fc050a3 None admin] Authenticating user token {{(pid=81767) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 15:47:30.216883 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58e13031-6e66-4e4b-bf61-65ec96bba8f0 None admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 15:47:30.228736 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29336495-5895-4589-a64f-d84d1fc050a3 None admin] Validating token access rules against request {{(pid=81767) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 15:47:30.252397 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b627058-3eb5-42e9-825b-a6dcd52b68d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 15:47:30.252991 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b627058-3eb5-42e9-825b-a6dcd52b68d3 None admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:47:30.253262 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b627058-3eb5-42e9-825b-a6dcd52b68d3 None admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:47:30.253528 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b627058-3eb5-42e9-825b-a6dcd52b68d3 None admin] PATH_INFO: `/v3/projects` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:47:30.254202 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b627058-3eb5-42e9-825b-a6dcd52b68d3 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81768) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 15:47:30.254501 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b627058-3eb5-42e9-825b-a6dcd52b68d3 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 15:47:30.254841 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG dbcounter [-] [81767] Writing DB stats keystone:SELECT=100 {{(pid=81767) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 15:47:30.255966 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b627058-3eb5-42e9-825b-a6dcd52b68d3 None admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 15:47:30.265930 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-473e2d75-e538-4057-9a86-89568251ee5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81767) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 15:47:30.266550 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-473e2d75-e538-4057-9a86-89568251ee5f None admin] REQUEST_METHOD: `GET` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:47:30.266828 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-473e2d75-e538-4057-9a86-89568251ee5f None admin] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:47:30.267101 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-473e2d75-e538-4057-9a86-89568251ee5f None admin] PATH_INFO: `/v3/projects/service` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:47:30.268136 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 292/577] 10.222.0.15 () {56 vars in 1148 bytes} [Mon Aug 22 15:47:30 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 15:47:30.268922 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b627058-3eb5-42e9-825b-a6dcd52b68d3 None admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 15:47:30.273799 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-473e2d75-e538-4057-9a86-89568251ee5f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 15:47:30.275228 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-473e2d75-e538-4057-9a86-89568251ee5f None admin] RBAC: Authorization granted {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 15:47:30.286060 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=100 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 15:47:30.287216 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application [None req-473e2d75-e538-4057-9a86-89568251ee5f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 22 15:47:30.287216 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 15:47:30.287216 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 15:47:30.287216 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 15:47:30.287216 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 15:47:30.287216 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 15:47:30.287216 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 15:47:30.287216 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 15:47:30.287216 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 15:47:30.287216 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 15:47:30.287216 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 15:47:30.287216 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 15:47:30.287216 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 22 15:47:30.287216 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 22 15:47:30.287216 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 22 15:47:30.288233 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 22 15:47:30.288233 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 15:47:30.288233 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 15:47:30.288233 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 15:47:30.288233 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 15:47:30.288233 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 15:47:30.288233 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 15:47:30.288233 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 15:47:30.288233 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application with Lock( Aug 22 15:47:30.288233 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 15:47:30.288233 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return self._enter() Aug 22 15:47:30.288233 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 15:47:30.288233 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 15:47:30.288233 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 15:47:30.288233 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return self.creator() Aug 22 15:47:30.288233 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 15:47:30.288233 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application created_value = creator( Aug 22 15:47:30.290425 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 22 15:47:30.290425 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 22 15:47:30.290425 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 22 15:47:30.290425 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 22 15:47:30.290425 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 22 15:47:30.290425 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 22 15:47:30.290425 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 22 15:47:30.290425 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application Aug 22 15:47:30.290425 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 286/578] 10.222.0.15 () {56 vars in 1111 bytes} [Mon Aug 22 15:47:30 2022] GET /identity/v3/projects/service => generated 88 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 15:47:30.290425 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-473e2d75-e538-4057-9a86-89568251ee5f None admin] Authenticating user token {{(pid=81767) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 15:47:30.381501 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b627058-3eb5-42e9-825b-a6dcd52b68d3 None admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 15:47:30.397019 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-473e2d75-e538-4057-9a86-89568251ee5f None admin] Validating token access rules against request {{(pid=81767) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 15:47:30.418716 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b27d8ce-f0c6-435b-858a-818a0d7c80f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 15:47:30.419372 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b27d8ce-f0c6-435b-858a-818a0d7c80f4 None admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:47:30.419681 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b27d8ce-f0c6-435b-858a-818a0d7c80f4 None admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:47:30.419925 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b27d8ce-f0c6-435b-858a-818a0d7c80f4 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:47:30.430358 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b27d8ce-f0c6-435b-858a-818a0d7c80f4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 22 15:47:30.431984 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0420f041-1a93-4d24-9510-c4a15a228ad5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81767) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 15:47:30.432609 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0420f041-1a93-4d24-9510-c4a15a228ad5 None admin] REQUEST_METHOD: `GET` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:47:30.432885 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0420f041-1a93-4d24-9510-c4a15a228ad5 None admin] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:47:30.433152 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0420f041-1a93-4d24-9510-c4a15a228ad5 None admin] PATH_INFO: `/v3/domains` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:47:30.433978 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0420f041-1a93-4d24-9510-c4a15a228ad5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81767) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 15:47:30.434274 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0420f041-1a93-4d24-9510-c4a15a228ad5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 15:47:30.435384 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application [None req-9b27d8ce-f0c6-435b-858a-818a0d7c80f4 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 22 15:47:30.435384 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 15:47:30.435384 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 15:47:30.435384 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 15:47:30.435384 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 15:47:30.435384 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 15:47:30.435384 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 15:47:30.435384 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 15:47:30.435384 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 15:47:30.435384 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 15:47:30.435384 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 15:47:30.435384 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 15:47:30.435384 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 22 15:47:30.435384 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 22 15:47:30.435384 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 22 15:47:30.435384 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 22 15:47:30.436468 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 22 15:47:30.436468 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 22 15:47:30.436468 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 22 15:47:30.436468 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 22 15:47:30.436468 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 15:47:30.436468 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 15:47:30.436468 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 15:47:30.436468 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 15:47:30.436468 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 15:47:30.436468 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 15:47:30.436468 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 15:47:30.436468 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application with Lock( Aug 22 15:47:30.436468 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 15:47:30.436468 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self._enter() Aug 22 15:47:30.436468 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 15:47:30.436468 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 15:47:30.437305 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 15:47:30.437305 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self.creator() Aug 22 15:47:30.437305 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 15:47:30.437305 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application created_value = creator( Aug 22 15:47:30.437305 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 22 15:47:30.437305 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 22 15:47:30.437305 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 22 15:47:30.437305 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 22 15:47:30.437305 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 22 15:47:30.437305 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 22 15:47:30.437305 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 22 15:47:30.437305 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application Aug 22 15:47:30.437945 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0420f041-1a93-4d24-9510-c4a15a228ad5 None admin] RBAC: Authorization granted {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 15:47:30.438022 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 293/579] 10.222.0.15 () {56 vars in 1102 bytes} [Mon Aug 22 15:47:30 2022] GET /identity/v3/roles/service => generated 85 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 15:47:30.438022 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b27d8ce-f0c6-435b-858a-818a0d7c80f4 None admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 15:47:30.452426 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 287/580] 10.222.0.15 () {56 vars in 1109 bytes} [Mon Aug 22 15:47:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 15:47:30.453258 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0420f041-1a93-4d24-9510-c4a15a228ad5 None admin] Authenticating user token {{(pid=81767) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 15:47:30.497985 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG dbcounter [-] [81767] Writing DB stats keystone:SELECT=100 {{(pid=81767) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 15:47:30.511104 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=100 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 15:47:30.541368 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b27d8ce-f0c6-435b-858a-818a0d7c80f4 None admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 15:47:30.557104 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0420f041-1a93-4d24-9510-c4a15a228ad5 None admin] Validating token access rules against request {{(pid=81767) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 15:47:30.574987 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1dbdee5-3321-4304-91bf-077e8379e086 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 15:47:30.575645 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1dbdee5-3321-4304-91bf-077e8379e086 None admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:47:30.576201 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1dbdee5-3321-4304-91bf-077e8379e086 None admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:47:30.576650 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1dbdee5-3321-4304-91bf-077e8379e086 None admin] PATH_INFO: `/v3/users/glance` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:47:30.584689 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1dbdee5-3321-4304-91bf-077e8379e086 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 15:47:30.586130 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1dbdee5-3321-4304-91bf-077e8379e086 None admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 15:47:30.593081 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0e9cdec-f8f5-4bce-ae9a-b45b6237d9cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81767) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 15:47:30.593617 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0e9cdec-f8f5-4bce-ae9a-b45b6237d9cd None admin] REQUEST_METHOD: `GET` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:47:30.593895 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0e9cdec-f8f5-4bce-ae9a-b45b6237d9cd None admin] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:47:30.594165 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0e9cdec-f8f5-4bce-ae9a-b45b6237d9cd None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:47:30.594911 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0e9cdec-f8f5-4bce-ae9a-b45b6237d9cd None admin] RBAC: Adding query filter params (role.id=a5fbba1dc615490a96cde27e3a102eb6, scope.project.id=c4ee0413f9ef4d4fb1f95a660babe090, user.id=6a09d128dee9438283633e51f4834643) {{(pid=81767) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 15:47:30.595212 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0e9cdec-f8f5-4bce-ae9a-b45b6237d9cd None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 15:47:30.596676 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0e9cdec-f8f5-4bce-ae9a-b45b6237d9cd None admin] RBAC: Authorization granted {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 15:47:30.598153 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application [None req-a1dbdee5-3321-4304-91bf-077e8379e086 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Aug 22 15:47:30.598153 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 15:47:30.598153 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 15:47:30.598153 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 15:47:30.598153 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 15:47:30.598153 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 15:47:30.598153 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 15:47:30.598153 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 15:47:30.598153 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 15:47:30.598153 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 15:47:30.598153 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 15:47:30.598153 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 15:47:30.598153 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 22 15:47:30.598153 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 22 15:47:30.598153 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 22 15:47:30.598153 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 22 15:47:30.599054 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 15:47:30.599054 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 15:47:30.599054 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 22 15:47:30.599054 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 22 15:47:30.599054 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 22 15:47:30.599054 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 22 15:47:30.599054 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 15:47:30.599054 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 15:47:30.599054 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 15:47:30.599054 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 15:47:30.599054 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 15:47:30.599054 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application with Lock( Aug 22 15:47:30.599054 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 15:47:30.599054 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self._enter() Aug 22 15:47:30.599054 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 15:47:30.599054 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 15:47:30.599054 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 15:47:30.599054 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self.creator() Aug 22 15:47:30.600147 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 15:47:30.600147 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application created_value = creator( Aug 22 15:47:30.600147 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 22 15:47:30.600147 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 22 15:47:30.600147 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 22 15:47:30.600147 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 22 15:47:30.600147 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 22 15:47:30.600147 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 22 15:47:30.600147 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Aug 22 15:47:30.600147 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application Aug 22 15:47:30.600147 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 294/581] 10.222.0.15 () {56 vars in 1099 bytes} [Mon Aug 22 15:47:30 2022] GET /identity/v3/users/glance => generated 84 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 15:47:30.600147 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1dbdee5-3321-4304-91bf-077e8379e086 None admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 15:47:30.605658 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 288/582] 10.222.0.15 () {56 vars in 1374 bytes} [Mon Aug 22 15:47:30 2022] GET /identity/v3/role_assignments?user.id=6a09d128dee9438283633e51f4834643&scope.project.id=c4ee0413f9ef4d4fb1f95a660babe090&role.id=a5fbba1dc615490a96cde27e3a102eb6 => generated 259 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 15:47:30.606453 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0e9cdec-f8f5-4bce-ae9a-b45b6237d9cd None admin] Authenticating user token {{(pid=81767) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 15:47:30.718129 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1dbdee5-3321-4304-91bf-077e8379e086 None admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 15:47:30.729782 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0e9cdec-f8f5-4bce-ae9a-b45b6237d9cd None admin] Validating token access rules against request {{(pid=81767) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 15:47:30.738981 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG dbcounter [-] [81767] Writing DB stats keystone:SELECT=100 {{(pid=81767) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 15:47:30.751465 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=100 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 15:47:30.756191 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45fd2444-9393-40ab-a48c-b9f16d82642d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 15:47:30.756795 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45fd2444-9393-40ab-a48c-b9f16d82642d None admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:47:30.757077 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45fd2444-9393-40ab-a48c-b9f16d82642d None admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:47:30.757350 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45fd2444-9393-40ab-a48c-b9f16d82642d None admin] PATH_INFO: `/v3/projects` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:47:30.758043 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45fd2444-9393-40ab-a48c-b9f16d82642d None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81768) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 15:47:30.758347 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45fd2444-9393-40ab-a48c-b9f16d82642d None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 15:47:30.759840 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45fd2444-9393-40ab-a48c-b9f16d82642d None admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 15:47:30.770470 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26cd7e78-3a1e-4615-8de4-dc2d540f3869 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81767) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 15:47:30.771158 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26cd7e78-3a1e-4615-8de4-dc2d540f3869 None admin] REQUEST_METHOD: `GET` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:47:30.771475 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26cd7e78-3a1e-4615-8de4-dc2d540f3869 None admin] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:47:30.771745 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26cd7e78-3a1e-4615-8de4-dc2d540f3869 None admin] PATH_INFO: `/v3/roles` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:47:30.772409 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26cd7e78-3a1e-4615-8de4-dc2d540f3869 None admin] RBAC: Adding query filter params (name=service) {{(pid=81767) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 15:47:30.772938 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26cd7e78-3a1e-4615-8de4-dc2d540f3869 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 15:47:30.773041 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 295/583] 10.222.0.15 () {56 vars in 1148 bytes} [Mon Aug 22 15:47:30 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 15:47:30.775456 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45fd2444-9393-40ab-a48c-b9f16d82642d None admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 15:47:30.775593 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26cd7e78-3a1e-4615-8de4-dc2d540f3869 None admin] RBAC: Authorization granted {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 15:47:30.786555 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 289/584] 10.222.0.15 () {56 vars in 1103 bytes} [Mon Aug 22 15:47:30 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 15:47:30.788135 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26cd7e78-3a1e-4615-8de4-dc2d540f3869 None admin] Authenticating user token {{(pid=81767) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 15:47:30.898150 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26cd7e78-3a1e-4615-8de4-dc2d540f3869 None admin] Validating token access rules against request {{(pid=81767) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 15:47:30.908917 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45fd2444-9393-40ab-a48c-b9f16d82642d None admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 15:47:30.942697 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef28e267-4e35-4125-a867-c346fa5f055b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81767) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 15:47:30.943329 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef28e267-4e35-4125-a867-c346fa5f055b None admin] REQUEST_METHOD: `GET` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:47:30.943615 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef28e267-4e35-4125-a867-c346fa5f055b None admin] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:47:30.943893 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef28e267-4e35-4125-a867-c346fa5f055b None admin] PATH_INFO: `/v3/users` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:47:30.944631 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef28e267-4e35-4125-a867-c346fa5f055b None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=81767) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 15:47:30.945048 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef28e267-4e35-4125-a867-c346fa5f055b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 15:47:30.946489 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef28e267-4e35-4125-a867-c346fa5f055b None admin] RBAC: Authorization granted {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 15:47:30.949395 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52ba7d2e-8cfe-46d0-8501-d6fb1c4203d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 15:47:30.950031 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52ba7d2e-8cfe-46d0-8501-d6fb1c4203d6 None admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:47:30.950310 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52ba7d2e-8cfe-46d0-8501-d6fb1c4203d6 None admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:47:30.950579 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52ba7d2e-8cfe-46d0-8501-d6fb1c4203d6 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:47:30.958336 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52ba7d2e-8cfe-46d0-8501-d6fb1c4203d6 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 15:47:30.959820 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52ba7d2e-8cfe-46d0-8501-d6fb1c4203d6 None admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 15:47:30.970033 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 290/585] 10.222.0.15 () {56 vars in 1137 bytes} [Mon Aug 22 15:47:30 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 15:47:30.970033 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef28e267-4e35-4125-a867-c346fa5f055b None admin] Authenticating user token {{(pid=81767) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 15:47:30.977898 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application [None req-52ba7d2e-8cfe-46d0-8501-d6fb1c4203d6 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Aug 22 15:47:30.977898 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 15:47:30.977898 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 15:47:30.977898 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 15:47:30.977898 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 15:47:30.977898 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 15:47:30.977898 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 15:47:30.977898 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 15:47:30.977898 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 15:47:30.977898 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 15:47:30.977898 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 15:47:30.977898 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 15:47:30.977898 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 22 15:47:30.977898 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 22 15:47:30.977898 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 22 15:47:30.977898 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 22 15:47:30.979015 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 15:47:30.979015 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 15:47:30.979015 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 22 15:47:30.979015 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 22 15:47:30.979015 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 22 15:47:30.979015 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 22 15:47:30.979015 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 15:47:30.979015 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 15:47:30.979015 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 15:47:30.979015 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 15:47:30.979015 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 15:47:30.979015 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application with Lock( Aug 22 15:47:30.979015 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 15:47:30.979015 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self._enter() Aug 22 15:47:30.979015 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 15:47:30.979015 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 15:47:30.979015 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 15:47:30.979015 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self.creator() Aug 22 15:47:30.982860 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 15:47:30.982860 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application created_value = creator( Aug 22 15:47:30.982860 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 22 15:47:30.982860 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 22 15:47:30.982860 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 22 15:47:30.982860 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 22 15:47:30.982860 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 22 15:47:30.982860 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 22 15:47:30.982860 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Aug 22 15:47:30.982860 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application Aug 22 15:47:30.982860 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 296/586] 10.222.0.15 () {56 vars in 1099 bytes} [Mon Aug 22 15:47:30 2022] GET /identity/v3/users/cinder => generated 84 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 15:47:30.982860 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52ba7d2e-8cfe-46d0-8501-d6fb1c4203d6 None admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 15:47:31.002355 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG dbcounter [-] [81767] Writing DB stats keystone:SELECT=100 {{(pid=81767) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 15:47:31.045477 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=100 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 15:47:31.091490 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef28e267-4e35-4125-a867-c346fa5f055b None admin] Validating token access rules against request {{(pid=81767) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 15:47:31.095851 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52ba7d2e-8cfe-46d0-8501-d6fb1c4203d6 None admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 15:47:31.138652 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac0f661f-e794-4090-8786-d4ad5c961f0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 15:47:31.138930 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a77f559e-93bc-497a-a502-2d55db4f351d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81767) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 15:47:31.139320 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac0f661f-e794-4090-8786-d4ad5c961f0e None admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:47:31.139592 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac0f661f-e794-4090-8786-d4ad5c961f0e None admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:47:31.139863 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac0f661f-e794-4090-8786-d4ad5c961f0e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:47:31.140309 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a77f559e-93bc-497a-a502-2d55db4f351d None admin] REQUEST_METHOD: `PUT` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:47:31.140833 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a77f559e-93bc-497a-a502-2d55db4f351d None admin] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:47:31.141294 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a77f559e-93bc-497a-a502-2d55db4f351d None admin] PATH_INFO: `/v3/projects/c4ee0413f9ef4d4fb1f95a660babe090/users/e4ef569e41fb46d0b721bb1c7663616f/roles/a5fbba1dc615490a96cde27e3a102eb6` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:47:31.151237 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac0f661f-e794-4090-8786-d4ad5c961f0e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 15:47:31.152858 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac0f661f-e794-4090-8786-d4ad5c961f0e None admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 15:47:31.160741 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a77f559e-93bc-497a-a502-2d55db4f351d None admin] RBAC: Authorizing `identity:create_grant(project_id=c4ee0413f9ef4d4fb1f95a660babe090, user_id=e4ef569e41fb46d0b721bb1c7663616f, role_id=a5fbba1dc615490a96cde27e3a102eb6)` {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 15:47:31.162400 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a77f559e-93bc-497a-a502-2d55db4f351d None admin] RBAC: Authorization granted {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 15:47:31.167354 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application [None req-ac0f661f-e794-4090-8786-d4ad5c961f0e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 22 15:47:31.167354 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 15:47:31.167354 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 15:47:31.167354 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 15:47:31.167354 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 15:47:31.167354 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 15:47:31.167354 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 15:47:31.167354 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 15:47:31.167354 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 15:47:31.167354 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 15:47:31.167354 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 15:47:31.167354 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 15:47:31.167354 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 22 15:47:31.167354 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 22 15:47:31.167354 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 22 15:47:31.167354 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 22 15:47:31.168470 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 15:47:31.168470 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 15:47:31.168470 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 15:47:31.168470 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 15:47:31.168470 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 15:47:31.168470 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 15:47:31.168470 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 15:47:31.168470 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application with Lock( Aug 22 15:47:31.168470 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 15:47:31.168470 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self._enter() Aug 22 15:47:31.168470 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 15:47:31.168470 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 15:47:31.168470 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 15:47:31.168470 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self.creator() Aug 22 15:47:31.168470 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 15:47:31.168470 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application created_value = creator( Aug 22 15:47:31.168470 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 22 15:47:31.179969 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 22 15:47:31.179969 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 22 15:47:31.179969 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application Aug 22 15:47:31.179969 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 297/587] 10.222.0.15 () {56 vars in 1108 bytes} [Mon Aug 22 15:47:30 2022] GET /identity/v3/domains/Default => generated 87 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 15:47:31.179969 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac0f661f-e794-4090-8786-d4ad5c961f0e None admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 15:47:31.251293 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 291/588] 10.222.0.15 () {58 vars in 1439 bytes} [Mon Aug 22 15:47:30 2022] PUT /identity/v3/projects/c4ee0413f9ef4d4fb1f95a660babe090/users/e4ef569e41fb46d0b721bb1c7663616f/roles/a5fbba1dc615490a96cde27e3a102eb6 => generated 0 bytes in 283 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 15:47:31.252201 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a77f559e-93bc-497a-a502-2d55db4f351d None admin] Authenticating user token {{(pid=81767) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 15:47:31.300706 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac0f661f-e794-4090-8786-d4ad5c961f0e None admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 15:47:31.306468 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=100 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 15:47:31.356354 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-077feea2-15cb-48b9-9454-ed7d4c4b3c76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 15:47:31.357173 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-077feea2-15cb-48b9-9454-ed7d4c4b3c76 None admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:47:31.357686 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-077feea2-15cb-48b9-9454-ed7d4c4b3c76 None admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:47:31.358112 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-077feea2-15cb-48b9-9454-ed7d4c4b3c76 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:47:31.373907 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG dbcounter [-] [81767] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81767) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 15:47:31.376234 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-077feea2-15cb-48b9-9454-ed7d4c4b3c76 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 15:47:31.378643 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-077feea2-15cb-48b9-9454-ed7d4c4b3c76 None admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 15:47:31.399928 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application [None req-077feea2-15cb-48b9-9454-ed7d4c4b3c76 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 22 15:47:31.399928 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 15:47:31.399928 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 15:47:31.399928 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 15:47:31.399928 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 15:47:31.399928 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 15:47:31.399928 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 15:47:31.399928 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 15:47:31.399928 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 15:47:31.399928 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 15:47:31.399928 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 15:47:31.399928 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 15:47:31.399928 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 22 15:47:31.399928 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 22 15:47:31.399928 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 22 15:47:31.399928 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 22 15:47:31.400948 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 15:47:31.400948 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 15:47:31.400948 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 15:47:31.400948 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 15:47:31.400948 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 15:47:31.400948 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 15:47:31.400948 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 15:47:31.400948 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application with Lock( Aug 22 15:47:31.400948 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 15:47:31.400948 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self._enter() Aug 22 15:47:31.400948 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 15:47:31.400948 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 15:47:31.400948 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 15:47:31.400948 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self.creator() Aug 22 15:47:31.400948 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 15:47:31.400948 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application created_value = creator( Aug 22 15:47:31.400948 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 22 15:47:31.401906 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 22 15:47:31.401906 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 22 15:47:31.401906 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application Aug 22 15:47:31.401906 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 298/589] 10.222.0.15 () {56 vars in 1108 bytes} [Mon Aug 22 15:47:31 2022] GET /identity/v3/domains/Default => generated 87 bytes in 232 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 15:47:31.402242 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-077feea2-15cb-48b9-9454-ed7d4c4b3c76 None admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 15:47:31.410598 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a77f559e-93bc-497a-a502-2d55db4f351d None admin] Validating token access rules against request {{(pid=81767) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 15:47:31.451867 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35424906-e220-4ba3-a573-8c3f2d78f69f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81767) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 15:47:31.452589 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35424906-e220-4ba3-a573-8c3f2d78f69f None admin] REQUEST_METHOD: `GET` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:47:31.452957 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35424906-e220-4ba3-a573-8c3f2d78f69f None admin] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:47:31.453360 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35424906-e220-4ba3-a573-8c3f2d78f69f None admin] PATH_INFO: `/v3/users` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:47:31.454164 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35424906-e220-4ba3-a573-8c3f2d78f69f None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=81767) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 15:47:31.454479 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35424906-e220-4ba3-a573-8c3f2d78f69f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 15:47:31.455968 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35424906-e220-4ba3-a573-8c3f2d78f69f None admin] RBAC: Authorization granted {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 15:47:31.477048 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 292/590] 10.222.0.15 () {56 vars in 1137 bytes} [Mon Aug 22 15:47:31 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 15:47:31.477973 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35424906-e220-4ba3-a573-8c3f2d78f69f None admin] Authenticating user token {{(pid=81767) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 15:47:31.530984 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-077feea2-15cb-48b9-9454-ed7d4c4b3c76 None admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 15:47:31.570623 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f4e08b0-135f-4957-8d82-b8bca452d539 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 15:47:31.571229 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f4e08b0-135f-4957-8d82-b8bca452d539 None admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:47:31.571545 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f4e08b0-135f-4957-8d82-b8bca452d539 None admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:47:31.571813 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f4e08b0-135f-4957-8d82-b8bca452d539 None admin] PATH_INFO: `/v3/domains` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:47:31.572453 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f4e08b0-135f-4957-8d82-b8bca452d539 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81768) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 15:47:31.572749 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f4e08b0-135f-4957-8d82-b8bca452d539 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 15:47:31.574166 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f4e08b0-135f-4957-8d82-b8bca452d539 None admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 15:47:31.591637 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 299/591] 10.222.0.15 () {56 vars in 1109 bytes} [Mon Aug 22 15:47:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 15:47:31.591637 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f4e08b0-135f-4957-8d82-b8bca452d539 None admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 15:47:31.617999 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=100 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 15:47:31.626312 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35424906-e220-4ba3-a573-8c3f2d78f69f None admin] Validating token access rules against request {{(pid=81767) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 15:47:31.672015 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08366ce1-38f2-45d2-8a3c-c7ecaafb6831 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81767) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 15:47:31.673201 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08366ce1-38f2-45d2-8a3c-c7ecaafb6831 None admin] REQUEST_METHOD: `GET` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:47:31.673877 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08366ce1-38f2-45d2-8a3c-c7ecaafb6831 None admin] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:47:31.674384 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08366ce1-38f2-45d2-8a3c-c7ecaafb6831 None admin] PATH_INFO: `/v3/domains` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:47:31.675894 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08366ce1-38f2-45d2-8a3c-c7ecaafb6831 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81767) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 15:47:31.676402 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08366ce1-38f2-45d2-8a3c-c7ecaafb6831 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 15:47:31.677931 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08366ce1-38f2-45d2-8a3c-c7ecaafb6831 None admin] RBAC: Authorization granted {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 15:47:31.680442 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG dbcounter [-] [81767] Writing DB stats keystone:SELECT=100 {{(pid=81767) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 15:47:31.694489 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 293/592] 10.222.0.15 () {56 vars in 1109 bytes} [Mon Aug 22 15:47:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 15:47:31.695807 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08366ce1-38f2-45d2-8a3c-c7ecaafb6831 None admin] Authenticating user token {{(pid=81767) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 15:47:31.722264 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f4e08b0-135f-4957-8d82-b8bca452d539 None admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 15:47:31.759569 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45ec390d-96ad-46c5-865d-d96fcb168b02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 15:47:31.760395 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ec390d-96ad-46c5-865d-d96fcb168b02 None admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:47:31.760829 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ec390d-96ad-46c5-865d-d96fcb168b02 None admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:47:31.761163 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ec390d-96ad-46c5-865d-d96fcb168b02 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:47:31.773694 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45ec390d-96ad-46c5-865d-d96fcb168b02 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 15:47:31.775380 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45ec390d-96ad-46c5-865d-d96fcb168b02 None admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 15:47:31.790832 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application [None req-45ec390d-96ad-46c5-865d-d96fcb168b02 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 22 15:47:31.790832 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 15:47:31.790832 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 15:47:31.790832 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 15:47:31.790832 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 15:47:31.790832 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 15:47:31.790832 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 15:47:31.790832 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 15:47:31.790832 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 15:47:31.790832 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 15:47:31.790832 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 15:47:31.790832 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 15:47:31.790832 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 22 15:47:31.790832 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 22 15:47:31.790832 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 22 15:47:31.790832 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 22 15:47:31.791958 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 15:47:31.791958 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 15:47:31.791958 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 15:47:31.791958 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 15:47:31.791958 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 15:47:31.791958 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 15:47:31.791958 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 15:47:31.791958 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application with Lock( Aug 22 15:47:31.791958 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 15:47:31.791958 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self._enter() Aug 22 15:47:31.791958 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 15:47:31.791958 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 15:47:31.791958 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 15:47:31.791958 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self.creator() Aug 22 15:47:31.791958 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 15:47:31.791958 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application created_value = creator( Aug 22 15:47:31.791958 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 22 15:47:31.792831 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 22 15:47:31.792831 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 22 15:47:31.792831 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application Aug 22 15:47:31.797215 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 300/593] 10.222.0.15 () {56 vars in 1108 bytes} [Mon Aug 22 15:47:31 2022] GET /identity/v3/domains/Default => generated 87 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 15:47:31.798136 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45ec390d-96ad-46c5-865d-d96fcb168b02 None admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 15:47:31.816023 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08366ce1-38f2-45d2-8a3c-c7ecaafb6831 None admin] Validating token access rules against request {{(pid=81767) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 15:47:31.853954 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d06178bf-3f5c-4af6-85dc-9f6f9104c47b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81767) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 15:47:31.855051 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d06178bf-3f5c-4af6-85dc-9f6f9104c47b None admin] REQUEST_METHOD: `GET` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:47:31.855685 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d06178bf-3f5c-4af6-85dc-9f6f9104c47b None admin] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:47:31.856175 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d06178bf-3f5c-4af6-85dc-9f6f9104c47b None admin] PATH_INFO: `/v3/users/neutron` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:47:31.865345 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d06178bf-3f5c-4af6-85dc-9f6f9104c47b None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 15:47:31.868055 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d06178bf-3f5c-4af6-85dc-9f6f9104c47b None admin] RBAC: Authorization granted {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 15:47:31.868326 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=100 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 15:47:31.880547 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application [None req-d06178bf-3f5c-4af6-85dc-9f6f9104c47b None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Aug 22 15:47:31.880547 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 15:47:31.880547 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 15:47:31.880547 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 15:47:31.880547 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 15:47:31.880547 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 15:47:31.880547 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 15:47:31.880547 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 15:47:31.880547 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 15:47:31.880547 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 15:47:31.880547 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 15:47:31.880547 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 15:47:31.880547 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 22 15:47:31.880547 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 22 15:47:31.880547 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 22 15:47:31.880547 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 22 15:47:31.881713 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 15:47:31.881713 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 15:47:31.881713 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 22 15:47:31.881713 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 22 15:47:31.881713 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 22 15:47:31.881713 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 22 15:47:31.881713 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 15:47:31.881713 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 15:47:31.881713 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 15:47:31.881713 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 15:47:31.881713 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 15:47:31.881713 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application with Lock( Aug 22 15:47:31.881713 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 15:47:31.881713 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return self._enter() Aug 22 15:47:31.881713 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 15:47:31.881713 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 15:47:31.881713 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 15:47:31.881713 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return self.creator() Aug 22 15:47:31.882847 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 15:47:31.882847 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application created_value = creator( Aug 22 15:47:31.882847 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 22 15:47:31.882847 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 22 15:47:31.882847 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 22 15:47:31.882847 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 22 15:47:31.882847 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 22 15:47:31.882847 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 22 15:47:31.882847 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Aug 22 15:47:31.882847 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application Aug 22 15:47:31.889344 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 294/594] 10.222.0.15 () {56 vars in 1102 bytes} [Mon Aug 22 15:47:31 2022] GET /identity/v3/users/neutron => generated 85 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 15:47:31.890318 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d06178bf-3f5c-4af6-85dc-9f6f9104c47b None admin] Authenticating user token {{(pid=81767) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 15:47:31.910065 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45ec390d-96ad-46c5-865d-d96fcb168b02 None admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 15:47:31.949277 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31f3f90a-e465-47e6-9a08-0b31ac2d69ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 15:47:31.950191 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31f3f90a-e465-47e6-9a08-0b31ac2d69ea None admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:47:31.950642 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31f3f90a-e465-47e6-9a08-0b31ac2d69ea None admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:47:31.951168 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31f3f90a-e465-47e6-9a08-0b31ac2d69ea None admin] PATH_INFO: `/v3/projects/service` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:47:31.956015 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG dbcounter [-] [81767] Writing DB stats keystone:SELECT=100 {{(pid=81767) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 15:47:31.958905 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31f3f90a-e465-47e6-9a08-0b31ac2d69ea None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 15:47:31.960574 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31f3f90a-e465-47e6-9a08-0b31ac2d69ea None admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 15:47:31.973253 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application [None req-31f3f90a-e465-47e6-9a08-0b31ac2d69ea None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 22 15:47:31.973253 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 15:47:31.973253 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 15:47:31.973253 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 15:47:31.973253 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 15:47:31.973253 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 15:47:31.973253 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 15:47:31.973253 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 15:47:31.973253 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 15:47:31.973253 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 15:47:31.973253 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 15:47:31.973253 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 15:47:31.973253 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 22 15:47:31.973253 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 22 15:47:31.973253 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 22 15:47:31.974223 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 22 15:47:31.974223 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 15:47:31.974223 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 15:47:31.974223 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 15:47:31.974223 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 15:47:31.974223 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 15:47:31.974223 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 15:47:31.974223 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 15:47:31.974223 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application with Lock( Aug 22 15:47:31.974223 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 15:47:31.974223 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self._enter() Aug 22 15:47:31.974223 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 15:47:31.974223 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 15:47:31.974223 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 15:47:31.974223 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self.creator() Aug 22 15:47:31.974223 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 15:47:31.974223 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application created_value = creator( Aug 22 15:47:31.975244 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 22 15:47:31.975244 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 22 15:47:31.975244 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 22 15:47:31.975244 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 22 15:47:31.975244 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 22 15:47:31.975244 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 22 15:47:31.975244 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 22 15:47:31.975244 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application Aug 22 15:47:31.978305 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 301/595] 10.222.0.15 () {56 vars in 1111 bytes} [Mon Aug 22 15:47:31 2022] GET /identity/v3/projects/service => generated 88 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 15:47:31.980307 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31f3f90a-e465-47e6-9a08-0b31ac2d69ea None admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 15:47:32.009037 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d06178bf-3f5c-4af6-85dc-9f6f9104c47b None admin] Validating token access rules against request {{(pid=81767) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 15:47:32.048633 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0430dc10-7ae1-4f71-aca3-c4b6f0fbc87a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81767) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 15:47:32.049378 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0430dc10-7ae1-4f71-aca3-c4b6f0fbc87a None admin] REQUEST_METHOD: `GET` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:47:32.049753 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0430dc10-7ae1-4f71-aca3-c4b6f0fbc87a None admin] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:47:32.050095 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0430dc10-7ae1-4f71-aca3-c4b6f0fbc87a None admin] PATH_INFO: `/v3/domains` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:47:32.050838 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0430dc10-7ae1-4f71-aca3-c4b6f0fbc87a None admin] RBAC: Adding query filter params (name=Default) {{(pid=81767) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 15:47:32.051213 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0430dc10-7ae1-4f71-aca3-c4b6f0fbc87a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 15:47:32.052694 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0430dc10-7ae1-4f71-aca3-c4b6f0fbc87a None admin] RBAC: Authorization granted {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 15:47:32.071350 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 295/596] 10.222.0.15 () {56 vars in 1109 bytes} [Mon Aug 22 15:47:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 15:47:32.071350 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0430dc10-7ae1-4f71-aca3-c4b6f0fbc87a None admin] Authenticating user token {{(pid=81767) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 15:47:32.097444 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31f3f90a-e465-47e6-9a08-0b31ac2d69ea None admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 15:47:32.125509 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=100 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 15:47:32.134353 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6d8659c-cd89-40b6-a8c1-d75bdb761f1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 15:47:32.135307 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d8659c-cd89-40b6-a8c1-d75bdb761f1a None admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:47:32.136267 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d8659c-cd89-40b6-a8c1-d75bdb761f1a None admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:47:32.142808 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d8659c-cd89-40b6-a8c1-d75bdb761f1a None admin] PATH_INFO: `/v3/users` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:47:32.142808 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6d8659c-cd89-40b6-a8c1-d75bdb761f1a None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=81768) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 15:47:32.142808 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6d8659c-cd89-40b6-a8c1-d75bdb761f1a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 15:47:32.142808 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6d8659c-cd89-40b6-a8c1-d75bdb761f1a None admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 15:47:32.161919 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 302/597] 10.222.0.15 () {56 vars in 1139 bytes} [Mon Aug 22 15:47:31 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 15:47:32.163175 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6d8659c-cd89-40b6-a8c1-d75bdb761f1a None admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 15:47:32.183477 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0430dc10-7ae1-4f71-aca3-c4b6f0fbc87a None admin] Validating token access rules against request {{(pid=81767) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 15:47:32.200572 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG dbcounter [-] [81767] Writing DB stats keystone:SELECT=100 {{(pid=81767) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 15:47:32.223355 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d44a909f-17e1-4982-8f5e-de435878366c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81767) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 15:47:32.223987 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d44a909f-17e1-4982-8f5e-de435878366c None admin] REQUEST_METHOD: `GET` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:47:32.224267 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d44a909f-17e1-4982-8f5e-de435878366c None admin] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:47:32.224539 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d44a909f-17e1-4982-8f5e-de435878366c None admin] PATH_INFO: `/v3/projects` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:47:32.225238 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d44a909f-17e1-4982-8f5e-de435878366c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81767) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 15:47:32.225645 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d44a909f-17e1-4982-8f5e-de435878366c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 15:47:32.227250 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d44a909f-17e1-4982-8f5e-de435878366c None admin] RBAC: Authorization granted {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 15:47:32.241586 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 296/598] 10.222.0.15 () {56 vars in 1148 bytes} [Mon Aug 22 15:47:32 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 15:47:32.242967 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d44a909f-17e1-4982-8f5e-de435878366c None admin] Authenticating user token {{(pid=81767) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 15:47:32.275075 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6d8659c-cd89-40b6-a8c1-d75bdb761f1a None admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 15:47:32.313942 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d709c8c-65a9-4dda-b45c-8158ab83247e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 15:47:32.315048 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d709c8c-65a9-4dda-b45c-8158ab83247e None admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:47:32.315722 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d709c8c-65a9-4dda-b45c-8158ab83247e None admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:47:32.316222 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d709c8c-65a9-4dda-b45c-8158ab83247e None admin] PATH_INFO: `/v3/projects/service` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:47:32.324383 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d709c8c-65a9-4dda-b45c-8158ab83247e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 15:47:32.326139 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d709c8c-65a9-4dda-b45c-8158ab83247e None admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 15:47:32.338114 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application [None req-4d709c8c-65a9-4dda-b45c-8158ab83247e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 22 15:47:32.338114 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 15:47:32.338114 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 15:47:32.338114 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 15:47:32.338114 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 15:47:32.338114 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 15:47:32.338114 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 15:47:32.338114 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 15:47:32.338114 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 15:47:32.338114 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 15:47:32.338114 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 15:47:32.338114 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 15:47:32.338114 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 22 15:47:32.338114 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 22 15:47:32.338114 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 22 15:47:32.339163 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 22 15:47:32.339163 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 15:47:32.339163 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 15:47:32.339163 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 15:47:32.339163 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 15:47:32.339163 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 15:47:32.339163 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 15:47:32.339163 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 15:47:32.339163 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application with Lock( Aug 22 15:47:32.339163 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 15:47:32.339163 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self._enter() Aug 22 15:47:32.339163 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 15:47:32.339163 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 15:47:32.339163 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 15:47:32.339163 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self.creator() Aug 22 15:47:32.339163 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 15:47:32.339163 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application created_value = creator( Aug 22 15:47:32.340191 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 22 15:47:32.340191 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 22 15:47:32.340191 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 22 15:47:32.340191 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 22 15:47:32.340191 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 22 15:47:32.340191 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 22 15:47:32.340191 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 22 15:47:32.340191 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application Aug 22 15:47:32.347567 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 303/599] 10.222.0.15 () {56 vars in 1111 bytes} [Mon Aug 22 15:47:32 2022] GET /identity/v3/projects/service => generated 88 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 15:47:32.348692 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d709c8c-65a9-4dda-b45c-8158ab83247e None admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 15:47:32.360624 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d44a909f-17e1-4982-8f5e-de435878366c None admin] Validating token access rules against request {{(pid=81767) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 15:47:32.399655 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=100 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 15:47:32.403034 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-103c3a79-7dd9-449b-af63-2da921b746c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81767) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 15:47:32.403899 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-103c3a79-7dd9-449b-af63-2da921b746c4 None admin] REQUEST_METHOD: `GET` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:47:32.404419 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-103c3a79-7dd9-449b-af63-2da921b746c4 None admin] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:47:32.404978 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-103c3a79-7dd9-449b-af63-2da921b746c4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:47:32.418783 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-103c3a79-7dd9-449b-af63-2da921b746c4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 15:47:32.420744 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-103c3a79-7dd9-449b-af63-2da921b746c4 None admin] RBAC: Authorization granted {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 15:47:32.439460 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application [None req-103c3a79-7dd9-449b-af63-2da921b746c4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 22 15:47:32.439460 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 15:47:32.439460 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 15:47:32.439460 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 15:47:32.439460 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 15:47:32.439460 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 15:47:32.439460 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 15:47:32.439460 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 15:47:32.439460 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 15:47:32.439460 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 15:47:32.439460 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 15:47:32.439460 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 15:47:32.439460 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 22 15:47:32.439460 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 22 15:47:32.439460 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 22 15:47:32.439460 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 22 15:47:32.440494 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 15:47:32.440494 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 15:47:32.440494 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 15:47:32.440494 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 15:47:32.440494 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 15:47:32.440494 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 15:47:32.440494 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 15:47:32.440494 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application with Lock( Aug 22 15:47:32.440494 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 15:47:32.440494 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return self._enter() Aug 22 15:47:32.440494 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 15:47:32.440494 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 15:47:32.440494 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 15:47:32.440494 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return self.creator() Aug 22 15:47:32.440494 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 15:47:32.440494 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application created_value = creator( Aug 22 15:47:32.440494 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 22 15:47:32.441417 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 22 15:47:32.441417 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 22 15:47:32.441417 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application Aug 22 15:47:32.443571 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 297/600] 10.222.0.15 () {56 vars in 1108 bytes} [Mon Aug 22 15:47:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 15:47:32.445508 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-103c3a79-7dd9-449b-af63-2da921b746c4 None admin] Authenticating user token {{(pid=81767) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 15:47:32.462122 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d709c8c-65a9-4dda-b45c-8158ab83247e None admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 15:47:32.478817 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG dbcounter [-] [81767] Writing DB stats keystone:SELECT=100 {{(pid=81767) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 15:47:32.501146 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54a927bf-f4c9-4df3-8dc1-de804d480671 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 15:47:32.501947 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54a927bf-f4c9-4df3-8dc1-de804d480671 None admin] REQUEST_METHOD: `PUT` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:47:32.502396 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54a927bf-f4c9-4df3-8dc1-de804d480671 None admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:47:32.502687 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54a927bf-f4c9-4df3-8dc1-de804d480671 None admin] PATH_INFO: `/v3/projects/c4ee0413f9ef4d4fb1f95a660babe090/users/91f565473c824a15930e2b12863c1ed3/roles/a5fbba1dc615490a96cde27e3a102eb6` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:47:32.519796 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54a927bf-f4c9-4df3-8dc1-de804d480671 None admin] RBAC: Authorizing `identity:create_grant(project_id=c4ee0413f9ef4d4fb1f95a660babe090, user_id=91f565473c824a15930e2b12863c1ed3, role_id=a5fbba1dc615490a96cde27e3a102eb6)` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 15:47:32.521226 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54a927bf-f4c9-4df3-8dc1-de804d480671 None admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 15:47:32.564684 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-103c3a79-7dd9-449b-af63-2da921b746c4 None admin] Validating token access rules against request {{(pid=81767) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 15:47:32.602896 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0418aa55-ae41-4871-b455-9c3a4c467ca1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81767) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 15:47:32.603526 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0418aa55-ae41-4871-b455-9c3a4c467ca1 None admin] REQUEST_METHOD: `GET` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:47:32.603807 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0418aa55-ae41-4871-b455-9c3a4c467ca1 None admin] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:47:32.604076 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0418aa55-ae41-4871-b455-9c3a4c467ca1 None admin] PATH_INFO: `/v3/projects` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:47:32.604793 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0418aa55-ae41-4871-b455-9c3a4c467ca1 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81767) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 15:47:32.605101 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0418aa55-ae41-4871-b455-9c3a4c467ca1 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 15:47:32.606612 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0418aa55-ae41-4871-b455-9c3a4c467ca1 None admin] RBAC: Authorization granted {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 15:47:32.618754 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 298/601] 10.222.0.15 () {56 vars in 1148 bytes} [Mon Aug 22 15:47:32 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 15:47:32.619664 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0418aa55-ae41-4871-b455-9c3a4c467ca1 None admin] Authenticating user token {{(pid=81767) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 15:47:32.626363 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 304/602] 10.222.0.15 () {58 vars in 1439 bytes} [Mon Aug 22 15:47:32 2022] PUT /identity/v3/projects/c4ee0413f9ef4d4fb1f95a660babe090/users/91f565473c824a15930e2b12863c1ed3/roles/a5fbba1dc615490a96cde27e3a102eb6 => generated 0 bytes in 278 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 15:47:32.627892 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b2c7ba2-a394-4ad8-b00b-d7ecdf0d02b4 None None] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:47:32.628325 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b2c7ba2-a394-4ad8-b00b-d7ecdf0d02b4 None None] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:47:32.628767 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b2c7ba2-a394-4ad8-b00b-d7ecdf0d02b4 None None] PATH_INFO: `/` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:47:32.631113 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 305/603] 10.222.0.15 () {54 vars in 927 bytes} [Mon Aug 22 15:47:32 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 15:47:32.632672 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b2c7ba2-a394-4ad8-b00b-d7ecdf0d02b4 None None] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 15:47:32.756371 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG dbcounter [-] [81767] Writing DB stats keystone:SELECT=100 {{(pid=81767) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 15:47:32.761117 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 15:47:32.769086 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b2c7ba2-a394-4ad8-b00b-d7ecdf0d02b4 None None] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 15:47:32.779907 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0418aa55-ae41-4871-b455-9c3a4c467ca1 None admin] Validating token access rules against request {{(pid=81767) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 15:47:32.814825 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ea2e273-9df4-4326-9345-93b388edf24f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 15:47:32.818349 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea2e273-9df4-4326-9345-93b388edf24f None admin] REQUEST_METHOD: `PUT` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:47:32.818349 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea2e273-9df4-4326-9345-93b388edf24f None admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:47:32.818349 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea2e273-9df4-4326-9345-93b388edf24f None admin] PATH_INFO: `/v3/projects/c4ee0413f9ef4d4fb1f95a660babe090/users/79706d480f864998b9f8594be407ceab/roles/a5fbba1dc615490a96cde27e3a102eb6` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:47:32.829924 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af9c0945-c1d6-4c98-a98a-dc31a7bd4147 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81767) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 15:47:32.830862 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af9c0945-c1d6-4c98-a98a-dc31a7bd4147 None admin] REQUEST_METHOD: `GET` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:47:32.831311 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af9c0945-c1d6-4c98-a98a-dc31a7bd4147 None admin] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:47:32.831745 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af9c0945-c1d6-4c98-a98a-dc31a7bd4147 None admin] PATH_INFO: `/v3/domains` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:47:32.832583 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af9c0945-c1d6-4c98-a98a-dc31a7bd4147 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81767) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 15:47:32.833119 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af9c0945-c1d6-4c98-a98a-dc31a7bd4147 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 15:47:32.834431 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af9c0945-c1d6-4c98-a98a-dc31a7bd4147 None admin] RBAC: Authorization granted {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 15:47:32.835784 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ea2e273-9df4-4326-9345-93b388edf24f None admin] RBAC: Authorizing `identity:create_grant(project_id=c4ee0413f9ef4d4fb1f95a660babe090, user_id=79706d480f864998b9f8594be407ceab, role_id=a5fbba1dc615490a96cde27e3a102eb6)` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 15:47:32.837258 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ea2e273-9df4-4326-9345-93b388edf24f None admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 15:47:32.851346 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 299/604] 10.222.0.15 () {56 vars in 1109 bytes} [Mon Aug 22 15:47:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 15:47:32.851346 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb045afd-0acf-4452-ab90-11cb3e0f9e7b None None] REQUEST_METHOD: `POST` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:47:32.851346 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb045afd-0acf-4452-ab90-11cb3e0f9e7b None None] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:47:32.851346 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb045afd-0acf-4452-ab90-11cb3e0f9e7b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:47:32.875049 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.auth.core [None req-cb045afd-0acf-4452-ab90-11cb3e0f9e7b None None] MFA Rules not processed for user `6b2199ae0fb0480aa917ada391f5ec54`. Rule list: `[]` (Enabled: `True`). {{(pid=81767) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 15:47:32.876321 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.fernet_utils [None req-cb045afd-0acf-4452-ab90-11cb3e0f9e7b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81767) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 15:47:32.959309 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 300/605] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Aug 22 15:47:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 22 15:47:32.959309 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb045afd-0acf-4452-ab90-11cb3e0f9e7b None None] Authenticating user token {{(pid=81767) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 15:47:33.012399 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 306/606] 10.222.0.15 () {58 vars in 1439 bytes} [Mon Aug 22 15:47:32 2022] PUT /identity/v3/projects/c4ee0413f9ef4d4fb1f95a660babe090/users/79706d480f864998b9f8594be407ceab/roles/a5fbba1dc615490a96cde27e3a102eb6 => generated 0 bytes in 374 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 15:47:33.154095 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG dbcounter [-] [81767] Writing DB stats keystone:SELECT=100 {{(pid=81767) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 15:47:33.188806 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb045afd-0acf-4452-ab90-11cb3e0f9e7b None None] Validating token access rules against request {{(pid=81767) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 15:47:33.227433 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ad860db-790a-4031-b36f-a932978392f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81767) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 15:47:33.228065 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad860db-790a-4031-b36f-a932978392f8 None admin] REQUEST_METHOD: `GET` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:47:33.228338 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad860db-790a-4031-b36f-a932978392f8 None admin] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:47:33.228604 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad860db-790a-4031-b36f-a932978392f8 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:47:33.236123 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ad860db-790a-4031-b36f-a932978392f8 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 15:47:33.237570 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ad860db-790a-4031-b36f-a932978392f8 None admin] RBAC: Authorization granted {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 15:47:33.250506 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application [None req-8ad860db-790a-4031-b36f-a932978392f8 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 22 15:47:33.250506 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 15:47:33.250506 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 15:47:33.250506 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 15:47:33.250506 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 15:47:33.250506 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 15:47:33.250506 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 15:47:33.250506 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 15:47:33.250506 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 15:47:33.250506 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 15:47:33.250506 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 15:47:33.250506 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 15:47:33.250506 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 22 15:47:33.250506 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 22 15:47:33.250506 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 22 15:47:33.251851 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 22 15:47:33.251851 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 15:47:33.251851 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 15:47:33.251851 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 15:47:33.251851 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 15:47:33.251851 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 15:47:33.251851 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 15:47:33.251851 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 15:47:33.251851 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application with Lock( Aug 22 15:47:33.251851 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 15:47:33.251851 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return self._enter() Aug 22 15:47:33.251851 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 15:47:33.251851 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 15:47:33.251851 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 15:47:33.251851 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return self.creator() Aug 22 15:47:33.251851 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 15:47:33.251851 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application created_value = creator( Aug 22 15:47:33.252889 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 22 15:47:33.252889 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 22 15:47:33.252889 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 22 15:47:33.252889 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 22 15:47:33.252889 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 22 15:47:33.252889 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 22 15:47:33.252889 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 22 15:47:33.252889 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application Aug 22 15:47:33.252889 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 301/607] 10.222.0.15 () {56 vars in 1111 bytes} [Mon Aug 22 15:47:32 2022] GET /identity/v3/projects/service => generated 88 bytes in 298 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 15:47:33.259747 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ea2e273-9df4-4326-9345-93b388edf24f None admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 15:47:33.319186 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ad860db-790a-4031-b36f-a932978392f8 None admin] Authenticating user token {{(pid=81767) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 15:47:33.402489 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ea2e273-9df4-4326-9345-93b388edf24f None admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 15:47:33.437634 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ad860db-790a-4031-b36f-a932978392f8 None admin] Validating token access rules against request {{(pid=81767) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 15:47:33.447125 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c0669b6-4709-4873-9782-ad31148a893f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 15:47:33.448179 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c0669b6-4709-4873-9782-ad31148a893f None admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:47:33.448458 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c0669b6-4709-4873-9782-ad31148a893f None admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:47:33.448763 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c0669b6-4709-4873-9782-ad31148a893f None admin] PATH_INFO: `/v3/projects` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:47:33.450023 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c0669b6-4709-4873-9782-ad31148a893f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81768) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 15:47:33.450451 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c0669b6-4709-4873-9782-ad31148a893f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 15:47:33.452104 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c0669b6-4709-4873-9782-ad31148a893f None admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 15:47:33.469058 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 307/608] 10.222.0.15 () {56 vars in 1148 bytes} [Mon Aug 22 15:47:33 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 22 15:47:33.471016 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e4e49a0-e7b6-4043-91e0-2155476bb495 None None] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:47:33.471319 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e4e49a0-e7b6-4043-91e0-2155476bb495 None None] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:47:33.471594 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e4e49a0-e7b6-4043-91e0-2155476bb495 None None] PATH_INFO: `/` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:47:33.473247 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 308/609] 10.222.0.15 () {54 vars in 927 bytes} [Mon Aug 22 15:47:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 15:47:33.474773 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b09d321-5dce-47f3-9f6f-8e43db09b8e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81767) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 15:47:33.475579 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b09d321-5dce-47f3-9f6f-8e43db09b8e0 None admin] REQUEST_METHOD: `GET` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:47:33.475691 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b09d321-5dce-47f3-9f6f-8e43db09b8e0 None admin] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:47:33.475961 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b09d321-5dce-47f3-9f6f-8e43db09b8e0 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:47:33.478171 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e4e49a0-e7b6-4043-91e0-2155476bb495 None None] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 15:47:33.480287 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG dbcounter [-] [81767] Writing DB stats keystone:SELECT=100 {{(pid=81767) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 15:47:33.489485 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 15:47:33.493289 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b09d321-5dce-47f3-9f6f-8e43db09b8e0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 22 15:47:33.497733 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application [None req-9b09d321-5dce-47f3-9f6f-8e43db09b8e0 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 22 15:47:33.497733 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 15:47:33.497733 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 15:47:33.497733 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 15:47:33.497733 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 15:47:33.497733 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 15:47:33.497733 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 15:47:33.497733 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 15:47:33.497733 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 15:47:33.497733 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 15:47:33.497733 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 15:47:33.497733 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 15:47:33.497733 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 22 15:47:33.497733 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 22 15:47:33.497733 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 22 15:47:33.497733 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 22 15:47:33.498793 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 22 15:47:33.498793 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 22 15:47:33.498793 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 22 15:47:33.498793 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 22 15:47:33.498793 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 15:47:33.498793 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 15:47:33.498793 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 15:47:33.498793 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 15:47:33.498793 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 15:47:33.498793 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 15:47:33.498793 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 15:47:33.498793 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application with Lock( Aug 22 15:47:33.498793 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 15:47:33.498793 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return self._enter() Aug 22 15:47:33.498793 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 15:47:33.498793 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 15:47:33.499787 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 15:47:33.499787 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return self.creator() Aug 22 15:47:33.499787 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 15:47:33.499787 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application created_value = creator( Aug 22 15:47:33.499787 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 22 15:47:33.499787 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 22 15:47:33.499787 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 22 15:47:33.499787 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 22 15:47:33.499787 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 22 15:47:33.499787 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 22 15:47:33.499787 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 22 15:47:33.499787 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application Aug 22 15:47:33.499787 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 302/610] 10.222.0.15 () {56 vars in 1102 bytes} [Mon Aug 22 15:47:33 2022] GET /identity/v3/roles/service => generated 85 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 22 15:47:33.501033 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea92ab4-2b6c-4a56-9167-ac4efeb56ec8 None None] REQUEST_METHOD: `POST` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:47:33.501815 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea92ab4-2b6c-4a56-9167-ac4efeb56ec8 None None] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:47:33.502108 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea92ab4-2b6c-4a56-9167-ac4efeb56ec8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:47:33.524292 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.auth.core [None req-cea92ab4-2b6c-4a56-9167-ac4efeb56ec8 None None] MFA Rules not processed for user `6b2199ae0fb0480aa917ada391f5ec54`. Rule list: `[]` (Enabled: `True`). {{(pid=81767) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 15:47:33.525644 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.fernet_utils [None req-cea92ab4-2b6c-4a56-9167-ac4efeb56ec8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81767) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 15:47:33.604065 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e4e49a0-e7b6-4043-91e0-2155476bb495 None None] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 15:47:33.626350 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 303/611] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Aug 22 15:47:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 22 15:47:33.626350 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cea92ab4-2b6c-4a56-9167-ac4efeb56ec8 None None] Authenticating user token {{(pid=81767) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 15:47:33.682823 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ab21aa0-7c19-46cc-aa23-281bfe6e3565 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 15:47:33.683519 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ab21aa0-7c19-46cc-aa23-281bfe6e3565 None admin] REQUEST_METHOD: `PUT` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:47:33.683801 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ab21aa0-7c19-46cc-aa23-281bfe6e3565 None admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:47:33.684076 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ab21aa0-7c19-46cc-aa23-281bfe6e3565 None admin] PATH_INFO: `/v3/projects/c4ee0413f9ef4d4fb1f95a660babe090/users/234687adad594f1aa45775a565ffbdc7/roles/a5fbba1dc615490a96cde27e3a102eb6` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:47:33.701699 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ab21aa0-7c19-46cc-aa23-281bfe6e3565 None admin] RBAC: Authorizing `identity:create_grant(project_id=c4ee0413f9ef4d4fb1f95a660babe090, user_id=234687adad594f1aa45775a565ffbdc7, role_id=a5fbba1dc615490a96cde27e3a102eb6)` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 15:47:33.703301 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ab21aa0-7c19-46cc-aa23-281bfe6e3565 None admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 15:47:33.757930 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cea92ab4-2b6c-4a56-9167-ac4efeb56ec8 None None] Validating token access rules against request {{(pid=81767) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 15:47:33.783585 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 309/612] 10.222.0.15 () {58 vars in 1439 bytes} [Mon Aug 22 15:47:33 2022] PUT /identity/v3/projects/c4ee0413f9ef4d4fb1f95a660babe090/users/234687adad594f1aa45775a565ffbdc7/roles/a5fbba1dc615490a96cde27e3a102eb6 => generated 0 bytes in 305 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 22 15:47:33.796304 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG dbcounter [-] [81767] Writing DB stats keystone:SELECT=100 {{(pid=81767) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 15:47:33.810269 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-632037cd-0b24-4171-81b2-536bb10c65b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81767) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 15:47:33.810887 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-632037cd-0b24-4171-81b2-536bb10c65b5 None admin] REQUEST_METHOD: `GET` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:47:33.811210 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-632037cd-0b24-4171-81b2-536bb10c65b5 None admin] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:47:33.811558 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-632037cd-0b24-4171-81b2-536bb10c65b5 None admin] PATH_INFO: `/v3/roles` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:47:33.812237 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-632037cd-0b24-4171-81b2-536bb10c65b5 None admin] RBAC: Adding query filter params (name=service) {{(pid=81767) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 15:47:33.812527 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-632037cd-0b24-4171-81b2-536bb10c65b5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 15:47:33.813889 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-632037cd-0b24-4171-81b2-536bb10c65b5 None admin] RBAC: Authorization granted {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 15:47:33.825015 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 304/613] 10.222.0.15 () {56 vars in 1103 bytes} [Mon Aug 22 15:47:33 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 15:47:33.831090 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ab21aa0-7c19-46cc-aa23-281bfe6e3565 None admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 15:47:33.926762 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 15:47:33.938536 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-632037cd-0b24-4171-81b2-536bb10c65b5 None admin] Authenticating user token {{(pid=81767) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 15:47:33.995737 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ab21aa0-7c19-46cc-aa23-281bfe6e3565 None admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 15:47:34.037671 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89b3749d-932e-4be3-99f1-f4347f0c9111 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 15:47:34.038304 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89b3749d-932e-4be3-99f1-f4347f0c9111 None admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:47:34.038864 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89b3749d-932e-4be3-99f1-f4347f0c9111 None admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:47:34.038991 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89b3749d-932e-4be3-99f1-f4347f0c9111 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:47:34.050855 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89b3749d-932e-4be3-99f1-f4347f0c9111 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 15:47:34.052659 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89b3749d-932e-4be3-99f1-f4347f0c9111 None admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 15:47:34.058928 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-632037cd-0b24-4171-81b2-536bb10c65b5 None admin] Validating token access rules against request {{(pid=81767) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 15:47:34.070505 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application [None req-89b3749d-932e-4be3-99f1-f4347f0c9111 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 22 15:47:34.070505 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 15:47:34.070505 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 15:47:34.070505 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 15:47:34.070505 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 15:47:34.070505 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 15:47:34.070505 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 15:47:34.070505 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 15:47:34.070505 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 15:47:34.070505 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 15:47:34.070505 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 15:47:34.070505 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 15:47:34.070505 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 22 15:47:34.070505 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 22 15:47:34.070505 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 22 15:47:34.070505 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 22 15:47:34.071516 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 15:47:34.071516 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 15:47:34.071516 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 15:47:34.071516 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 15:47:34.071516 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 15:47:34.071516 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 15:47:34.071516 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 15:47:34.071516 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application with Lock( Aug 22 15:47:34.071516 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 15:47:34.071516 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self._enter() Aug 22 15:47:34.071516 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 15:47:34.071516 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 15:47:34.071516 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 15:47:34.071516 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self.creator() Aug 22 15:47:34.071516 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 15:47:34.071516 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application created_value = creator( Aug 22 15:47:34.071516 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 22 15:47:34.072403 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 22 15:47:34.072403 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 22 15:47:34.072403 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application Aug 22 15:47:34.072403 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 310/614] 10.222.0.15 () {56 vars in 1108 bytes} [Mon Aug 22 15:47:33 2022] GET /identity/v3/domains/Default => generated 87 bytes in 241 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 15:47:34.079766 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89b3749d-932e-4be3-99f1-f4347f0c9111 None admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 15:47:34.101811 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d4f51ee-6e23-4f88-8308-3be6e21794d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81767) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 15:47:34.102574 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d4f51ee-6e23-4f88-8308-3be6e21794d6 None admin] REQUEST_METHOD: `GET` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:47:34.102851 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d4f51ee-6e23-4f88-8308-3be6e21794d6 None admin] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:47:34.103115 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d4f51ee-6e23-4f88-8308-3be6e21794d6 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:47:34.115891 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d4f51ee-6e23-4f88-8308-3be6e21794d6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 22 15:47:34.121570 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application [None req-3d4f51ee-6e23-4f88-8308-3be6e21794d6 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 22 15:47:34.121570 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 15:47:34.121570 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 15:47:34.121570 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 15:47:34.121570 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 15:47:34.121570 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 15:47:34.121570 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 15:47:34.121570 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 15:47:34.121570 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 15:47:34.121570 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 15:47:34.121570 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 15:47:34.121570 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 15:47:34.121570 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 22 15:47:34.121570 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 22 15:47:34.121570 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 22 15:47:34.121570 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 22 15:47:34.122538 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 22 15:47:34.122538 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 22 15:47:34.122538 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 22 15:47:34.122538 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 22 15:47:34.122538 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 15:47:34.122538 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 15:47:34.122538 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 15:47:34.122538 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 15:47:34.122538 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 15:47:34.122538 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 15:47:34.122538 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 15:47:34.122538 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application with Lock( Aug 22 15:47:34.122538 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 15:47:34.122538 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return self._enter() Aug 22 15:47:34.122538 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 15:47:34.122538 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 15:47:34.123388 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 15:47:34.123388 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return self.creator() Aug 22 15:47:34.123388 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 15:47:34.123388 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application created_value = creator( Aug 22 15:47:34.123388 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 22 15:47:34.123388 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 22 15:47:34.123388 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 22 15:47:34.123388 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 22 15:47:34.123388 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 22 15:47:34.123388 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 22 15:47:34.123388 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 22 15:47:34.123388 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application Aug 22 15:47:34.126747 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 305/615] 10.222.0.15 () {56 vars in 1102 bytes} [Mon Aug 22 15:47:33 2022] GET /identity/v3/roles/service => generated 85 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 15:47:34.129678 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d4f51ee-6e23-4f88-8308-3be6e21794d6 None admin] Authenticating user token {{(pid=81767) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 15:47:34.186306 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG dbcounter [-] [81767] Writing DB stats keystone:SELECT=100 {{(pid=81767) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 15:47:34.197774 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89b3749d-932e-4be3-99f1-f4347f0c9111 None admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 15:47:34.209274 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=100 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 15:47:34.235543 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23cb6629-ffa3-4b58-abad-b133b3f4820d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 15:47:34.236224 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23cb6629-ffa3-4b58-abad-b133b3f4820d None admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:47:34.236507 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23cb6629-ffa3-4b58-abad-b133b3f4820d None admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:47:34.236783 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23cb6629-ffa3-4b58-abad-b133b3f4820d None admin] PATH_INFO: `/v3/domains` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:47:34.237431 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23cb6629-ffa3-4b58-abad-b133b3f4820d None admin] RBAC: Adding query filter params (name=Default) {{(pid=81768) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 15:47:34.237828 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23cb6629-ffa3-4b58-abad-b133b3f4820d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 15:47:34.239407 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23cb6629-ffa3-4b58-abad-b133b3f4820d None admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 15:47:34.241764 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d4f51ee-6e23-4f88-8308-3be6e21794d6 None admin] Validating token access rules against request {{(pid=81767) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 15:47:34.253373 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 311/616] 10.222.0.15 () {56 vars in 1109 bytes} [Mon Aug 22 15:47:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 15:47:34.259182 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23cb6629-ffa3-4b58-abad-b133b3f4820d None admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 15:47:34.279807 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a884cdaf-a56b-4caf-9d0c-1534d5528150 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81767) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 15:47:34.280438 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a884cdaf-a56b-4caf-9d0c-1534d5528150 None admin] REQUEST_METHOD: `GET` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:47:34.280710 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a884cdaf-a56b-4caf-9d0c-1534d5528150 None admin] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:47:34.280978 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a884cdaf-a56b-4caf-9d0c-1534d5528150 None admin] PATH_INFO: `/v3/roles` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:47:34.281647 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a884cdaf-a56b-4caf-9d0c-1534d5528150 None admin] RBAC: Adding query filter params (name=service) {{(pid=81767) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 15:47:34.281943 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a884cdaf-a56b-4caf-9d0c-1534d5528150 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 15:47:34.283459 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a884cdaf-a56b-4caf-9d0c-1534d5528150 None admin] RBAC: Authorization granted {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 15:47:34.298565 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 306/617] 10.222.0.15 () {56 vars in 1103 bytes} [Mon Aug 22 15:47:34 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 22 15:47:34.300082 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a884cdaf-a56b-4caf-9d0c-1534d5528150 None admin] Authenticating user token {{(pid=81767) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 15:47:34.374843 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23cb6629-ffa3-4b58-abad-b133b3f4820d None admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 15:47:34.412667 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a884cdaf-a56b-4caf-9d0c-1534d5528150 None admin] Validating token access rules against request {{(pid=81767) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 15:47:34.413434 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c35fd75a-5263-4d31-b496-e4c65e0be20c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 15:47:34.414180 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c35fd75a-5263-4d31-b496-e4c65e0be20c None admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:47:34.414466 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c35fd75a-5263-4d31-b496-e4c65e0be20c None admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:47:34.414747 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c35fd75a-5263-4d31-b496-e4c65e0be20c None admin] PATH_INFO: `/v3/users/swift` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:47:34.423532 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c35fd75a-5263-4d31-b496-e4c65e0be20c None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 15:47:34.425001 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG dbcounter [-] [81767] Writing DB stats keystone:SELECT=100 {{(pid=81767) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 15:47:34.426184 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c35fd75a-5263-4d31-b496-e4c65e0be20c None admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 15:47:34.440441 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application [None req-c35fd75a-5263-4d31-b496-e4c65e0be20c None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 22 15:47:34.440441 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 15:47:34.440441 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 15:47:34.440441 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 15:47:34.440441 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 15:47:34.440441 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 15:47:34.440441 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 15:47:34.440441 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 15:47:34.440441 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 15:47:34.440441 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 15:47:34.440441 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 15:47:34.440441 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 15:47:34.440441 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 22 15:47:34.440441 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 22 15:47:34.440441 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 22 15:47:34.440441 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 22 15:47:34.442615 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 15:47:34.442615 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 15:47:34.442615 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 22 15:47:34.442615 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 22 15:47:34.442615 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 22 15:47:34.442615 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 22 15:47:34.442615 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 15:47:34.442615 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 15:47:34.442615 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 15:47:34.442615 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 15:47:34.442615 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 15:47:34.442615 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application with Lock( Aug 22 15:47:34.442615 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 15:47:34.442615 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self._enter() Aug 22 15:47:34.442615 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 15:47:34.442615 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 15:47:34.442615 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 15:47:34.442615 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self.creator() Aug 22 15:47:34.445004 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 15:47:34.445004 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application created_value = creator( Aug 22 15:47:34.445004 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 22 15:47:34.445004 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 22 15:47:34.445004 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 22 15:47:34.445004 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 22 15:47:34.445004 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 22 15:47:34.445004 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 22 15:47:34.445004 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 22 15:47:34.445004 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application Aug 22 15:47:34.445004 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 312/618] 10.222.0.15 () {56 vars in 1096 bytes} [Mon Aug 22 15:47:34 2022] GET /identity/v3/users/swift => generated 83 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 22 15:47:34.455310 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c35fd75a-5263-4d31-b496-e4c65e0be20c None admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 15:47:34.455632 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26bb6a6a-aa9b-4b38-b98c-bb86341964fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81767) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 15:47:34.460853 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26bb6a6a-aa9b-4b38-b98c-bb86341964fe None admin] REQUEST_METHOD: `GET` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:47:34.462909 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26bb6a6a-aa9b-4b38-b98c-bb86341964fe None admin] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:47:34.463216 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26bb6a6a-aa9b-4b38-b98c-bb86341964fe None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:47:34.493544 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26bb6a6a-aa9b-4b38-b98c-bb86341964fe None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 15:47:34.495079 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26bb6a6a-aa9b-4b38-b98c-bb86341964fe None admin] RBAC: Authorization granted {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 15:47:34.511854 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=100 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 15:47:34.536981 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application [None req-26bb6a6a-aa9b-4b38-b98c-bb86341964fe None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 22 15:47:34.536981 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 15:47:34.536981 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 15:47:34.536981 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 15:47:34.536981 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 15:47:34.536981 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 15:47:34.536981 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 15:47:34.536981 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 15:47:34.536981 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 15:47:34.536981 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 15:47:34.536981 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 15:47:34.536981 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 15:47:34.536981 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 22 15:47:34.536981 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 22 15:47:34.536981 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 22 15:47:34.536981 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 22 15:47:34.539356 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 15:47:34.539356 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 15:47:34.539356 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 15:47:34.539356 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 15:47:34.539356 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 15:47:34.539356 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 15:47:34.539356 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 15:47:34.539356 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application with Lock( Aug 22 15:47:34.539356 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 15:47:34.539356 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return self._enter() Aug 22 15:47:34.539356 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 15:47:34.539356 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 15:47:34.539356 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 15:47:34.539356 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return self.creator() Aug 22 15:47:34.539356 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 15:47:34.539356 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application created_value = creator( Aug 22 15:47:34.539356 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 22 15:47:34.541480 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 22 15:47:34.541480 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 22 15:47:34.541480 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application Aug 22 15:47:34.541480 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 307/619] 10.222.0.15 () {56 vars in 1108 bytes} [Mon Aug 22 15:47:34 2022] GET /identity/v3/domains/Default => generated 87 bytes in 239 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 15:47:34.544360 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26bb6a6a-aa9b-4b38-b98c-bb86341964fe None admin] Authenticating user token {{(pid=81767) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 15:47:34.599720 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c35fd75a-5263-4d31-b496-e4c65e0be20c None admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 15:47:34.640359 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-714ab09c-7e78-47b3-8bb1-9d2975ea0d9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 15:47:34.640985 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-714ab09c-7e78-47b3-8bb1-9d2975ea0d9c None admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:47:34.641261 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-714ab09c-7e78-47b3-8bb1-9d2975ea0d9c None admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:47:34.641856 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-714ab09c-7e78-47b3-8bb1-9d2975ea0d9c None admin] PATH_INFO: `/v3/users` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:47:34.643207 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-714ab09c-7e78-47b3-8bb1-9d2975ea0d9c None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81768) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 15:47:34.643553 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-714ab09c-7e78-47b3-8bb1-9d2975ea0d9c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 15:47:34.645067 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-714ab09c-7e78-47b3-8bb1-9d2975ea0d9c None admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 15:47:34.668186 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 313/620] 10.222.0.15 () {56 vars in 1135 bytes} [Mon Aug 22 15:47:34 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 15:47:34.674575 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-714ab09c-7e78-47b3-8bb1-9d2975ea0d9c None admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 15:47:34.783761 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=100 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 15:47:34.802740 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-714ab09c-7e78-47b3-8bb1-9d2975ea0d9c None admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 15:47:34.847669 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65007c2c-3215-4a24-b2c6-7a3003d018c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 15:47:34.848334 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65007c2c-3215-4a24-b2c6-7a3003d018c9 None admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:47:34.848620 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65007c2c-3215-4a24-b2c6-7a3003d018c9 None admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:47:34.848898 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65007c2c-3215-4a24-b2c6-7a3003d018c9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:47:34.863247 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65007c2c-3215-4a24-b2c6-7a3003d018c9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 15:47:34.865550 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65007c2c-3215-4a24-b2c6-7a3003d018c9 None admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 15:47:34.883236 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application [None req-65007c2c-3215-4a24-b2c6-7a3003d018c9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 22 15:47:34.883236 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 15:47:34.883236 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 15:47:34.883236 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 15:47:34.883236 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 15:47:34.883236 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 15:47:34.883236 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 15:47:34.883236 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 15:47:34.883236 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 15:47:34.883236 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 15:47:34.883236 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 15:47:34.883236 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 15:47:34.883236 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 22 15:47:34.883236 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 22 15:47:34.883236 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 22 15:47:34.883236 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 22 15:47:34.884556 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 15:47:34.884556 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 15:47:34.884556 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 15:47:34.884556 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 15:47:34.884556 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 15:47:34.884556 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 15:47:34.884556 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 15:47:34.884556 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application with Lock( Aug 22 15:47:34.884556 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 15:47:34.884556 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self._enter() Aug 22 15:47:34.884556 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 15:47:34.884556 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 15:47:34.884556 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 15:47:34.884556 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self.creator() Aug 22 15:47:34.884556 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 15:47:34.884556 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application created_value = creator( Aug 22 15:47:34.884556 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 22 15:47:34.886345 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 22 15:47:34.886345 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 22 15:47:34.886345 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application Aug 22 15:47:34.886345 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 314/621] 10.222.0.15 () {56 vars in 1108 bytes} [Mon Aug 22 15:47:34 2022] GET /identity/v3/domains/Default => generated 87 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 22 15:47:34.892025 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65007c2c-3215-4a24-b2c6-7a3003d018c9 None admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 15:47:34.894142 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26bb6a6a-aa9b-4b38-b98c-bb86341964fe None admin] Validating token access rules against request {{(pid=81767) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 15:47:34.975305 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce8090f0-99bb-461c-908f-3dbb75b33fdb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81767) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 15:47:34.975936 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce8090f0-99bb-461c-908f-3dbb75b33fdb None admin] REQUEST_METHOD: `GET` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:47:34.976207 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce8090f0-99bb-461c-908f-3dbb75b33fdb None admin] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:47:34.976481 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce8090f0-99bb-461c-908f-3dbb75b33fdb None admin] PATH_INFO: `/v3/domains` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:47:34.977148 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce8090f0-99bb-461c-908f-3dbb75b33fdb None admin] RBAC: Adding query filter params (name=Default) {{(pid=81767) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 15:47:34.977449 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce8090f0-99bb-461c-908f-3dbb75b33fdb None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 15:47:34.978836 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce8090f0-99bb-461c-908f-3dbb75b33fdb None admin] RBAC: Authorization granted {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 15:47:35.014684 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 308/622] 10.222.0.15 () {56 vars in 1109 bytes} [Mon Aug 22 15:47:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 471 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 15:47:35.020744 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce8090f0-99bb-461c-908f-3dbb75b33fdb None admin] Authenticating user token {{(pid=81767) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 15:47:35.041165 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65007c2c-3215-4a24-b2c6-7a3003d018c9 None admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 15:47:35.066588 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG dbcounter [-] [81767] Writing DB stats keystone:SELECT=100 {{(pid=81767) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 15:47:35.115005 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c16ea0bd-4245-49ed-a4bc-10338522dc6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 15:47:35.115655 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c16ea0bd-4245-49ed-a4bc-10338522dc6f None admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:47:35.115937 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c16ea0bd-4245-49ed-a4bc-10338522dc6f None admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:47:35.116206 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c16ea0bd-4245-49ed-a4bc-10338522dc6f None admin] PATH_INFO: `/v3/domains` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:47:35.116863 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c16ea0bd-4245-49ed-a4bc-10338522dc6f None admin] RBAC: Adding query filter params (name=Default) {{(pid=81768) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 15:47:35.117162 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c16ea0bd-4245-49ed-a4bc-10338522dc6f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 15:47:35.119401 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c16ea0bd-4245-49ed-a4bc-10338522dc6f None admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 15:47:35.137583 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 315/623] 10.222.0.15 () {56 vars in 1109 bytes} [Mon Aug 22 15:47:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 246 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 22 15:47:35.138101 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=100 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 15:47:35.140638 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf80de04-e111-4c7b-932b-72003ca4ab72 None None] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:47:35.140817 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf80de04-e111-4c7b-932b-72003ca4ab72 None None] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:47:35.141192 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf80de04-e111-4c7b-932b-72003ca4ab72 None None] PATH_INFO: `/` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:47:35.146833 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 316/624] 10.222.0.15 () {54 vars in 927 bytes} [Mon Aug 22 15:47:35 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 15:47:35.147881 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf80de04-e111-4c7b-932b-72003ca4ab72 None None] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 15:47:35.184184 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce8090f0-99bb-461c-908f-3dbb75b33fdb None admin] Validating token access rules against request {{(pid=81767) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 15:47:35.228847 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc9affd9-8ce8-486b-adc3-33a86922e6c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81767) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 15:47:35.229549 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc9affd9-8ce8-486b-adc3-33a86922e6c1 None admin] REQUEST_METHOD: `GET` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:47:35.229828 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc9affd9-8ce8-486b-adc3-33a86922e6c1 None admin] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:47:35.230101 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc9affd9-8ce8-486b-adc3-33a86922e6c1 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:47:35.240589 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc9affd9-8ce8-486b-adc3-33a86922e6c1 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 15:47:35.242830 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc9affd9-8ce8-486b-adc3-33a86922e6c1 None admin] RBAC: Authorization granted {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 15:47:35.265612 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application [None req-dc9affd9-8ce8-486b-adc3-33a86922e6c1 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 22 15:47:35.265612 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 15:47:35.265612 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 15:47:35.265612 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 15:47:35.265612 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 15:47:35.265612 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 15:47:35.265612 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 15:47:35.265612 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 15:47:35.265612 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 15:47:35.265612 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 15:47:35.265612 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 15:47:35.265612 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 15:47:35.265612 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 22 15:47:35.265612 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 22 15:47:35.265612 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 22 15:47:35.265612 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 22 15:47:35.266619 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 15:47:35.266619 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 15:47:35.266619 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 22 15:47:35.266619 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 22 15:47:35.266619 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 22 15:47:35.266619 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 22 15:47:35.266619 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 15:47:35.266619 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 15:47:35.266619 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 15:47:35.266619 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 15:47:35.266619 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 15:47:35.266619 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application with Lock( Aug 22 15:47:35.266619 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 15:47:35.266619 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return self._enter() Aug 22 15:47:35.266619 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 15:47:35.266619 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 15:47:35.266619 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 15:47:35.266619 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return self.creator() Aug 22 15:47:35.272249 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 15:47:35.272249 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application created_value = creator( Aug 22 15:47:35.272249 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 22 15:47:35.272249 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 22 15:47:35.272249 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 22 15:47:35.272249 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 22 15:47:35.272249 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 22 15:47:35.272249 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 22 15:47:35.272249 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 22 15:47:35.272249 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application Aug 22 15:47:35.272249 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 309/625] 10.222.0.15 () {56 vars in 1093 bytes} [Mon Aug 22 15:47:35 2022] GET /identity/v3/users/nova => generated 82 bytes in 247 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 22 15:47:35.272249 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2fe826c-dc39-4a8f-9aa3-f16ee19fc2b0 None None] REQUEST_METHOD: `POST` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:47:35.272249 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2fe826c-dc39-4a8f-9aa3-f16ee19fc2b0 None None] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:47:35.273170 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2fe826c-dc39-4a8f-9aa3-f16ee19fc2b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:47:35.288442 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf80de04-e111-4c7b-932b-72003ca4ab72 None None] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 15:47:35.292091 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.auth.core [None req-f2fe826c-dc39-4a8f-9aa3-f16ee19fc2b0 None None] MFA Rules not processed for user `6b2199ae0fb0480aa917ada391f5ec54`. Rule list: `[]` (Enabled: `True`). {{(pid=81767) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 15:47:35.293640 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.fernet_utils [None req-f2fe826c-dc39-4a8f-9aa3-f16ee19fc2b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81767) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 15:47:35.333957 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fb19ecd-bf88-4ec6-acf0-f5823343966d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 15:47:35.334601 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fb19ecd-bf88-4ec6-acf0-f5823343966d None admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:47:35.334878 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fb19ecd-bf88-4ec6-acf0-f5823343966d None admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:47:35.335146 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fb19ecd-bf88-4ec6-acf0-f5823343966d None admin] PATH_INFO: `/v3/projects/service` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:47:35.344261 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fb19ecd-bf88-4ec6-acf0-f5823343966d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 15:47:35.346149 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fb19ecd-bf88-4ec6-acf0-f5823343966d None admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 15:47:35.359647 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application [None req-1fb19ecd-bf88-4ec6-acf0-f5823343966d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 22 15:47:35.359647 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 15:47:35.359647 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 15:47:35.359647 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 15:47:35.359647 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 15:47:35.359647 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 15:47:35.359647 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 15:47:35.359647 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 15:47:35.359647 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 15:47:35.359647 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 15:47:35.359647 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 15:47:35.359647 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 15:47:35.359647 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 22 15:47:35.359647 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 22 15:47:35.359647 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 22 15:47:35.360778 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 22 15:47:35.360778 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 15:47:35.360778 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 15:47:35.360778 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 15:47:35.360778 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 15:47:35.360778 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 15:47:35.360778 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 15:47:35.360778 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 15:47:35.360778 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application with Lock( Aug 22 15:47:35.360778 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 15:47:35.360778 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self._enter() Aug 22 15:47:35.360778 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 15:47:35.360778 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 15:47:35.360778 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 15:47:35.360778 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self.creator() Aug 22 15:47:35.360778 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 15:47:35.360778 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application created_value = creator( Aug 22 15:47:35.361882 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 22 15:47:35.361882 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 22 15:47:35.361882 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 22 15:47:35.361882 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 22 15:47:35.361882 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 22 15:47:35.361882 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 22 15:47:35.361882 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 22 15:47:35.361882 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application Aug 22 15:47:35.361882 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 317/626] 10.222.0.15 () {56 vars in 1111 bytes} [Mon Aug 22 15:47:35 2022] GET /identity/v3/projects/service => generated 88 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 15:47:35.362487 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fb19ecd-bf88-4ec6-acf0-f5823343966d None admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 15:47:35.387713 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 310/627] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Aug 22 15:47:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 22 15:47:35.390027 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-374a1cc0-1960-4e40-b58b-cf9ffad5d4a8 None None] REQUEST_METHOD: `GET` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:47:35.390391 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-374a1cc0-1960-4e40-b58b-cf9ffad5d4a8 None None] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:47:35.390710 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-374a1cc0-1960-4e40-b58b-cf9ffad5d4a8 None None] PATH_INFO: `/` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:47:35.393630 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 311/628] 10.222.0.15 () {54 vars in 927 bytes} [Mon Aug 22 15:47:35 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 15:47:35.395090 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-374a1cc0-1960-4e40-b58b-cf9ffad5d4a8 None None] Authenticating user token {{(pid=81767) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 15:47:35.419824 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG dbcounter [-] [81767] Writing DB stats keystone:SELECT=100 {{(pid=81767) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 15:47:35.440815 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=100 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 15:47:35.492607 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fb19ecd-bf88-4ec6-acf0-f5823343966d None admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 15:47:35.539251 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c829956b-c06f-4fb4-9a10-f19846d7edc5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 15:47:35.539894 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c829956b-c06f-4fb4-9a10-f19846d7edc5 None admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:47:35.540447 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c829956b-c06f-4fb4-9a10-f19846d7edc5 None admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:47:35.540715 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c829956b-c06f-4fb4-9a10-f19846d7edc5 None admin] PATH_INFO: `/v3/users` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:47:35.542015 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c829956b-c06f-4fb4-9a10-f19846d7edc5 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81768) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 15:47:35.542651 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c829956b-c06f-4fb4-9a10-f19846d7edc5 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 15:47:35.544398 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c829956b-c06f-4fb4-9a10-f19846d7edc5 None admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 15:47:35.553474 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-374a1cc0-1960-4e40-b58b-cf9ffad5d4a8 None None] Validating token access rules against request {{(pid=81767) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 15:47:35.570316 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 318/629] 10.222.0.15 () {56 vars in 1133 bytes} [Mon Aug 22 15:47:35 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 15:47:35.577653 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92f0e409-949a-4cb0-bef8-02a5ed70158f None None] REQUEST_METHOD: `POST` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:47:35.579499 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92f0e409-949a-4cb0-bef8-02a5ed70158f None None] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:47:35.580216 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92f0e409-949a-4cb0-bef8-02a5ed70158f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:47:35.603651 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.auth.core [None req-92f0e409-949a-4cb0-bef8-02a5ed70158f None None] MFA Rules not processed for user `6b2199ae0fb0480aa917ada391f5ec54`. Rule list: `[]` (Enabled: `True`). {{(pid=81768) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 15:47:35.606004 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-92f0e409-949a-4cb0-bef8-02a5ed70158f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 15:47:35.612331 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13a8c4cf-d6bc-4dec-9161-797560a6b58e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81767) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 15:47:35.613251 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a8c4cf-d6bc-4dec-9161-797560a6b58e None admin] REQUEST_METHOD: `GET` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:47:35.613667 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a8c4cf-d6bc-4dec-9161-797560a6b58e None admin] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:47:35.614086 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a8c4cf-d6bc-4dec-9161-797560a6b58e None admin] PATH_INFO: `/v3/projects` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:47:35.615027 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13a8c4cf-d6bc-4dec-9161-797560a6b58e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81767) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 15:47:35.615508 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13a8c4cf-d6bc-4dec-9161-797560a6b58e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 15:47:35.617149 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13a8c4cf-d6bc-4dec-9161-797560a6b58e None admin] RBAC: Authorization granted {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 15:47:35.631216 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 312/630] 10.222.0.15 () {56 vars in 1148 bytes} [Mon Aug 22 15:47:35 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 237 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 15:47:35.632467 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13a8c4cf-d6bc-4dec-9161-797560a6b58e None admin] Authenticating user token {{(pid=81767) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 15:47:35.706541 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 319/631] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Aug 22 15:47:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 22 15:47:35.707643 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92f0e409-949a-4cb0-bef8-02a5ed70158f None None] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 15:47:35.736205 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG dbcounter [-] [81767] Writing DB stats keystone:SELECT=100 {{(pid=81767) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 15:47:35.774594 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=100 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 15:47:35.776863 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13a8c4cf-d6bc-4dec-9161-797560a6b58e None admin] Validating token access rules against request {{(pid=81767) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 15:47:35.826100 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3429bf88-177d-4ab8-998b-3acd299382c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81767) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 15:47:35.827048 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3429bf88-177d-4ab8-998b-3acd299382c8 None admin] REQUEST_METHOD: `GET` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:47:35.827549 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3429bf88-177d-4ab8-998b-3acd299382c8 None admin] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:47:35.828036 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3429bf88-177d-4ab8-998b-3acd299382c8 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:47:35.854430 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92f0e409-949a-4cb0-bef8-02a5ed70158f None None] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 15:47:35.902727 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3429bf88-177d-4ab8-998b-3acd299382c8 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 15:47:35.903855 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1d10cf5-7131-4188-a340-61b5a86f0b10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 15:47:35.904534 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d10cf5-7131-4188-a340-61b5a86f0b10 None admin] REQUEST_METHOD: `PUT` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:47:35.904813 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d10cf5-7131-4188-a340-61b5a86f0b10 None admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:47:35.905086 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d10cf5-7131-4188-a340-61b5a86f0b10 None admin] PATH_INFO: `/v3/projects/c4ee0413f9ef4d4fb1f95a660babe090/users/6a09d128dee9438283633e51f4834643/roles/a5fbba1dc615490a96cde27e3a102eb6` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:47:35.905439 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3429bf88-177d-4ab8-998b-3acd299382c8 None admin] RBAC: Authorization granted {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 15:47:35.924469 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1d10cf5-7131-4188-a340-61b5a86f0b10 None admin] RBAC: Authorizing `identity:create_grant(project_id=c4ee0413f9ef4d4fb1f95a660babe090, user_id=6a09d128dee9438283633e51f4834643, role_id=a5fbba1dc615490a96cde27e3a102eb6)` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 15:47:35.926196 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1d10cf5-7131-4188-a340-61b5a86f0b10 None admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 15:47:35.984369 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG dbcounter [-] [81767] Writing DB stats keystone:SELECT=100 {{(pid=81767) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 15:47:36.015547 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 313/632] 10.222.0.15 () {58 vars in 1282 bytes} [Mon Aug 22 15:47:35 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 383 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 22 15:47:36.016399 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3429bf88-177d-4ab8-998b-3acd299382c8 None admin] Authenticating user token {{(pid=81767) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 15:47:36.024778 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 320/633] 10.222.0.15 () {58 vars in 1439 bytes} [Mon Aug 22 15:47:35 2022] PUT /identity/v3/projects/c4ee0413f9ef4d4fb1f95a660babe090/users/6a09d128dee9438283633e51f4834643/roles/a5fbba1dc615490a96cde27e3a102eb6 => generated 0 bytes in 317 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 15:47:36.026032 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1d10cf5-7131-4188-a340-61b5a86f0b10 None admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 15:47:36.162665 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 15:47:36.170006 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1d10cf5-7131-4188-a340-61b5a86f0b10 None admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 15:47:36.175666 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3429bf88-177d-4ab8-998b-3acd299382c8 None admin] Validating token access rules against request {{(pid=81767) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 15:47:36.213923 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09df2d10-6acc-4e90-92a8-5c00de25efa0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 15:47:36.214576 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09df2d10-6acc-4e90-92a8-5c00de25efa0 None admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:47:36.214852 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09df2d10-6acc-4e90-92a8-5c00de25efa0 None admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:47:36.215324 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09df2d10-6acc-4e90-92a8-5c00de25efa0 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:47:36.223677 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac2aaccf-7a7e-4904-b3fe-f2fc797264c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81767) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 15:47:36.224897 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac2aaccf-7a7e-4904-b3fe-f2fc797264c3 None admin] REQUEST_METHOD: `GET` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:47:36.225352 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac2aaccf-7a7e-4904-b3fe-f2fc797264c3 None admin] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:47:36.225829 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac2aaccf-7a7e-4904-b3fe-f2fc797264c3 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:47:36.227504 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09df2d10-6acc-4e90-92a8-5c00de25efa0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 22 15:47:36.233248 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application [None req-09df2d10-6acc-4e90-92a8-5c00de25efa0 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 22 15:47:36.233248 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 15:47:36.233248 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 15:47:36.233248 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 15:47:36.233248 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 15:47:36.233248 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 15:47:36.233248 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 15:47:36.233248 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 15:47:36.233248 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 15:47:36.233248 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 15:47:36.233248 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 15:47:36.233248 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 15:47:36.233248 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 22 15:47:36.233248 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 22 15:47:36.233248 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 22 15:47:36.233248 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 22 15:47:36.234161 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 22 15:47:36.234161 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 22 15:47:36.234161 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 22 15:47:36.234161 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 22 15:47:36.234161 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 15:47:36.234161 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 15:47:36.234161 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 15:47:36.234161 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 15:47:36.234161 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 15:47:36.234161 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 15:47:36.234161 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 15:47:36.234161 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application with Lock( Aug 22 15:47:36.234161 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 15:47:36.234161 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self._enter() Aug 22 15:47:36.234161 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 15:47:36.234161 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 15:47:36.235063 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 15:47:36.235063 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self.creator() Aug 22 15:47:36.235063 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 15:47:36.235063 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application created_value = creator( Aug 22 15:47:36.235063 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 22 15:47:36.235063 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 22 15:47:36.235063 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 22 15:47:36.235063 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 22 15:47:36.235063 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 22 15:47:36.235063 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 22 15:47:36.235063 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 22 15:47:36.235063 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application Aug 22 15:47:36.238065 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 321/634] 10.222.0.15 () {56 vars in 1102 bytes} [Mon Aug 22 15:47:36 2022] GET /identity/v3/roles/service => generated 85 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 15:47:36.239059 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09df2d10-6acc-4e90-92a8-5c00de25efa0 None admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 15:47:36.242275 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac2aaccf-7a7e-4904-b3fe-f2fc797264c3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 22 15:47:36.248380 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application [None req-ac2aaccf-7a7e-4904-b3fe-f2fc797264c3 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 22 15:47:36.248380 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 15:47:36.248380 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 15:47:36.248380 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 15:47:36.248380 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 15:47:36.248380 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 15:47:36.248380 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 15:47:36.248380 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 15:47:36.248380 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 15:47:36.248380 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 15:47:36.248380 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 15:47:36.248380 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 15:47:36.248380 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 22 15:47:36.248380 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 22 15:47:36.248380 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 22 15:47:36.248380 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 22 15:47:36.249732 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 22 15:47:36.249732 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 22 15:47:36.249732 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 22 15:47:36.249732 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 22 15:47:36.249732 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 15:47:36.249732 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 15:47:36.249732 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 15:47:36.249732 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 15:47:36.249732 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 15:47:36.249732 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 15:47:36.249732 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 15:47:36.249732 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application with Lock( Aug 22 15:47:36.249732 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 15:47:36.249732 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return self._enter() Aug 22 15:47:36.249732 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 15:47:36.249732 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 15:47:36.250538 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 15:47:36.250538 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return self.creator() Aug 22 15:47:36.250538 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 15:47:36.250538 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application created_value = creator( Aug 22 15:47:36.250538 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 22 15:47:36.250538 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 22 15:47:36.250538 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 22 15:47:36.250538 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 22 15:47:36.250538 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 22 15:47:36.250538 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 22 15:47:36.250538 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 22 15:47:36.250538 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application Aug 22 15:47:36.250538 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 314/635] 10.222.0.15 () {56 vars in 1102 bytes} [Mon Aug 22 15:47:36 2022] GET /identity/v3/roles/service => generated 85 bytes in 233 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 15:47:36.250538 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac2aaccf-7a7e-4904-b3fe-f2fc797264c3 None admin] Authenticating user token {{(pid=81767) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 15:47:36.300222 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG dbcounter [-] [81767] Writing DB stats keystone:SELECT=100 {{(pid=81767) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 15:47:36.362809 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09df2d10-6acc-4e90-92a8-5c00de25efa0 None admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 15:47:36.386532 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac2aaccf-7a7e-4904-b3fe-f2fc797264c3 None admin] Validating token access rules against request {{(pid=81767) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 15:47:36.399219 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1686f79-fe1e-4dd6-b236-39767219ee91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 15:47:36.399888 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1686f79-fe1e-4dd6-b236-39767219ee91 None admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:47:36.400159 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1686f79-fe1e-4dd6-b236-39767219ee91 None admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:47:36.400425 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1686f79-fe1e-4dd6-b236-39767219ee91 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:47:36.411938 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1686f79-fe1e-4dd6-b236-39767219ee91 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 15:47:36.413447 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1686f79-fe1e-4dd6-b236-39767219ee91 None admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 15:47:36.426939 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6de0209-2c00-4039-862e-7081340f506e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81767) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 15:47:36.427574 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6de0209-2c00-4039-862e-7081340f506e None admin] REQUEST_METHOD: `GET` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:47:36.427882 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6de0209-2c00-4039-862e-7081340f506e None admin] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:47:36.428153 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6de0209-2c00-4039-862e-7081340f506e None admin] PATH_INFO: `/v3/roles` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:47:36.428817 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6de0209-2c00-4039-862e-7081340f506e None admin] RBAC: Adding query filter params (name=service) {{(pid=81767) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 15:47:36.429056 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application [None req-a1686f79-fe1e-4dd6-b236-39767219ee91 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 22 15:47:36.429056 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 15:47:36.429056 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 15:47:36.429056 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 15:47:36.429056 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 15:47:36.429056 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 15:47:36.429056 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 15:47:36.429056 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 15:47:36.429056 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 15:47:36.429056 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 15:47:36.429056 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 15:47:36.429056 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 15:47:36.429056 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 22 15:47:36.429056 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 22 15:47:36.429056 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 22 15:47:36.429056 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 22 15:47:36.429931 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 15:47:36.429931 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 15:47:36.429931 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 15:47:36.429931 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 15:47:36.429931 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 15:47:36.429931 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 15:47:36.429931 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 15:47:36.429931 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application with Lock( Aug 22 15:47:36.429931 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 15:47:36.429931 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self._enter() Aug 22 15:47:36.429931 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 15:47:36.429931 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 15:47:36.429931 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 15:47:36.429931 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self.creator() Aug 22 15:47:36.429931 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 15:47:36.429931 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application created_value = creator( Aug 22 15:47:36.429931 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 22 15:47:36.430817 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 22 15:47:36.430817 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 22 15:47:36.430817 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application Aug 22 15:47:36.430992 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6de0209-2c00-4039-862e-7081340f506e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 15:47:36.430992 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6de0209-2c00-4039-862e-7081340f506e None admin] RBAC: Authorization granted {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 15:47:36.435464 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 322/636] 10.222.0.15 () {56 vars in 1108 bytes} [Mon Aug 22 15:47:36 2022] GET /identity/v3/domains/Default => generated 87 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 15:47:36.436397 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1686f79-fe1e-4dd6-b236-39767219ee91 None admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 15:47:36.440891 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 315/637] 10.222.0.15 () {56 vars in 1103 bytes} [Mon Aug 22 15:47:36 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 15:47:36.441714 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33af40e7-4fde-4814-94d2-c528c578b092 None None] REQUEST_METHOD: `GET` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:47:36.441996 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33af40e7-4fde-4814-94d2-c528c578b092 None None] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:47:36.442264 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33af40e7-4fde-4814-94d2-c528c578b092 None None] PATH_INFO: `/` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:47:36.447658 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 316/638] 10.222.0.15 () {54 vars in 927 bytes} [Mon Aug 22 15:47:36 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 15:47:36.448553 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33af40e7-4fde-4814-94d2-c528c578b092 None None] Authenticating user token {{(pid=81767) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 15:47:36.453189 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=100 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 15:47:36.559981 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG dbcounter [-] [81767] Writing DB stats keystone:SELECT=100 {{(pid=81767) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 15:47:36.562899 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1686f79-fe1e-4dd6-b236-39767219ee91 None admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 15:47:36.567923 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33af40e7-4fde-4814-94d2-c528c578b092 None None] Validating token access rules against request {{(pid=81767) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 15:47:36.605492 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6de6e8d2-5642-41a2-bae3-12c17f846da9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81767) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 15:47:36.605921 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33f445a6-e6f6-4e3f-b051-4d3f04e628ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 15:47:36.606135 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6de6e8d2-5642-41a2-bae3-12c17f846da9 None admin] REQUEST_METHOD: `GET` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:47:36.606374 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6de6e8d2-5642-41a2-bae3-12c17f846da9 None admin] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:47:36.606531 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33f445a6-e6f6-4e3f-b051-4d3f04e628ef None admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:47:36.606677 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6de6e8d2-5642-41a2-bae3-12c17f846da9 None admin] PATH_INFO: `/v3/domains` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:47:36.606840 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33f445a6-e6f6-4e3f-b051-4d3f04e628ef None admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:47:36.607053 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33f445a6-e6f6-4e3f-b051-4d3f04e628ef None admin] PATH_INFO: `/v3/roles` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:47:36.607380 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6de6e8d2-5642-41a2-bae3-12c17f846da9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81767) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 15:47:36.607685 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6de6e8d2-5642-41a2-bae3-12c17f846da9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 15:47:36.607777 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33f445a6-e6f6-4e3f-b051-4d3f04e628ef None admin] RBAC: Adding query filter params (name=service) {{(pid=81768) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 15:47:36.608028 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33f445a6-e6f6-4e3f-b051-4d3f04e628ef None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 15:47:36.609152 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6de6e8d2-5642-41a2-bae3-12c17f846da9 None admin] RBAC: Authorization granted {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 15:47:36.609594 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33f445a6-e6f6-4e3f-b051-4d3f04e628ef None admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 15:47:36.618698 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 323/639] 10.222.0.15 () {56 vars in 1103 bytes} [Mon Aug 22 15:47:36 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 15:47:36.620213 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33f445a6-e6f6-4e3f-b051-4d3f04e628ef None admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 15:47:36.623884 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 317/640] 10.222.0.15 () {56 vars in 1109 bytes} [Mon Aug 22 15:47:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 15:47:36.626178 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1087d5f4-7a84-492f-a776-e438f801c496 None None] REQUEST_METHOD: `POST` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:47:36.626468 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1087d5f4-7a84-492f-a776-e438f801c496 None None] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:47:36.626740 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1087d5f4-7a84-492f-a776-e438f801c496 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:47:36.646283 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.auth.core [None req-1087d5f4-7a84-492f-a776-e438f801c496 None None] MFA Rules not processed for user `6b2199ae0fb0480aa917ada391f5ec54`. Rule list: `[]` (Enabled: `True`). {{(pid=81767) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 15:47:36.647641 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.fernet_utils [None req-1087d5f4-7a84-492f-a776-e438f801c496 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81767) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 15:47:36.699558 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=100 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 15:47:36.730815 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 318/641] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Aug 22 15:47:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 22 15:47:36.732072 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1087d5f4-7a84-492f-a776-e438f801c496 None None] Authenticating user token {{(pid=81767) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 15:47:36.749796 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33f445a6-e6f6-4e3f-b051-4d3f04e628ef None admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 15:47:36.806329 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b902012c-afe2-4598-ac8e-14112e66678b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 15:47:36.807234 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b902012c-afe2-4598-ac8e-14112e66678b None admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:47:36.807531 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b902012c-afe2-4598-ac8e-14112e66678b None admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:47:36.807799 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b902012c-afe2-4598-ac8e-14112e66678b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:47:36.821058 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b902012c-afe2-4598-ac8e-14112e66678b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 15:47:36.822413 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b902012c-afe2-4598-ac8e-14112e66678b None admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 15:47:36.841584 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG dbcounter [-] [81767] Writing DB stats keystone:SELECT=100 {{(pid=81767) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 15:47:36.851790 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application [None req-b902012c-afe2-4598-ac8e-14112e66678b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 22 15:47:36.851790 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 15:47:36.851790 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 15:47:36.851790 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 15:47:36.851790 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 15:47:36.851790 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 15:47:36.851790 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 15:47:36.851790 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 15:47:36.851790 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 15:47:36.851790 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 15:47:36.851790 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 15:47:36.851790 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 15:47:36.851790 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 22 15:47:36.851790 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 22 15:47:36.851790 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 22 15:47:36.851790 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 22 15:47:36.852839 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 15:47:36.852839 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 15:47:36.852839 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 15:47:36.852839 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 15:47:36.852839 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 15:47:36.852839 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 15:47:36.852839 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 15:47:36.852839 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application with Lock( Aug 22 15:47:36.852839 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 15:47:36.852839 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self._enter() Aug 22 15:47:36.852839 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 15:47:36.852839 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 15:47:36.852839 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 15:47:36.852839 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self.creator() Aug 22 15:47:36.852839 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 15:47:36.852839 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application created_value = creator( Aug 22 15:47:36.852839 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 22 15:47:36.853729 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 22 15:47:36.853729 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 22 15:47:36.853729 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application Aug 22 15:47:36.856615 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 324/642] 10.222.0.15 () {56 vars in 1108 bytes} [Mon Aug 22 15:47:36 2022] GET /identity/v3/domains/Default => generated 87 bytes in 237 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 15:47:36.857749 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b902012c-afe2-4598-ac8e-14112e66678b None admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 15:47:36.866487 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1087d5f4-7a84-492f-a776-e438f801c496 None None] Validating token access rules against request {{(pid=81767) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 15:47:36.912795 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd184496-e34e-4d09-9b16-0aa171571014 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81767) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 15:47:36.913539 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd184496-e34e-4d09-9b16-0aa171571014 None admin] REQUEST_METHOD: `GET` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:47:36.913902 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd184496-e34e-4d09-9b16-0aa171571014 None admin] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:47:36.914286 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd184496-e34e-4d09-9b16-0aa171571014 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:47:36.927471 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd184496-e34e-4d09-9b16-0aa171571014 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 15:47:36.929010 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd184496-e34e-4d09-9b16-0aa171571014 None admin] RBAC: Authorization granted {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 15:47:36.946532 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application [None req-cd184496-e34e-4d09-9b16-0aa171571014 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 22 15:47:36.946532 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 15:47:36.946532 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 15:47:36.946532 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 15:47:36.946532 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 15:47:36.946532 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 15:47:36.946532 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 15:47:36.946532 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 15:47:36.946532 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 15:47:36.946532 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 15:47:36.946532 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 15:47:36.946532 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 15:47:36.946532 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 22 15:47:36.946532 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 22 15:47:36.946532 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 22 15:47:36.946532 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 22 15:47:36.952959 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 15:47:36.952959 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 15:47:36.952959 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 15:47:36.952959 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 15:47:36.952959 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 15:47:36.952959 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 15:47:36.952959 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 15:47:36.952959 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application with Lock( Aug 22 15:47:36.952959 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 15:47:36.952959 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return self._enter() Aug 22 15:47:36.952959 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 15:47:36.952959 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 15:47:36.952959 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 15:47:36.952959 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return self.creator() Aug 22 15:47:36.952959 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 15:47:36.952959 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application created_value = creator( Aug 22 15:47:36.952959 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 22 15:47:36.953879 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 22 15:47:36.953879 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 22 15:47:36.953879 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application Aug 22 15:47:36.953879 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 319/643] 10.222.0.15 () {56 vars in 1108 bytes} [Mon Aug 22 15:47:36 2022] GET /identity/v3/domains/Default => generated 87 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 15:47:36.953879 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd184496-e34e-4d09-9b16-0aa171571014 None admin] Authenticating user token {{(pid=81767) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 15:47:37.006507 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b902012c-afe2-4598-ac8e-14112e66678b None admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 15:47:37.040435 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=100 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 15:47:37.051845 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6605c3ee-d350-4eb4-ab92-0eb0abc55428 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 15:47:37.054601 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6605c3ee-d350-4eb4-ab92-0eb0abc55428 None admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:47:37.054601 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6605c3ee-d350-4eb4-ab92-0eb0abc55428 None admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:47:37.054601 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6605c3ee-d350-4eb4-ab92-0eb0abc55428 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:47:37.061448 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6605c3ee-d350-4eb4-ab92-0eb0abc55428 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 15:47:37.062896 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6605c3ee-d350-4eb4-ab92-0eb0abc55428 None admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 15:47:37.069101 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd184496-e34e-4d09-9b16-0aa171571014 None admin] Validating token access rules against request {{(pid=81767) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 15:47:37.075769 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application [None req-6605c3ee-d350-4eb4-ab92-0eb0abc55428 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 22 15:47:37.075769 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 15:47:37.075769 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 15:47:37.075769 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 15:47:37.075769 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 15:47:37.075769 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 15:47:37.075769 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 15:47:37.075769 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 15:47:37.075769 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 15:47:37.075769 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 15:47:37.075769 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 15:47:37.075769 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 15:47:37.075769 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 22 15:47:37.075769 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 22 15:47:37.075769 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 22 15:47:37.076666 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 22 15:47:37.076666 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 15:47:37.076666 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 15:47:37.076666 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 15:47:37.076666 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 15:47:37.076666 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 15:47:37.076666 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 15:47:37.076666 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 15:47:37.076666 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application with Lock( Aug 22 15:47:37.076666 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 15:47:37.076666 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self._enter() Aug 22 15:47:37.076666 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 15:47:37.076666 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 15:47:37.076666 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 15:47:37.076666 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self.creator() Aug 22 15:47:37.076666 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 15:47:37.076666 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application created_value = creator( Aug 22 15:47:37.077619 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 22 15:47:37.077619 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 22 15:47:37.077619 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 22 15:47:37.077619 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 22 15:47:37.077619 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 22 15:47:37.077619 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 22 15:47:37.077619 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 22 15:47:37.077619 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application Aug 22 15:47:37.077619 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 325/644] 10.222.0.15 () {56 vars in 1111 bytes} [Mon Aug 22 15:47:36 2022] GET /identity/v3/projects/service => generated 88 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 15:47:37.078765 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6605c3ee-d350-4eb4-ab92-0eb0abc55428 None admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 15:47:37.113985 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ea7c3b4-094d-4173-84ac-09beabea0bed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81767) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 15:47:37.114590 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ea7c3b4-094d-4173-84ac-09beabea0bed None admin] REQUEST_METHOD: `GET` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:47:37.114864 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ea7c3b4-094d-4173-84ac-09beabea0bed None admin] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:47:37.115132 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ea7c3b4-094d-4173-84ac-09beabea0bed None admin] PATH_INFO: `/v3/domains` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:47:37.115807 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ea7c3b4-094d-4173-84ac-09beabea0bed None admin] RBAC: Adding query filter params (name=Default) {{(pid=81767) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 15:47:37.116102 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ea7c3b4-094d-4173-84ac-09beabea0bed None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 15:47:37.117459 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ea7c3b4-094d-4173-84ac-09beabea0bed None admin] RBAC: Authorization granted {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 15:47:37.120734 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG dbcounter [-] [81767] Writing DB stats keystone:SELECT=100 {{(pid=81767) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 15:47:37.135398 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 320/645] 10.222.0.15 () {56 vars in 1109 bytes} [Mon Aug 22 15:47:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 15:47:37.135398 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ea7c3b4-094d-4173-84ac-09beabea0bed None admin] Authenticating user token {{(pid=81767) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 15:47:37.202211 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6605c3ee-d350-4eb4-ab92-0eb0abc55428 None admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 15:47:37.238919 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e0d007a-072c-4387-a04e-acf0877e49fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 15:47:37.239541 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e0d007a-072c-4387-a04e-acf0877e49fd None admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:47:37.240004 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e0d007a-072c-4387-a04e-acf0877e49fd None admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:47:37.240376 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e0d007a-072c-4387-a04e-acf0877e49fd None admin] PATH_INFO: `/v3/domains` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:47:37.241103 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e0d007a-072c-4387-a04e-acf0877e49fd None admin] RBAC: Adding query filter params (name=Default) {{(pid=81768) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 15:47:37.241405 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e0d007a-072c-4387-a04e-acf0877e49fd None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 15:47:37.242797 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e0d007a-072c-4387-a04e-acf0877e49fd None admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 15:47:37.249223 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ea7c3b4-094d-4173-84ac-09beabea0bed None admin] Validating token access rules against request {{(pid=81767) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 15:47:37.262094 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 326/646] 10.222.0.15 () {56 vars in 1109 bytes} [Mon Aug 22 15:47:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 15:47:37.262094 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e0d007a-072c-4387-a04e-acf0877e49fd None admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 15:47:37.298453 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4b0fbf5-9a72-4825-8557-56b5b6239d98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81767) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 15:47:37.299085 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4b0fbf5-9a72-4825-8557-56b5b6239d98 None admin] REQUEST_METHOD: `GET` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:47:37.299385 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4b0fbf5-9a72-4825-8557-56b5b6239d98 None admin] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:47:37.299650 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4b0fbf5-9a72-4825-8557-56b5b6239d98 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:47:37.305794 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=100 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 15:47:37.311020 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4b0fbf5-9a72-4825-8557-56b5b6239d98 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 22 15:47:37.315738 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application [None req-b4b0fbf5-9a72-4825-8557-56b5b6239d98 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 22 15:47:37.315738 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 15:47:37.315738 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 15:47:37.315738 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 15:47:37.315738 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 15:47:37.315738 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 15:47:37.315738 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 15:47:37.315738 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 15:47:37.315738 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 15:47:37.315738 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 15:47:37.315738 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 15:47:37.315738 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 15:47:37.315738 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 22 15:47:37.315738 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 22 15:47:37.315738 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 22 15:47:37.315738 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 22 15:47:37.316670 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 22 15:47:37.316670 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 22 15:47:37.316670 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 22 15:47:37.316670 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 22 15:47:37.316670 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 15:47:37.316670 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 15:47:37.316670 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 15:47:37.316670 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 15:47:37.316670 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 15:47:37.316670 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 15:47:37.316670 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 15:47:37.316670 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application with Lock( Aug 22 15:47:37.316670 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 15:47:37.316670 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return self._enter() Aug 22 15:47:37.316670 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 15:47:37.316670 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 15:47:37.317491 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 15:47:37.317491 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return self.creator() Aug 22 15:47:37.317491 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 15:47:37.317491 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application created_value = creator( Aug 22 15:47:37.317491 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 22 15:47:37.317491 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 22 15:47:37.317491 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 22 15:47:37.317491 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 22 15:47:37.317491 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 22 15:47:37.317491 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 22 15:47:37.317491 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 22 15:47:37.317491 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application Aug 22 15:47:37.317491 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 321/647] 10.222.0.15 () {56 vars in 1102 bytes} [Mon Aug 22 15:47:37 2022] GET /identity/v3/roles/service => generated 85 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 15:47:37.318164 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4b0fbf5-9a72-4825-8557-56b5b6239d98 None admin] Authenticating user token {{(pid=81767) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 15:47:37.372129 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e0d007a-072c-4387-a04e-acf0877e49fd None admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 15:47:37.388869 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG dbcounter [-] [81767] Writing DB stats keystone:SELECT=100 {{(pid=81767) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 15:47:37.409935 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-135695b3-a417-4189-98d8-6bf154d5ca87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 15:47:37.410590 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-135695b3-a417-4189-98d8-6bf154d5ca87 None admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:47:37.410878 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-135695b3-a417-4189-98d8-6bf154d5ca87 None admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:47:37.411149 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-135695b3-a417-4189-98d8-6bf154d5ca87 None admin] PATH_INFO: `/v3/projects` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:47:37.411879 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-135695b3-a417-4189-98d8-6bf154d5ca87 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81768) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 15:47:37.412182 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-135695b3-a417-4189-98d8-6bf154d5ca87 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 15:47:37.413629 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-135695b3-a417-4189-98d8-6bf154d5ca87 None admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 15:47:37.426578 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 327/648] 10.222.0.15 () {56 vars in 1148 bytes} [Mon Aug 22 15:47:37 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 15:47:37.433056 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-135695b3-a417-4189-98d8-6bf154d5ca87 None admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 15:47:37.439554 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4b0fbf5-9a72-4825-8557-56b5b6239d98 None admin] Validating token access rules against request {{(pid=81767) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 15:47:37.477139 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f80167d-c13d-4a6d-8386-ba6005f17b89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81767) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 15:47:37.477923 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f80167d-c13d-4a6d-8386-ba6005f17b89 None admin] REQUEST_METHOD: `GET` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:47:37.478199 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f80167d-c13d-4a6d-8386-ba6005f17b89 None admin] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:47:37.478471 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f80167d-c13d-4a6d-8386-ba6005f17b89 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:47:37.487849 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f80167d-c13d-4a6d-8386-ba6005f17b89 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 15:47:37.489649 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f80167d-c13d-4a6d-8386-ba6005f17b89 None admin] RBAC: Authorization granted {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 15:47:37.503122 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application [None req-7f80167d-c13d-4a6d-8386-ba6005f17b89 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Aug 22 15:47:37.503122 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 15:47:37.503122 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 15:47:37.503122 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 15:47:37.503122 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 15:47:37.503122 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 15:47:37.503122 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 15:47:37.503122 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 15:47:37.503122 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 15:47:37.503122 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 15:47:37.503122 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 15:47:37.503122 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 15:47:37.503122 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 22 15:47:37.503122 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 22 15:47:37.503122 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 22 15:47:37.503122 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 22 15:47:37.504317 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 15:47:37.504317 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 15:47:37.504317 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 22 15:47:37.504317 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 22 15:47:37.504317 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 22 15:47:37.504317 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 22 15:47:37.504317 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 15:47:37.504317 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 15:47:37.504317 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 15:47:37.504317 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 15:47:37.504317 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 15:47:37.504317 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application with Lock( Aug 22 15:47:37.504317 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 15:47:37.504317 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return self._enter() Aug 22 15:47:37.504317 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 15:47:37.504317 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 15:47:37.504317 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 15:47:37.504317 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return self.creator() Aug 22 15:47:37.505401 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 15:47:37.505401 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application created_value = creator( Aug 22 15:47:37.505401 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 22 15:47:37.505401 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 22 15:47:37.505401 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 22 15:47:37.505401 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 22 15:47:37.505401 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 22 15:47:37.505401 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 22 15:47:37.505401 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Aug 22 15:47:37.505401 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application Aug 22 15:47:37.505401 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 322/649] 10.222.0.15 () {56 vars in 1099 bytes} [Mon Aug 22 15:47:37 2022] GET /identity/v3/users/cinder => generated 84 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 15:47:37.505401 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f80167d-c13d-4a6d-8386-ba6005f17b89 None admin] Authenticating user token {{(pid=81767) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 15:47:37.542604 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-135695b3-a417-4189-98d8-6bf154d5ca87 None admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 15:47:37.544432 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=100 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 15:47:37.581623 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7695c97-1c1c-43dd-adc1-90865bf053aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 15:47:37.582252 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7695c97-1c1c-43dd-adc1-90865bf053aa None admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:47:37.582524 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7695c97-1c1c-43dd-adc1-90865bf053aa None admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:47:37.582790 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7695c97-1c1c-43dd-adc1-90865bf053aa None admin] PATH_INFO: `/v3/users/glance` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:47:37.593253 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7695c97-1c1c-43dd-adc1-90865bf053aa None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 15:47:37.594715 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7695c97-1c1c-43dd-adc1-90865bf053aa None admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 15:47:37.607038 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application [None req-d7695c97-1c1c-43dd-adc1-90865bf053aa None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Aug 22 15:47:37.607038 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 15:47:37.607038 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 15:47:37.607038 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 15:47:37.607038 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 15:47:37.607038 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 15:47:37.607038 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 15:47:37.607038 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 15:47:37.607038 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 15:47:37.607038 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 15:47:37.607038 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 15:47:37.607038 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 15:47:37.607038 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 22 15:47:37.607038 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 22 15:47:37.607038 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 22 15:47:37.607038 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 22 15:47:37.608187 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 15:47:37.608187 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 15:47:37.608187 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 22 15:47:37.608187 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 22 15:47:37.608187 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 22 15:47:37.608187 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 22 15:47:37.608187 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 15:47:37.608187 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 15:47:37.608187 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 15:47:37.608187 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 15:47:37.608187 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 15:47:37.608187 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application with Lock( Aug 22 15:47:37.608187 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 15:47:37.608187 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self._enter() Aug 22 15:47:37.608187 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 15:47:37.608187 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 15:47:37.608187 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 15:47:37.608187 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self.creator() Aug 22 15:47:37.609097 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 15:47:37.609097 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application created_value = creator( Aug 22 15:47:37.609097 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 22 15:47:37.609097 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 22 15:47:37.609097 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 22 15:47:37.609097 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 22 15:47:37.609097 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 22 15:47:37.609097 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 22 15:47:37.609097 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Aug 22 15:47:37.609097 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application Aug 22 15:47:37.609097 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 328/650] 10.222.0.15 () {56 vars in 1099 bytes} [Mon Aug 22 15:47:37 2022] GET /identity/v3/users/glance => generated 84 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 15:47:37.609097 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7695c97-1c1c-43dd-adc1-90865bf053aa None admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 15:47:37.626439 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f80167d-c13d-4a6d-8386-ba6005f17b89 None admin] Validating token access rules against request {{(pid=81767) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 15:47:37.644399 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG dbcounter [-] [81767] Writing DB stats keystone:SELECT=100 {{(pid=81767) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 15:47:37.667718 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-614ef55d-47f4-406c-8cec-3fbe33915b5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81767) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 15:47:37.668571 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-614ef55d-47f4-406c-8cec-3fbe33915b5e None admin] REQUEST_METHOD: `GET` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:47:37.669440 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-614ef55d-47f4-406c-8cec-3fbe33915b5e None admin] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:47:37.669946 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-614ef55d-47f4-406c-8cec-3fbe33915b5e None admin] PATH_INFO: `/v3/roles` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:47:37.671059 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-614ef55d-47f4-406c-8cec-3fbe33915b5e None admin] RBAC: Adding query filter params (name=service) {{(pid=81767) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 15:47:37.671684 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-614ef55d-47f4-406c-8cec-3fbe33915b5e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 15:47:37.673888 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-614ef55d-47f4-406c-8cec-3fbe33915b5e None admin] RBAC: Authorization granted {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 15:47:37.690256 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 323/651] 10.222.0.15 () {56 vars in 1103 bytes} [Mon Aug 22 15:47:37 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 15:47:37.690256 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-614ef55d-47f4-406c-8cec-3fbe33915b5e None admin] Authenticating user token {{(pid=81767) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 15:47:37.721963 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7695c97-1c1c-43dd-adc1-90865bf053aa None admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 15:47:37.759402 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc9a9b40-a1e1-45bd-ad5a-f8d251743d09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 15:47:37.760310 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc9a9b40-a1e1-45bd-ad5a-f8d251743d09 None admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:47:37.760590 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc9a9b40-a1e1-45bd-ad5a-f8d251743d09 None admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:47:37.760860 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc9a9b40-a1e1-45bd-ad5a-f8d251743d09 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:47:37.761607 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc9a9b40-a1e1-45bd-ad5a-f8d251743d09 None admin] RBAC: Adding query filter params (role.id=a5fbba1dc615490a96cde27e3a102eb6, scope.project.id=c4ee0413f9ef4d4fb1f95a660babe090, user.id=e4ef569e41fb46d0b721bb1c7663616f) {{(pid=81768) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 15:47:37.761908 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc9a9b40-a1e1-45bd-ad5a-f8d251743d09 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 15:47:37.763417 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc9a9b40-a1e1-45bd-ad5a-f8d251743d09 None admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 15:47:37.770239 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 329/652] 10.222.0.15 () {56 vars in 1374 bytes} [Mon Aug 22 15:47:37 2022] GET /identity/v3/role_assignments?user.id=e4ef569e41fb46d0b721bb1c7663616f&scope.project.id=c4ee0413f9ef4d4fb1f95a660babe090&role.id=a5fbba1dc615490a96cde27e3a102eb6 => generated 608 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 15:47:37.771011 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc9a9b40-a1e1-45bd-ad5a-f8d251743d09 None admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 15:47:37.809927 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=100 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 15:47:37.821421 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-614ef55d-47f4-406c-8cec-3fbe33915b5e None admin] Validating token access rules against request {{(pid=81767) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 15:47:37.878367 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6c15e40-00e1-4b9d-bbea-2c28ddc611cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81767) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 15:47:37.879341 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6c15e40-00e1-4b9d-bbea-2c28ddc611cc None admin] REQUEST_METHOD: `GET` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:47:37.879828 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6c15e40-00e1-4b9d-bbea-2c28ddc611cc None admin] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:47:37.880375 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6c15e40-00e1-4b9d-bbea-2c28ddc611cc None admin] PATH_INFO: `/v3/users` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:47:37.881409 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6c15e40-00e1-4b9d-bbea-2c28ddc611cc None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=81767) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 15:47:37.881894 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6c15e40-00e1-4b9d-bbea-2c28ddc611cc None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 15:47:37.883933 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6c15e40-00e1-4b9d-bbea-2c28ddc611cc None admin] RBAC: Authorization granted {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 15:47:37.898646 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc9a9b40-a1e1-45bd-ad5a-f8d251743d09 None admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 15:47:37.912319 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 324/653] 10.222.0.15 () {56 vars in 1137 bytes} [Mon Aug 22 15:47:37 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 15:47:37.913702 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6c15e40-00e1-4b9d-bbea-2c28ddc611cc None admin] Authenticating user token {{(pid=81767) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 15:47:37.940940 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d38aaa29-5ae4-42ef-8dcd-a055daa72139 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 15:47:37.941577 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d38aaa29-5ae4-42ef-8dcd-a055daa72139 None admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:47:37.941940 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d38aaa29-5ae4-42ef-8dcd-a055daa72139 None admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:47:37.942209 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d38aaa29-5ae4-42ef-8dcd-a055daa72139 None admin] PATH_INFO: `/v3/users` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:47:37.942934 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d38aaa29-5ae4-42ef-8dcd-a055daa72139 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=81768) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 15:47:37.943232 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d38aaa29-5ae4-42ef-8dcd-a055daa72139 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 15:47:37.945873 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d38aaa29-5ae4-42ef-8dcd-a055daa72139 None admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 15:47:37.962575 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG dbcounter [-] [81767] Writing DB stats keystone:SELECT=100 {{(pid=81767) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 15:47:37.969525 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 330/654] 10.222.0.15 () {56 vars in 1137 bytes} [Mon Aug 22 15:47:37 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 15:47:37.975329 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d38aaa29-5ae4-42ef-8dcd-a055daa72139 None admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 15:47:38.058061 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6c15e40-00e1-4b9d-bbea-2c28ddc611cc None admin] Validating token access rules against request {{(pid=81767) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 15:47:38.086965 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=100 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 15:47:38.099983 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1531b52f-82cc-497f-ac0a-06e68fcf252f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81767) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 15:47:38.100620 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1531b52f-82cc-497f-ac0a-06e68fcf252f None admin] REQUEST_METHOD: `GET` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:47:38.100940 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1531b52f-82cc-497f-ac0a-06e68fcf252f None admin] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:47:38.101203 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1531b52f-82cc-497f-ac0a-06e68fcf252f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:47:38.106743 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d38aaa29-5ae4-42ef-8dcd-a055daa72139 None admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 15:47:38.113715 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1531b52f-82cc-497f-ac0a-06e68fcf252f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 15:47:38.115340 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1531b52f-82cc-497f-ac0a-06e68fcf252f None admin] RBAC: Authorization granted {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 15:47:38.131644 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application [None req-1531b52f-82cc-497f-ac0a-06e68fcf252f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 22 15:47:38.131644 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 15:47:38.131644 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 15:47:38.131644 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 15:47:38.131644 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 15:47:38.131644 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 15:47:38.131644 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 15:47:38.131644 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 15:47:38.131644 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 15:47:38.131644 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 15:47:38.131644 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 15:47:38.131644 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 15:47:38.131644 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 22 15:47:38.131644 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 22 15:47:38.131644 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 22 15:47:38.131644 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 22 15:47:38.132591 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 15:47:38.132591 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 15:47:38.132591 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 15:47:38.132591 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 15:47:38.132591 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 15:47:38.132591 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 15:47:38.132591 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 15:47:38.132591 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application with Lock( Aug 22 15:47:38.132591 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 15:47:38.132591 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return self._enter() Aug 22 15:47:38.132591 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 15:47:38.132591 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 15:47:38.132591 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 15:47:38.132591 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return self.creator() Aug 22 15:47:38.132591 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 15:47:38.132591 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application created_value = creator( Aug 22 15:47:38.132591 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 22 15:47:38.133476 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 22 15:47:38.133476 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 22 15:47:38.133476 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application Aug 22 15:47:38.133476 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 325/655] 10.222.0.15 () {56 vars in 1108 bytes} [Mon Aug 22 15:47:37 2022] GET /identity/v3/domains/Default => generated 87 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 15:47:38.133691 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1531b52f-82cc-497f-ac0a-06e68fcf252f None admin] Authenticating user token {{(pid=81767) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 15:47:38.148226 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25ec56d6-62b0-46b2-b1b7-9fbde9dbb15f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 15:47:38.149525 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25ec56d6-62b0-46b2-b1b7-9fbde9dbb15f None admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:47:38.150300 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25ec56d6-62b0-46b2-b1b7-9fbde9dbb15f None admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:47:38.150770 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25ec56d6-62b0-46b2-b1b7-9fbde9dbb15f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:47:38.227085 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25ec56d6-62b0-46b2-b1b7-9fbde9dbb15f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 15:47:38.228644 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25ec56d6-62b0-46b2-b1b7-9fbde9dbb15f None admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 15:47:38.233915 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG dbcounter [-] [81767] Writing DB stats keystone:SELECT=100 {{(pid=81767) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 15:47:38.251865 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1531b52f-82cc-497f-ac0a-06e68fcf252f None admin] Validating token access rules against request {{(pid=81767) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 15:47:38.299614 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da672daa-279b-4561-9975-dc2c33a741bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81767) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 15:47:38.300231 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da672daa-279b-4561-9975-dc2c33a741bb None admin] REQUEST_METHOD: `GET` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:47:38.300603 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da672daa-279b-4561-9975-dc2c33a741bb None admin] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:47:38.300750 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da672daa-279b-4561-9975-dc2c33a741bb None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:47:38.342914 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=100 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 15:47:38.375199 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 331/656] 10.222.0.15 () {58 vars in 1282 bytes} [Mon Aug 22 15:47:37 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 405 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 22 15:47:38.377435 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-560603b2-4168-43fc-86a3-42a5f00cc063 None None] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:47:38.378002 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-560603b2-4168-43fc-86a3-42a5f00cc063 None None] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:47:38.378488 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-560603b2-4168-43fc-86a3-42a5f00cc063 None None] PATH_INFO: `/` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:47:38.386413 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 332/657] 10.222.0.15 () {54 vars in 927 bytes} [Mon Aug 22 15:47:38 2022] GET /identity => generated 270 bytes in 10 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 15:47:38.387546 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-560603b2-4168-43fc-86a3-42a5f00cc063 None None] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 15:47:38.394674 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da672daa-279b-4561-9975-dc2c33a741bb None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 15:47:38.397003 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da672daa-279b-4561-9975-dc2c33a741bb None admin] RBAC: Authorization granted {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 15:47:38.509747 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG dbcounter [-] [81767] Writing DB stats keystone:SELECT=100 {{(pid=81767) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 15:47:38.520740 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-560603b2-4168-43fc-86a3-42a5f00cc063 None None] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 15:47:38.523801 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 326/658] 10.222.0.15 () {58 vars in 1282 bytes} [Mon Aug 22 15:47:38 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 390 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 22 15:47:38.524602 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da672daa-279b-4561-9975-dc2c33a741bb None admin] Authenticating user token {{(pid=81767) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 15:47:38.573146 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8980800-345b-4eb8-96b7-aba8b99a67ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 15:47:38.573803 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8980800-345b-4eb8-96b7-aba8b99a67ba None admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:47:38.574087 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8980800-345b-4eb8-96b7-aba8b99a67ba None admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:47:38.574362 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8980800-345b-4eb8-96b7-aba8b99a67ba None admin] PATH_INFO: `/v3/domains` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:47:38.575070 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8980800-345b-4eb8-96b7-aba8b99a67ba None admin] RBAC: Adding query filter params (name=Default) {{(pid=81768) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 15:47:38.575423 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8980800-345b-4eb8-96b7-aba8b99a67ba None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 15:47:38.576908 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8980800-345b-4eb8-96b7-aba8b99a67ba None admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 15:47:38.590441 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 333/659] 10.222.0.15 () {56 vars in 1109 bytes} [Mon Aug 22 15:47:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 15:47:38.591496 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b5a1a3c-47d9-4bf9-b9ae-44467a900510 None None] REQUEST_METHOD: `POST` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:47:38.591797 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b5a1a3c-47d9-4bf9-b9ae-44467a900510 None None] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:47:38.592092 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b5a1a3c-47d9-4bf9-b9ae-44467a900510 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:47:38.611785 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.auth.core [None req-3b5a1a3c-47d9-4bf9-b9ae-44467a900510 None None] MFA Rules not processed for user `6b2199ae0fb0480aa917ada391f5ec54`. Rule list: `[]` (Enabled: `True`). {{(pid=81768) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 15:47:38.613343 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-3b5a1a3c-47d9-4bf9-b9ae-44467a900510 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 15:47:38.649563 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da672daa-279b-4561-9975-dc2c33a741bb None admin] Validating token access rules against request {{(pid=81767) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 15:47:38.658942 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=100 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 15:47:38.689000 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16262360-c9c1-4cbc-99f0-fe1df1eefd5c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81767) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 15:47:38.699554 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 334/660] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Aug 22 15:47:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 22 15:47:38.699712 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16262360-c9c1-4cbc-99f0-fe1df1eefd5c None admin] REQUEST_METHOD: `GET` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:47:38.699712 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16262360-c9c1-4cbc-99f0-fe1df1eefd5c None admin] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:47:38.699712 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16262360-c9c1-4cbc-99f0-fe1df1eefd5c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:47:38.699947 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b5a1a3c-47d9-4bf9-b9ae-44467a900510 None None] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 15:47:38.716901 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16262360-c9c1-4cbc-99f0-fe1df1eefd5c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 15:47:38.719242 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16262360-c9c1-4cbc-99f0-fe1df1eefd5c None admin] RBAC: Authorization granted {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 15:47:38.740676 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application [None req-16262360-c9c1-4cbc-99f0-fe1df1eefd5c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 22 15:47:38.740676 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 15:47:38.740676 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 15:47:38.740676 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 15:47:38.740676 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 15:47:38.740676 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 15:47:38.740676 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 15:47:38.740676 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 15:47:38.740676 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 15:47:38.740676 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 15:47:38.740676 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 15:47:38.740676 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 15:47:38.740676 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 22 15:47:38.740676 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 22 15:47:38.740676 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 22 15:47:38.740676 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 22 15:47:38.741653 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 15:47:38.741653 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 15:47:38.741653 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 15:47:38.741653 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 15:47:38.741653 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 15:47:38.741653 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 15:47:38.741653 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 15:47:38.741653 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application with Lock( Aug 22 15:47:38.741653 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 15:47:38.741653 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return self._enter() Aug 22 15:47:38.741653 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 15:47:38.741653 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 15:47:38.741653 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 15:47:38.741653 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return self.creator() Aug 22 15:47:38.741653 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 15:47:38.741653 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application created_value = creator( Aug 22 15:47:38.741653 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 22 15:47:38.742531 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 22 15:47:38.742531 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 22 15:47:38.742531 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application Aug 22 15:47:38.743130 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 327/661] 10.222.0.15 () {56 vars in 1108 bytes} [Mon Aug 22 15:47:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 15:47:38.743681 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16262360-c9c1-4cbc-99f0-fe1df1eefd5c None admin] Authenticating user token {{(pid=81767) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 15:47:38.802876 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG dbcounter [-] [81767] Writing DB stats keystone:SELECT=100 {{(pid=81767) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 15:47:38.818326 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b5a1a3c-47d9-4bf9-b9ae-44467a900510 None None] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 15:47:38.855934 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29d7497a-1ca1-42ca-973a-21cfcd1c579a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 15:47:38.856602 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29d7497a-1ca1-42ca-973a-21cfcd1c579a None admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:47:38.856879 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29d7497a-1ca1-42ca-973a-21cfcd1c579a None admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:47:38.857149 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29d7497a-1ca1-42ca-973a-21cfcd1c579a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:47:38.870497 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29d7497a-1ca1-42ca-973a-21cfcd1c579a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 15:47:38.872402 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29d7497a-1ca1-42ca-973a-21cfcd1c579a None admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 15:47:38.886117 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16262360-c9c1-4cbc-99f0-fe1df1eefd5c None admin] Validating token access rules against request {{(pid=81767) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 15:47:38.890370 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application [None req-29d7497a-1ca1-42ca-973a-21cfcd1c579a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 22 15:47:38.890370 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 15:47:38.890370 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 15:47:38.890370 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 15:47:38.890370 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 15:47:38.890370 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 15:47:38.890370 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 15:47:38.890370 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 15:47:38.890370 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 15:47:38.890370 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 15:47:38.890370 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 15:47:38.890370 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 15:47:38.890370 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 22 15:47:38.890370 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 22 15:47:38.890370 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 22 15:47:38.890370 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 22 15:47:38.891295 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 15:47:38.891295 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 15:47:38.891295 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 15:47:38.891295 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 15:47:38.891295 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 15:47:38.891295 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 15:47:38.891295 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 15:47:38.891295 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application with Lock( Aug 22 15:47:38.891295 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 15:47:38.891295 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self._enter() Aug 22 15:47:38.891295 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 15:47:38.891295 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 15:47:38.891295 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 15:47:38.891295 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self.creator() Aug 22 15:47:38.891295 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 15:47:38.891295 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application created_value = creator( Aug 22 15:47:38.891295 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 22 15:47:38.892189 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 22 15:47:38.892189 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 22 15:47:38.892189 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application Aug 22 15:47:38.896463 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 335/662] 10.222.0.15 () {56 vars in 1108 bytes} [Mon Aug 22 15:47:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 15:47:38.897765 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29d7497a-1ca1-42ca-973a-21cfcd1c579a None admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 15:47:38.934345 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73365846-7607-46a4-8926-703b31381ccf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81767) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 15:47:38.934991 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73365846-7607-46a4-8926-703b31381ccf None admin] REQUEST_METHOD: `GET` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:47:38.935264 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73365846-7607-46a4-8926-703b31381ccf None admin] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:47:38.935553 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73365846-7607-46a4-8926-703b31381ccf None admin] PATH_INFO: `/v3/users/neutron` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:47:38.941579 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=100 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 15:47:38.949426 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73365846-7607-46a4-8926-703b31381ccf None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 15:47:38.952132 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73365846-7607-46a4-8926-703b31381ccf None admin] RBAC: Authorization granted {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 15:47:38.970532 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application [None req-73365846-7607-46a4-8926-703b31381ccf None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Aug 22 15:47:38.970532 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 15:47:38.970532 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 15:47:38.970532 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 15:47:38.970532 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 15:47:38.970532 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 15:47:38.970532 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 15:47:38.970532 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 15:47:38.970532 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 15:47:38.970532 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 15:47:38.970532 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 15:47:38.970532 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 15:47:38.970532 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 22 15:47:38.970532 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 22 15:47:38.970532 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 22 15:47:38.970532 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 22 15:47:38.971590 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 15:47:38.971590 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 15:47:38.971590 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 22 15:47:38.971590 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 22 15:47:38.971590 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 22 15:47:38.971590 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 22 15:47:38.971590 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 15:47:38.971590 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 15:47:38.971590 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 15:47:38.971590 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 15:47:38.971590 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 15:47:38.971590 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application with Lock( Aug 22 15:47:38.971590 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 15:47:38.971590 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return self._enter() Aug 22 15:47:38.971590 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 15:47:38.971590 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 15:47:38.971590 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 15:47:38.971590 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return self.creator() Aug 22 15:47:38.972511 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 15:47:38.972511 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application created_value = creator( Aug 22 15:47:38.972511 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 22 15:47:38.972511 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 22 15:47:38.972511 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 22 15:47:38.972511 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 22 15:47:38.972511 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 22 15:47:38.972511 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 22 15:47:38.972511 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Aug 22 15:47:38.972511 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application Aug 22 15:47:38.973990 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 328/663] 10.222.0.15 () {56 vars in 1102 bytes} [Mon Aug 22 15:47:38 2022] GET /identity/v3/users/neutron => generated 85 bytes in 229 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 15:47:38.973990 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73365846-7607-46a4-8926-703b31381ccf None admin] Authenticating user token {{(pid=81767) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 15:47:39.038298 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29d7497a-1ca1-42ca-973a-21cfcd1c579a None admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 15:47:39.076975 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa3cebf9-a727-4b2d-b168-487295eb0019 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 15:47:39.077592 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa3cebf9-a727-4b2d-b168-487295eb0019 None admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:47:39.077867 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa3cebf9-a727-4b2d-b168-487295eb0019 None admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:47:39.078142 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa3cebf9-a727-4b2d-b168-487295eb0019 None admin] PATH_INFO: `/v3/domains` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:47:39.078788 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa3cebf9-a727-4b2d-b168-487295eb0019 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81768) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 15:47:39.079087 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa3cebf9-a727-4b2d-b168-487295eb0019 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 15:47:39.080521 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa3cebf9-a727-4b2d-b168-487295eb0019 None admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 15:47:39.094496 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 336/664] 10.222.0.15 () {56 vars in 1109 bytes} [Mon Aug 22 15:47:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 15:47:39.095728 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa3cebf9-a727-4b2d-b168-487295eb0019 None admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 15:47:39.101000 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG dbcounter [-] [81767] Writing DB stats keystone:SELECT=100 {{(pid=81767) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 15:47:39.105559 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73365846-7607-46a4-8926-703b31381ccf None admin] Validating token access rules against request {{(pid=81767) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 15:47:39.146368 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c9752a8-274c-4f16-947c-81013b46f2f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81767) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 15:47:39.146892 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c9752a8-274c-4f16-947c-81013b46f2f6 None admin] REQUEST_METHOD: `GET` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:47:39.147505 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c9752a8-274c-4f16-947c-81013b46f2f6 None admin] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:47:39.147833 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c9752a8-274c-4f16-947c-81013b46f2f6 None admin] PATH_INFO: `/v3/domains` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:47:39.148850 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c9752a8-274c-4f16-947c-81013b46f2f6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81767) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 15:47:39.149149 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c9752a8-274c-4f16-947c-81013b46f2f6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 15:47:39.150670 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c9752a8-274c-4f16-947c-81013b46f2f6 None admin] RBAC: Authorization granted {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 15:47:39.163554 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 329/665] 10.222.0.15 () {56 vars in 1109 bytes} [Mon Aug 22 15:47:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 15:47:39.163554 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c9752a8-274c-4f16-947c-81013b46f2f6 None admin] Authenticating user token {{(pid=81767) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 15:47:39.202508 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=100 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 15:47:39.221204 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa3cebf9-a727-4b2d-b168-487295eb0019 None admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 15:47:39.262823 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-863ee8c6-4094-4ad2-9f81-3e53c7dd6a61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 15:47:39.263800 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-863ee8c6-4094-4ad2-9f81-3e53c7dd6a61 None admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:47:39.263800 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-863ee8c6-4094-4ad2-9f81-3e53c7dd6a61 None admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:47:39.264114 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-863ee8c6-4094-4ad2-9f81-3e53c7dd6a61 None admin] PATH_INFO: `/v3/users` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:47:39.264937 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-863ee8c6-4094-4ad2-9f81-3e53c7dd6a61 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=81768) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 15:47:39.265315 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-863ee8c6-4094-4ad2-9f81-3e53c7dd6a61 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 15:47:39.266815 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-863ee8c6-4094-4ad2-9f81-3e53c7dd6a61 None admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 15:47:39.286803 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 337/666] 10.222.0.15 () {56 vars in 1139 bytes} [Mon Aug 22 15:47:39 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 15:47:39.287756 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-863ee8c6-4094-4ad2-9f81-3e53c7dd6a61 None admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 15:47:39.297841 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c9752a8-274c-4f16-947c-81013b46f2f6 None admin] Validating token access rules against request {{(pid=81767) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 15:47:39.344272 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c83cc64-b872-494c-b58a-ecfca8d56eaa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81767) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 15:47:39.345016 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c83cc64-b872-494c-b58a-ecfca8d56eaa None admin] REQUEST_METHOD: `GET` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:47:39.345400 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c83cc64-b872-494c-b58a-ecfca8d56eaa None admin] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:47:39.345689 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c83cc64-b872-494c-b58a-ecfca8d56eaa None admin] PATH_INFO: `/v3/roles/service` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:47:39.364806 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c83cc64-b872-494c-b58a-ecfca8d56eaa None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 22 15:47:39.373745 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application [None req-4c83cc64-b872-494c-b58a-ecfca8d56eaa None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 22 15:47:39.373745 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 15:47:39.373745 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 15:47:39.373745 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 15:47:39.373745 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 15:47:39.373745 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 15:47:39.373745 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 15:47:39.373745 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 15:47:39.373745 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 15:47:39.373745 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 15:47:39.373745 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 15:47:39.373745 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 15:47:39.373745 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 22 15:47:39.373745 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 22 15:47:39.373745 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 22 15:47:39.373745 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 22 15:47:39.374953 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 22 15:47:39.374953 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 22 15:47:39.374953 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 22 15:47:39.374953 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 22 15:47:39.374953 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 15:47:39.374953 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 15:47:39.374953 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 15:47:39.374953 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 15:47:39.374953 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 15:47:39.374953 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 15:47:39.374953 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 15:47:39.374953 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application with Lock( Aug 22 15:47:39.374953 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 15:47:39.374953 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return self._enter() Aug 22 15:47:39.374953 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 15:47:39.374953 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 15:47:39.376095 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 15:47:39.376095 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return self.creator() Aug 22 15:47:39.376095 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 15:47:39.376095 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application created_value = creator( Aug 22 15:47:39.376095 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 22 15:47:39.376095 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 22 15:47:39.376095 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 22 15:47:39.376095 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 22 15:47:39.376095 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 22 15:47:39.376095 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 22 15:47:39.376095 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 22 15:47:39.376095 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application Aug 22 15:47:39.376818 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 330/667] 10.222.0.15 () {56 vars in 1102 bytes} [Mon Aug 22 15:47:39 2022] GET /identity/v3/roles/service => generated 85 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 15:47:39.377657 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c83cc64-b872-494c-b58a-ecfca8d56eaa None admin] Authenticating user token {{(pid=81767) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 15:47:39.405277 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG dbcounter [-] [81767] Writing DB stats keystone:SELECT=100 {{(pid=81767) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 15:47:39.421780 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-863ee8c6-4094-4ad2-9f81-3e53c7dd6a61 None admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 15:47:39.461743 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20407ffc-4ecd-4b3e-9ee8-820b50acb23b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 15:47:39.462452 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20407ffc-4ecd-4b3e-9ee8-820b50acb23b None admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:47:39.462730 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20407ffc-4ecd-4b3e-9ee8-820b50acb23b None admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:47:39.463060 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20407ffc-4ecd-4b3e-9ee8-820b50acb23b None admin] PATH_INFO: `/v3/projects/service` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:47:39.471753 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20407ffc-4ecd-4b3e-9ee8-820b50acb23b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 15:47:39.473344 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20407ffc-4ecd-4b3e-9ee8-820b50acb23b None admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 15:47:39.485758 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application [None req-20407ffc-4ecd-4b3e-9ee8-820b50acb23b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 22 15:47:39.485758 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 15:47:39.485758 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 15:47:39.485758 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 15:47:39.485758 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 15:47:39.485758 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 15:47:39.485758 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 15:47:39.485758 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 15:47:39.485758 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 15:47:39.485758 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 15:47:39.485758 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 15:47:39.485758 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 15:47:39.485758 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 22 15:47:39.485758 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 22 15:47:39.485758 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 22 15:47:39.486728 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 22 15:47:39.486728 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 15:47:39.486728 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 15:47:39.486728 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 15:47:39.486728 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 15:47:39.486728 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 15:47:39.486728 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 15:47:39.486728 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 15:47:39.486728 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application with Lock( Aug 22 15:47:39.486728 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 15:47:39.486728 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self._enter() Aug 22 15:47:39.486728 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 15:47:39.486728 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 15:47:39.486728 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 15:47:39.486728 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self.creator() Aug 22 15:47:39.486728 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 15:47:39.486728 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application created_value = creator( Aug 22 15:47:39.487641 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 22 15:47:39.487641 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 22 15:47:39.487641 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 22 15:47:39.487641 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 22 15:47:39.487641 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 22 15:47:39.487641 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 22 15:47:39.487641 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 22 15:47:39.487641 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application Aug 22 15:47:39.487641 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 338/668] 10.222.0.15 () {56 vars in 1111 bytes} [Mon Aug 22 15:47:39 2022] GET /identity/v3/projects/service => generated 88 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 15:47:39.489188 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20407ffc-4ecd-4b3e-9ee8-820b50acb23b None admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 15:47:39.492959 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=100 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 15:47:39.498510 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c83cc64-b872-494c-b58a-ecfca8d56eaa None admin] Validating token access rules against request {{(pid=81767) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 15:47:39.537581 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8ac5381-4ceb-4806-abc8-1df2efdaf862 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81767) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 15:47:39.538261 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8ac5381-4ceb-4806-abc8-1df2efdaf862 None admin] REQUEST_METHOD: `GET` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:47:39.538536 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8ac5381-4ceb-4806-abc8-1df2efdaf862 None admin] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:47:39.538804 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8ac5381-4ceb-4806-abc8-1df2efdaf862 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:47:39.547412 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8ac5381-4ceb-4806-abc8-1df2efdaf862 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 15:47:39.548863 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8ac5381-4ceb-4806-abc8-1df2efdaf862 None admin] RBAC: Authorization granted {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 15:47:39.561435 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application [None req-e8ac5381-4ceb-4806-abc8-1df2efdaf862 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 22 15:47:39.561435 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 15:47:39.561435 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 15:47:39.561435 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 15:47:39.561435 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 15:47:39.561435 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 15:47:39.561435 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 15:47:39.561435 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 15:47:39.561435 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 15:47:39.561435 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 15:47:39.561435 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 15:47:39.561435 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 15:47:39.561435 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 22 15:47:39.561435 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 22 15:47:39.561435 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 22 15:47:39.562396 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 22 15:47:39.562396 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 15:47:39.562396 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 15:47:39.562396 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 15:47:39.562396 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 15:47:39.562396 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 15:47:39.562396 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 15:47:39.562396 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 15:47:39.562396 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application with Lock( Aug 22 15:47:39.562396 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 15:47:39.562396 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return self._enter() Aug 22 15:47:39.562396 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 15:47:39.562396 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 15:47:39.562396 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 15:47:39.562396 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return self.creator() Aug 22 15:47:39.562396 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 15:47:39.562396 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application created_value = creator( Aug 22 15:47:39.564017 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 22 15:47:39.564017 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 22 15:47:39.564017 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 22 15:47:39.564017 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 22 15:47:39.564017 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 22 15:47:39.564017 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 22 15:47:39.564017 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 22 15:47:39.564017 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application Aug 22 15:47:39.564017 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 331/669] 10.222.0.15 () {56 vars in 1111 bytes} [Mon Aug 22 15:47:39 2022] GET /identity/v3/projects/service => generated 88 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 15:47:39.564017 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8ac5381-4ceb-4806-abc8-1df2efdaf862 None admin] Authenticating user token {{(pid=81767) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 15:47:39.608639 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20407ffc-4ecd-4b3e-9ee8-820b50acb23b None admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 15:47:39.643847 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d90b1440-d3c0-4380-8eea-69894a24c15d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 15:47:39.644482 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d90b1440-d3c0-4380-8eea-69894a24c15d None admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:47:39.644789 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d90b1440-d3c0-4380-8eea-69894a24c15d None admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:47:39.645778 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d90b1440-d3c0-4380-8eea-69894a24c15d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:47:39.665448 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG dbcounter [-] [81767] Writing DB stats keystone:SELECT=100 {{(pid=81767) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 15:47:39.687706 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8ac5381-4ceb-4806-abc8-1df2efdaf862 None admin] Validating token access rules against request {{(pid=81767) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 15:47:39.725094 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d90b1440-d3c0-4380-8eea-69894a24c15d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 15:47:39.726687 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d90b1440-d3c0-4380-8eea-69894a24c15d None admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 15:47:39.727191 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-767f64e8-1f06-45f1-bcde-5073126cc859 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81767) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 15:47:39.727826 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-767f64e8-1f06-45f1-bcde-5073126cc859 None admin] REQUEST_METHOD: `GET` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:47:39.728097 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-767f64e8-1f06-45f1-bcde-5073126cc859 None admin] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:47:39.728361 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-767f64e8-1f06-45f1-bcde-5073126cc859 None admin] PATH_INFO: `/v3/roles` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:47:39.729863 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-767f64e8-1f06-45f1-bcde-5073126cc859 None admin] RBAC: Adding query filter params (name=service) {{(pid=81767) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 15:47:39.730388 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=100 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 15:47:39.731098 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-767f64e8-1f06-45f1-bcde-5073126cc859 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 15:47:39.732496 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-767f64e8-1f06-45f1-bcde-5073126cc859 None admin] RBAC: Authorization granted {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 15:47:39.747132 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 332/670] 10.222.0.15 () {56 vars in 1103 bytes} [Mon Aug 22 15:47:39 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 15:47:39.747132 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-767f64e8-1f06-45f1-bcde-5073126cc859 None admin] Authenticating user token {{(pid=81767) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 15:47:39.848016 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 339/671] 10.222.0.15 () {58 vars in 1282 bytes} [Mon Aug 22 15:47:39 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 356 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 22 15:47:39.848016 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d90b1440-d3c0-4380-8eea-69894a24c15d None admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 15:47:39.858160 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-767f64e8-1f06-45f1-bcde-5073126cc859 None admin] Validating token access rules against request {{(pid=81767) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 15:47:39.901595 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7891b590-1de5-4cef-b554-620456480c98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81767) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 15:47:39.902239 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7891b590-1de5-4cef-b554-620456480c98 None admin] REQUEST_METHOD: `GET` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:47:39.902515 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7891b590-1de5-4cef-b554-620456480c98 None admin] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:47:39.902783 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7891b590-1de5-4cef-b554-620456480c98 None admin] PATH_INFO: `/v3/projects` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:47:39.903529 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7891b590-1de5-4cef-b554-620456480c98 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81767) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 15:47:39.903833 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7891b590-1de5-4cef-b554-620456480c98 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 15:47:39.905300 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7891b590-1de5-4cef-b554-620456480c98 None admin] RBAC: Authorization granted {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 15:47:39.920378 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 333/672] 10.222.0.15 () {56 vars in 1148 bytes} [Mon Aug 22 15:47:39 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 15:47:39.921266 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7891b590-1de5-4cef-b554-620456480c98 None admin] Authenticating user token {{(pid=81767) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 15:47:39.924366 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG dbcounter [-] [81767] Writing DB stats keystone:SELECT=100 {{(pid=81767) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 15:47:39.989141 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d90b1440-d3c0-4380-8eea-69894a24c15d None admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 15:47:39.991846 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=100 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 15:47:40.030008 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-694647fb-8c04-4ded-90db-022c25f1206f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 15:47:40.030843 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-694647fb-8c04-4ded-90db-022c25f1206f None admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:47:40.031305 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-694647fb-8c04-4ded-90db-022c25f1206f None admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:47:40.031684 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-694647fb-8c04-4ded-90db-022c25f1206f None admin] PATH_INFO: `/v3/projects` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:47:40.032514 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-694647fb-8c04-4ded-90db-022c25f1206f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81768) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 15:47:40.032679 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7891b590-1de5-4cef-b554-620456480c98 None admin] Validating token access rules against request {{(pid=81767) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 15:47:40.033001 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-694647fb-8c04-4ded-90db-022c25f1206f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 15:47:40.034792 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-694647fb-8c04-4ded-90db-022c25f1206f None admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 15:47:40.048946 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 340/673] 10.222.0.15 () {56 vars in 1148 bytes} [Mon Aug 22 15:47:39 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 15:47:40.049835 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-694647fb-8c04-4ded-90db-022c25f1206f None admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 15:47:40.074582 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4832f578-dab8-4b32-b916-62128b21416b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81767) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 15:47:40.075231 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4832f578-dab8-4b32-b916-62128b21416b None admin] REQUEST_METHOD: `GET` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:47:40.075557 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4832f578-dab8-4b32-b916-62128b21416b None admin] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:47:40.075827 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4832f578-dab8-4b32-b916-62128b21416b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:47:40.087148 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4832f578-dab8-4b32-b916-62128b21416b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 15:47:40.089152 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4832f578-dab8-4b32-b916-62128b21416b None admin] RBAC: Authorization granted {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 15:47:40.106344 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application [None req-4832f578-dab8-4b32-b916-62128b21416b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 22 15:47:40.106344 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 15:47:40.106344 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 15:47:40.106344 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 15:47:40.106344 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 15:47:40.106344 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 15:47:40.106344 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 15:47:40.106344 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 15:47:40.106344 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 15:47:40.106344 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 15:47:40.106344 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 15:47:40.106344 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 15:47:40.106344 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 22 15:47:40.106344 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 22 15:47:40.106344 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 22 15:47:40.106344 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 22 15:47:40.107448 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 15:47:40.107448 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 15:47:40.107448 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 15:47:40.107448 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 15:47:40.107448 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 15:47:40.107448 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 15:47:40.107448 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 15:47:40.107448 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application with Lock( Aug 22 15:47:40.107448 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 15:47:40.107448 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return self._enter() Aug 22 15:47:40.107448 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 15:47:40.107448 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 15:47:40.107448 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 15:47:40.107448 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return self.creator() Aug 22 15:47:40.107448 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 15:47:40.107448 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application created_value = creator( Aug 22 15:47:40.107448 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 22 15:47:40.108319 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 22 15:47:40.108319 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 22 15:47:40.108319 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application Aug 22 15:47:40.108319 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 334/674] 10.222.0.15 () {56 vars in 1108 bytes} [Mon Aug 22 15:47:39 2022] GET /identity/v3/domains/Default => generated 87 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 15:47:40.108858 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaf546fd-1262-41e3-802f-377d2c84b885 None None] REQUEST_METHOD: `GET` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:47:40.109108 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaf546fd-1262-41e3-802f-377d2c84b885 None None] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:47:40.109378 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaf546fd-1262-41e3-802f-377d2c84b885 None None] PATH_INFO: `/` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:47:40.110822 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 335/675] 10.222.0.15 () {54 vars in 927 bytes} [Mon Aug 22 15:47:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 15:47:40.111462 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaf546fd-1262-41e3-802f-377d2c84b885 None None] Authenticating user token {{(pid=81767) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 15:47:40.168959 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-694647fb-8c04-4ded-90db-022c25f1206f None admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 15:47:40.175659 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG dbcounter [-] [81767] Writing DB stats keystone:SELECT=100 {{(pid=81767) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 15:47:40.205268 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7aa3be2-03bb-4a6d-8131-ddc807e18ce3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 15:47:40.205913 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7aa3be2-03bb-4a6d-8131-ddc807e18ce3 None admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:47:40.206188 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7aa3be2-03bb-4a6d-8131-ddc807e18ce3 None admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:47:40.206455 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7aa3be2-03bb-4a6d-8131-ddc807e18ce3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:47:40.219869 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7aa3be2-03bb-4a6d-8131-ddc807e18ce3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 15:47:40.221476 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7aa3be2-03bb-4a6d-8131-ddc807e18ce3 None admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 15:47:40.226515 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaf546fd-1262-41e3-802f-377d2c84b885 None None] Validating token access rules against request {{(pid=81767) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 15:47:40.239188 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application [None req-a7aa3be2-03bb-4a6d-8131-ddc807e18ce3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 22 15:47:40.239188 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 15:47:40.239188 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 15:47:40.239188 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 15:47:40.239188 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 15:47:40.239188 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 15:47:40.239188 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 15:47:40.239188 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 15:47:40.239188 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 15:47:40.239188 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 15:47:40.239188 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 15:47:40.239188 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 15:47:40.239188 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 22 15:47:40.239188 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 22 15:47:40.239188 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 22 15:47:40.239188 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 22 15:47:40.240387 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 15:47:40.240387 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 15:47:40.240387 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 15:47:40.240387 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 15:47:40.240387 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 15:47:40.240387 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 15:47:40.240387 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 15:47:40.240387 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application with Lock( Aug 22 15:47:40.240387 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 15:47:40.240387 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self._enter() Aug 22 15:47:40.240387 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 15:47:40.240387 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 15:47:40.240387 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 15:47:40.240387 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self.creator() Aug 22 15:47:40.240387 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 15:47:40.240387 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application created_value = creator( Aug 22 15:47:40.240387 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 22 15:47:40.241265 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 22 15:47:40.241265 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 22 15:47:40.241265 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application Aug 22 15:47:40.241265 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 341/676] 10.222.0.15 () {56 vars in 1108 bytes} [Mon Aug 22 15:47:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 15:47:40.241495 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7aa3be2-03bb-4a6d-8131-ddc807e18ce3 None admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 15:47:40.263504 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=100 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 15:47:40.270589 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aef93613-87ba-424a-91ec-d0168444720d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81767) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 15:47:40.271203 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aef93613-87ba-424a-91ec-d0168444720d None admin] REQUEST_METHOD: `GET` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:47:40.271518 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aef93613-87ba-424a-91ec-d0168444720d None admin] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:47:40.271791 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aef93613-87ba-424a-91ec-d0168444720d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:47:40.272520 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aef93613-87ba-424a-91ec-d0168444720d None admin] RBAC: Adding query filter params (role.id=a5fbba1dc615490a96cde27e3a102eb6, scope.project.id=c4ee0413f9ef4d4fb1f95a660babe090, user.id=79706d480f864998b9f8594be407ceab) {{(pid=81767) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 15:47:40.272873 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aef93613-87ba-424a-91ec-d0168444720d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 15:47:40.274332 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aef93613-87ba-424a-91ec-d0168444720d None admin] RBAC: Authorization granted {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 15:47:40.281298 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 336/677] 10.222.0.15 () {56 vars in 1374 bytes} [Mon Aug 22 15:47:40 2022] GET /identity/v3/role_assignments?user.id=79706d480f864998b9f8594be407ceab&scope.project.id=c4ee0413f9ef4d4fb1f95a660babe090&role.id=a5fbba1dc615490a96cde27e3a102eb6 => generated 608 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 15:47:40.291692 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aef93613-87ba-424a-91ec-d0168444720d None admin] Authenticating user token {{(pid=81767) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 15:47:40.378072 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7aa3be2-03bb-4a6d-8131-ddc807e18ce3 None admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 15:47:40.423321 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aef93613-87ba-424a-91ec-d0168444720d None admin] Validating token access rules against request {{(pid=81767) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 15:47:40.425560 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06d61e83-b262-4915-96c7-3b0e077d789b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 15:47:40.426249 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06d61e83-b262-4915-96c7-3b0e077d789b None admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:47:40.426456 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06d61e83-b262-4915-96c7-3b0e077d789b None admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:47:40.426724 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06d61e83-b262-4915-96c7-3b0e077d789b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:47:40.427653 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06d61e83-b262-4915-96c7-3b0e077d789b None admin] RBAC: Adding query filter params (role.id=a5fbba1dc615490a96cde27e3a102eb6, scope.project.id=c4ee0413f9ef4d4fb1f95a660babe090, user.id=91f565473c824a15930e2b12863c1ed3) {{(pid=81768) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 15:47:40.427962 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06d61e83-b262-4915-96c7-3b0e077d789b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 15:47:40.429436 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06d61e83-b262-4915-96c7-3b0e077d789b None admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 15:47:40.436944 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 342/678] 10.222.0.15 () {56 vars in 1374 bytes} [Mon Aug 22 15:47:40 2022] GET /identity/v3/role_assignments?user.id=91f565473c824a15930e2b12863c1ed3&scope.project.id=c4ee0413f9ef4d4fb1f95a660babe090&role.id=a5fbba1dc615490a96cde27e3a102eb6 => generated 608 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 15:47:40.438184 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-154500f8-6b6f-489d-89a6-5f73738b5195 None None] REQUEST_METHOD: `POST` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:47:40.438525 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-154500f8-6b6f-489d-89a6-5f73738b5195 None None] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:47:40.438732 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-154500f8-6b6f-489d-89a6-5f73738b5195 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:47:40.454088 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG dbcounter [-] [81767] Writing DB stats keystone:SELECT=100 {{(pid=81767) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 15:47:40.458808 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.auth.core [None req-154500f8-6b6f-489d-89a6-5f73738b5195 None None] MFA Rules not processed for user `6b2199ae0fb0480aa917ada391f5ec54`. Rule list: `[]` (Enabled: `True`). {{(pid=81768) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 15:47:40.460083 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-154500f8-6b6f-489d-89a6-5f73738b5195 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 15:47:40.478025 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44a2b529-390a-4c20-aa79-2d31bf6fb410 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81767) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 15:47:40.479032 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a2b529-390a-4c20-aa79-2d31bf6fb410 None admin] REQUEST_METHOD: `GET` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:47:40.492977 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a2b529-390a-4c20-aa79-2d31bf6fb410 None admin] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:47:40.492977 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a2b529-390a-4c20-aa79-2d31bf6fb410 None admin] PATH_INFO: `/v3/domains` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:47:40.492977 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44a2b529-390a-4c20-aa79-2d31bf6fb410 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81767) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 15:47:40.492977 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44a2b529-390a-4c20-aa79-2d31bf6fb410 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 15:47:40.492977 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44a2b529-390a-4c20-aa79-2d31bf6fb410 None admin] RBAC: Authorization granted {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 15:47:40.501978 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 337/679] 10.222.0.15 () {56 vars in 1109 bytes} [Mon Aug 22 15:47:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 15:47:40.503920 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44a2b529-390a-4c20-aa79-2d31bf6fb410 None admin] Authenticating user token {{(pid=81767) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 15:47:40.558610 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 343/680] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Aug 22 15:47:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 22 15:47:40.559237 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-154500f8-6b6f-489d-89a6-5f73738b5195 None None] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 15:47:40.576930 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=100 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 15:47:40.664942 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44a2b529-390a-4c20-aa79-2d31bf6fb410 None admin] Validating token access rules against request {{(pid=81767) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 15:47:40.708309 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-154500f8-6b6f-489d-89a6-5f73738b5195 None None] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 15:47:40.711464 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4cc15f5-bef0-44a5-8235-ed4dfe673a3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81767) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 15:47:40.712083 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4cc15f5-bef0-44a5-8235-ed4dfe673a3f None admin] REQUEST_METHOD: `GET` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:47:40.712360 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4cc15f5-bef0-44a5-8235-ed4dfe673a3f None admin] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:47:40.712630 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4cc15f5-bef0-44a5-8235-ed4dfe673a3f None admin] PATH_INFO: `/v3/domains` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:47:40.713315 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4cc15f5-bef0-44a5-8235-ed4dfe673a3f None admin] RBAC: Adding query filter params (name=Default) {{(pid=81767) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 15:47:40.713620 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4cc15f5-bef0-44a5-8235-ed4dfe673a3f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 15:47:40.715001 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4cc15f5-bef0-44a5-8235-ed4dfe673a3f None admin] RBAC: Authorization granted {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 15:47:40.730871 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 338/681] 10.222.0.15 () {56 vars in 1109 bytes} [Mon Aug 22 15:47:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 15:47:40.754043 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4cc15f5-bef0-44a5-8235-ed4dfe673a3f None admin] Authenticating user token {{(pid=81767) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 15:47:40.762177 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b7c7f0a-7629-42e9-9cb8-185a5b1e0ebb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 15:47:40.762827 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b7c7f0a-7629-42e9-9cb8-185a5b1e0ebb None admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:47:40.763104 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b7c7f0a-7629-42e9-9cb8-185a5b1e0ebb None admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:47:40.763421 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b7c7f0a-7629-42e9-9cb8-185a5b1e0ebb None admin] PATH_INFO: `/v3/users/swift` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:47:40.773256 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b7c7f0a-7629-42e9-9cb8-185a5b1e0ebb None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 15:47:40.774825 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b7c7f0a-7629-42e9-9cb8-185a5b1e0ebb None admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 15:47:40.788502 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application [None req-6b7c7f0a-7629-42e9-9cb8-185a5b1e0ebb None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 22 15:47:40.788502 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 15:47:40.788502 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 15:47:40.788502 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 15:47:40.788502 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 15:47:40.788502 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 15:47:40.788502 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 15:47:40.788502 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 15:47:40.788502 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 15:47:40.788502 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 15:47:40.788502 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 15:47:40.788502 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 15:47:40.788502 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 22 15:47:40.788502 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 22 15:47:40.788502 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 22 15:47:40.788502 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 22 15:47:40.789546 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 15:47:40.789546 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 15:47:40.789546 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 22 15:47:40.789546 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 22 15:47:40.789546 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 22 15:47:40.789546 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 22 15:47:40.789546 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 15:47:40.789546 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 15:47:40.789546 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 15:47:40.789546 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 15:47:40.789546 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 15:47:40.789546 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application with Lock( Aug 22 15:47:40.789546 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 15:47:40.789546 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self._enter() Aug 22 15:47:40.789546 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 15:47:40.789546 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 15:47:40.789546 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 15:47:40.789546 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self.creator() Aug 22 15:47:40.790471 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 15:47:40.790471 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application created_value = creator( Aug 22 15:47:40.790471 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 22 15:47:40.790471 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 22 15:47:40.790471 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 22 15:47:40.790471 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 22 15:47:40.790471 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 22 15:47:40.790471 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 22 15:47:40.790471 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 22 15:47:40.790471 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application Aug 22 15:47:40.790471 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 344/682] 10.222.0.15 () {56 vars in 1096 bytes} [Mon Aug 22 15:47:40 2022] GET /identity/v3/users/swift => generated 83 bytes in 231 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 15:47:40.795511 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b7c7f0a-7629-42e9-9cb8-185a5b1e0ebb None admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 15:47:40.811187 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG dbcounter [-] [81767] Writing DB stats keystone:SELECT=100 {{(pid=81767) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 15:47:40.877766 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4cc15f5-bef0-44a5-8235-ed4dfe673a3f None admin] Validating token access rules against request {{(pid=81767) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 15:47:40.879179 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=100 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 15:47:40.918116 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b7c7f0a-7629-42e9-9cb8-185a5b1e0ebb None admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 15:47:40.919593 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d500f94-450f-4356-9f10-41af33df00d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81767) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 15:47:40.920305 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d500f94-450f-4356-9f10-41af33df00d5 None admin] REQUEST_METHOD: `GET` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:47:40.920664 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d500f94-450f-4356-9f10-41af33df00d5 None admin] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:47:40.920937 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d500f94-450f-4356-9f10-41af33df00d5 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:47:40.929258 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d500f94-450f-4356-9f10-41af33df00d5 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 15:47:40.930718 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d500f94-450f-4356-9f10-41af33df00d5 None admin] RBAC: Authorization granted {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 15:47:40.943406 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application [None req-6d500f94-450f-4356-9f10-41af33df00d5 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 22 15:47:40.943406 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 15:47:40.943406 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 15:47:40.943406 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 15:47:40.943406 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 15:47:40.943406 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 15:47:40.943406 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 15:47:40.943406 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 15:47:40.943406 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 15:47:40.943406 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 15:47:40.943406 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 15:47:40.943406 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 15:47:40.943406 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 22 15:47:40.943406 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 22 15:47:40.943406 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 22 15:47:40.944417 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 22 15:47:40.944417 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 15:47:40.944417 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 15:47:40.944417 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 15:47:40.944417 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 15:47:40.944417 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 15:47:40.944417 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 15:47:40.944417 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 15:47:40.944417 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application with Lock( Aug 22 15:47:40.944417 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 15:47:40.944417 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return self._enter() Aug 22 15:47:40.944417 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 15:47:40.944417 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 15:47:40.944417 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 15:47:40.944417 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return self.creator() Aug 22 15:47:40.944417 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 15:47:40.944417 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application created_value = creator( Aug 22 15:47:40.945494 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 22 15:47:40.945494 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 22 15:47:40.945494 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 22 15:47:40.945494 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 22 15:47:40.945494 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 22 15:47:40.945494 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 22 15:47:40.945494 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 22 15:47:40.945494 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application Aug 22 15:47:40.945494 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 339/683] 10.222.0.15 () {56 vars in 1111 bytes} [Mon Aug 22 15:47:40 2022] GET /identity/v3/projects/service => generated 88 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 15:47:40.946102 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d500f94-450f-4356-9f10-41af33df00d5 None admin] Authenticating user token {{(pid=81767) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 15:47:40.968878 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceb7169a-0aaa-439b-93c3-5df1f4e1e9e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 15:47:40.969413 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb7169a-0aaa-439b-93c3-5df1f4e1e9e9 None admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:47:40.969741 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb7169a-0aaa-439b-93c3-5df1f4e1e9e9 None admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:47:40.970018 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb7169a-0aaa-439b-93c3-5df1f4e1e9e9 None admin] PATH_INFO: `/v3/users` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:47:40.971416 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceb7169a-0aaa-439b-93c3-5df1f4e1e9e9 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81768) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 15:47:40.971725 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceb7169a-0aaa-439b-93c3-5df1f4e1e9e9 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 15:47:40.973646 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceb7169a-0aaa-439b-93c3-5df1f4e1e9e9 None admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 15:47:41.003792 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 345/684] 10.222.0.15 () {56 vars in 1135 bytes} [Mon Aug 22 15:47:40 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 22 15:47:41.004851 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceb7169a-0aaa-439b-93c3-5df1f4e1e9e9 None admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 15:47:41.065636 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG dbcounter [-] [81767] Writing DB stats keystone:SELECT=100 {{(pid=81767) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 15:47:41.070024 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d500f94-450f-4356-9f10-41af33df00d5 None admin] Validating token access rules against request {{(pid=81767) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 15:47:41.110845 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2afd402-4713-49f4-8461-05754cb62449 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81767) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 15:47:41.111723 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2afd402-4713-49f4-8461-05754cb62449 None admin] REQUEST_METHOD: `GET` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:47:41.112105 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2afd402-4713-49f4-8461-05754cb62449 None admin] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:47:41.112496 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2afd402-4713-49f4-8461-05754cb62449 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:47:41.128407 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2afd402-4713-49f4-8461-05754cb62449 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 22 15:47:41.135074 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application [None req-e2afd402-4713-49f4-8461-05754cb62449 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 22 15:47:41.135074 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 15:47:41.135074 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 15:47:41.135074 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 15:47:41.135074 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 15:47:41.135074 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 15:47:41.135074 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 15:47:41.135074 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 15:47:41.135074 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 15:47:41.135074 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 15:47:41.135074 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 15:47:41.135074 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 15:47:41.135074 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 22 15:47:41.135074 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 22 15:47:41.135074 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 22 15:47:41.135074 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 22 15:47:41.137443 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 22 15:47:41.137443 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 22 15:47:41.137443 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 22 15:47:41.137443 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 22 15:47:41.137443 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 15:47:41.137443 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 15:47:41.137443 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 15:47:41.137443 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 15:47:41.137443 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 15:47:41.137443 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 15:47:41.137443 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 15:47:41.137443 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application with Lock( Aug 22 15:47:41.137443 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 15:47:41.137443 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return self._enter() Aug 22 15:47:41.137443 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 15:47:41.137443 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 15:47:41.138492 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 15:47:41.138492 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return self.creator() Aug 22 15:47:41.138492 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 15:47:41.138492 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application created_value = creator( Aug 22 15:47:41.138492 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 22 15:47:41.138492 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 22 15:47:41.138492 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 22 15:47:41.138492 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 22 15:47:41.138492 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 22 15:47:41.138492 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 22 15:47:41.138492 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 22 15:47:41.138492 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application Aug 22 15:47:41.142680 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 340/685] 10.222.0.15 () {56 vars in 1096 bytes} [Mon Aug 22 15:47:40 2022] GET /identity/v3/roles/admin => generated 83 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 15:47:41.144002 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2afd402-4713-49f4-8461-05754cb62449 None admin] Authenticating user token {{(pid=81767) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 15:47:41.152498 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceb7169a-0aaa-439b-93c3-5df1f4e1e9e9 None admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 15:47:41.191707 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=100 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 15:47:41.202667 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-236c533b-977a-43ae-85c9-af1412fb2e2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 15:47:41.208136 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-236c533b-977a-43ae-85c9-af1412fb2e2a None admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:47:41.208136 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-236c533b-977a-43ae-85c9-af1412fb2e2a None admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:47:41.208136 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-236c533b-977a-43ae-85c9-af1412fb2e2a None admin] PATH_INFO: `/v3/projects` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:47:41.208136 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-236c533b-977a-43ae-85c9-af1412fb2e2a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81768) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 15:47:41.208136 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-236c533b-977a-43ae-85c9-af1412fb2e2a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 15:47:41.208788 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-236c533b-977a-43ae-85c9-af1412fb2e2a None admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 15:47:41.225041 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 346/686] 10.222.0.15 () {56 vars in 1148 bytes} [Mon Aug 22 15:47:41 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 15:47:41.226709 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-236c533b-977a-43ae-85c9-af1412fb2e2a None admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 15:47:41.280020 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2afd402-4713-49f4-8461-05754cb62449 None admin] Validating token access rules against request {{(pid=81767) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 15:47:41.325459 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c0a5184-9a03-4ac0-93ce-40e59031f0f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81767) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 15:47:41.326135 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c0a5184-9a03-4ac0-93ce-40e59031f0f4 None admin] REQUEST_METHOD: `GET` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:47:41.326512 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c0a5184-9a03-4ac0-93ce-40e59031f0f4 None admin] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:47:41.326885 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c0a5184-9a03-4ac0-93ce-40e59031f0f4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:47:41.359584 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-236c533b-977a-43ae-85c9-af1412fb2e2a None admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 15:47:41.364183 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG dbcounter [-] [81767] Writing DB stats keystone:SELECT=100 {{(pid=81767) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 15:47:41.398680 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3be7b1d-159e-47de-a7e5-93cfb9fb84d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 15:47:41.399668 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3be7b1d-159e-47de-a7e5-93cfb9fb84d4 None admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:47:41.400149 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3be7b1d-159e-47de-a7e5-93cfb9fb84d4 None admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:47:41.400610 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3be7b1d-159e-47de-a7e5-93cfb9fb84d4 None admin] PATH_INFO: `/v3/roles` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:47:41.401969 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3be7b1d-159e-47de-a7e5-93cfb9fb84d4 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81768) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 15:47:41.402484 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3be7b1d-159e-47de-a7e5-93cfb9fb84d4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 15:47:41.404075 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3be7b1d-159e-47de-a7e5-93cfb9fb84d4 None admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 15:47:41.408903 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c0a5184-9a03-4ac0-93ce-40e59031f0f4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 15:47:41.411417 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c0a5184-9a03-4ac0-93ce-40e59031f0f4 None admin] RBAC: Authorization granted {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 15:47:41.420408 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 347/687] 10.222.0.15 () {56 vars in 1099 bytes} [Mon Aug 22 15:47:41 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 15:47:41.421749 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3be7b1d-159e-47de-a7e5-93cfb9fb84d4 None admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 15:47:41.486633 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=100 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 15:47:41.543313 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3be7b1d-159e-47de-a7e5-93cfb9fb84d4 None admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 15:47:41.544128 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 341/688] 10.222.0.15 () {58 vars in 1282 bytes} [Mon Aug 22 15:47:41 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 401 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 22 15:47:41.546481 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c0a5184-9a03-4ac0-93ce-40e59031f0f4 None admin] Authenticating user token {{(pid=81767) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 15:47:41.582486 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bccbb79-2e1d-4e11-9227-891efb4b9be7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 15:47:41.583100 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bccbb79-2e1d-4e11-9227-891efb4b9be7 None admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:47:41.583410 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bccbb79-2e1d-4e11-9227-891efb4b9be7 None admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:47:41.583681 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bccbb79-2e1d-4e11-9227-891efb4b9be7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:47:41.584561 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bccbb79-2e1d-4e11-9227-891efb4b9be7 None admin] RBAC: Adding query filter params (role.id=a5fbba1dc615490a96cde27e3a102eb6, scope.project.id=c4ee0413f9ef4d4fb1f95a660babe090, user.id=234687adad594f1aa45775a565ffbdc7) {{(pid=81768) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 15:47:41.584872 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bccbb79-2e1d-4e11-9227-891efb4b9be7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 15:47:41.586372 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bccbb79-2e1d-4e11-9227-891efb4b9be7 None admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 15:47:41.594081 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 348/689] 10.222.0.15 () {56 vars in 1374 bytes} [Mon Aug 22 15:47:41 2022] GET /identity/v3/role_assignments?user.id=234687adad594f1aa45775a565ffbdc7&scope.project.id=c4ee0413f9ef4d4fb1f95a660babe090&role.id=a5fbba1dc615490a96cde27e3a102eb6 => generated 608 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 15:47:41.594995 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bccbb79-2e1d-4e11-9227-891efb4b9be7 None admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 15:47:41.635727 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG dbcounter [-] [81767] Writing DB stats keystone:SELECT=100 {{(pid=81767) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 15:47:41.733847 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c0a5184-9a03-4ac0-93ce-40e59031f0f4 None admin] Validating token access rules against request {{(pid=81767) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 15:47:41.788140 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeb40065-b79c-4410-b7d6-0a2301f55ab1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81767) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 15:47:41.789023 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeb40065-b79c-4410-b7d6-0a2301f55ab1 None admin] REQUEST_METHOD: `GET` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:47:41.789490 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeb40065-b79c-4410-b7d6-0a2301f55ab1 None admin] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:47:41.789930 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeb40065-b79c-4410-b7d6-0a2301f55ab1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:47:41.790212 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bccbb79-2e1d-4e11-9227-891efb4b9be7 None admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 15:47:41.802575 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeb40065-b79c-4410-b7d6-0a2301f55ab1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 15:47:41.804438 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeb40065-b79c-4410-b7d6-0a2301f55ab1 None admin] RBAC: Authorization granted {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 15:47:41.806864 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=100 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 15:47:41.820977 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application [None req-aeb40065-b79c-4410-b7d6-0a2301f55ab1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 22 15:47:41.820977 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 15:47:41.820977 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 15:47:41.820977 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 15:47:41.820977 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 15:47:41.820977 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 15:47:41.820977 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 15:47:41.820977 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 15:47:41.820977 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 15:47:41.820977 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 15:47:41.820977 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 15:47:41.820977 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 15:47:41.820977 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 22 15:47:41.820977 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 22 15:47:41.820977 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 22 15:47:41.820977 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 22 15:47:41.825477 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 15:47:41.825477 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 15:47:41.825477 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 15:47:41.825477 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 15:47:41.825477 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 15:47:41.825477 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 15:47:41.825477 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 15:47:41.825477 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application with Lock( Aug 22 15:47:41.825477 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 15:47:41.825477 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return self._enter() Aug 22 15:47:41.825477 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 15:47:41.825477 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 15:47:41.825477 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 15:47:41.825477 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return self.creator() Aug 22 15:47:41.825477 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 15:47:41.825477 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application created_value = creator( Aug 22 15:47:41.825477 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 22 15:47:41.826650 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 22 15:47:41.826650 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 22 15:47:41.826650 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application Aug 22 15:47:41.826650 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 342/690] 10.222.0.15 () {56 vars in 1108 bytes} [Mon Aug 22 15:47:41 2022] GET /identity/v3/domains/Default => generated 87 bytes in 276 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 15:47:41.831586 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeb40065-b79c-4410-b7d6-0a2301f55ab1 None admin] Authenticating user token {{(pid=81767) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 15:47:41.858741 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d94ec831-2d1d-4659-a0a8-8e0c37e6e7cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 15:47:41.859676 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d94ec831-2d1d-4659-a0a8-8e0c37e6e7cd None admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:47:41.860144 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d94ec831-2d1d-4659-a0a8-8e0c37e6e7cd None admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:47:41.860623 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d94ec831-2d1d-4659-a0a8-8e0c37e6e7cd None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:47:41.873986 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d94ec831-2d1d-4659-a0a8-8e0c37e6e7cd None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 15:47:41.875832 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d94ec831-2d1d-4659-a0a8-8e0c37e6e7cd None admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 15:47:41.892030 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application [None req-d94ec831-2d1d-4659-a0a8-8e0c37e6e7cd None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 22 15:47:41.892030 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 15:47:41.892030 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 15:47:41.892030 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 15:47:41.892030 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 15:47:41.892030 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 15:47:41.892030 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 15:47:41.892030 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 15:47:41.892030 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 15:47:41.892030 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 15:47:41.892030 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 15:47:41.892030 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 15:47:41.892030 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 22 15:47:41.892030 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 22 15:47:41.892030 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 22 15:47:41.892030 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 22 15:47:41.893002 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 15:47:41.893002 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 15:47:41.893002 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 15:47:41.893002 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 15:47:41.893002 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 15:47:41.893002 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 15:47:41.893002 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 15:47:41.893002 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application with Lock( Aug 22 15:47:41.893002 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 15:47:41.893002 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self._enter() Aug 22 15:47:41.893002 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 15:47:41.893002 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 15:47:41.893002 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 15:47:41.893002 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self.creator() Aug 22 15:47:41.893002 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 15:47:41.893002 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application created_value = creator( Aug 22 15:47:41.893002 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 22 15:47:41.893926 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 22 15:47:41.893926 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 22 15:47:41.893926 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application Aug 22 15:47:41.896080 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 349/691] 10.222.0.15 () {56 vars in 1108 bytes} [Mon Aug 22 15:47:41 2022] GET /identity/v3/domains/Default => generated 87 bytes in 301 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 15:47:41.903319 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d94ec831-2d1d-4659-a0a8-8e0c37e6e7cd None admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 15:47:41.957178 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeb40065-b79c-4410-b7d6-0a2301f55ab1 None admin] Validating token access rules against request {{(pid=81767) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 15:47:41.972884 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG dbcounter [-] [81767] Writing DB stats keystone:SELECT=100 {{(pid=81767) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 15:47:42.012358 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-709527d4-aa19-4ee4-9f1b-7cc4c160ee7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81767) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 15:47:42.013361 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-709527d4-aa19-4ee4-9f1b-7cc4c160ee7a None admin] REQUEST_METHOD: `GET` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:47:42.013945 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-709527d4-aa19-4ee4-9f1b-7cc4c160ee7a None admin] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:47:42.014416 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-709527d4-aa19-4ee4-9f1b-7cc4c160ee7a None admin] PATH_INFO: `/v3/domains` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:47:42.015265 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-709527d4-aa19-4ee4-9f1b-7cc4c160ee7a None admin] RBAC: Adding query filter params (name=Default) {{(pid=81767) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 15:47:42.016356 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-709527d4-aa19-4ee4-9f1b-7cc4c160ee7a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 15:47:42.017945 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-709527d4-aa19-4ee4-9f1b-7cc4c160ee7a None admin] RBAC: Authorization granted {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 15:47:42.039239 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 343/692] 10.222.0.15 () {56 vars in 1109 bytes} [Mon Aug 22 15:47:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 22 15:47:42.040971 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-709527d4-aa19-4ee4-9f1b-7cc4c160ee7a None admin] Authenticating user token {{(pid=81767) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 15:47:42.043341 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d94ec831-2d1d-4659-a0a8-8e0c37e6e7cd None admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 15:47:42.081120 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ae0e014-ba23-4de8-93df-797660af6477 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 15:47:42.082487 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ae0e014-ba23-4de8-93df-797660af6477 None admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:47:42.082487 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ae0e014-ba23-4de8-93df-797660af6477 None admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:47:42.082487 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ae0e014-ba23-4de8-93df-797660af6477 None admin] PATH_INFO: `/v3/domains` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:47:42.082986 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ae0e014-ba23-4de8-93df-797660af6477 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81768) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 15:47:42.083317 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ae0e014-ba23-4de8-93df-797660af6477 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 15:47:42.084733 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ae0e014-ba23-4de8-93df-797660af6477 None admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 15:47:42.102412 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 350/693] 10.222.0.15 () {56 vars in 1109 bytes} [Mon Aug 22 15:47:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 15:47:42.103777 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ae0e014-ba23-4de8-93df-797660af6477 None admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 15:47:42.133542 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=100 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 15:47:42.189497 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-709527d4-aa19-4ee4-9f1b-7cc4c160ee7a None admin] Validating token access rules against request {{(pid=81767) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 15:47:42.230088 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ae614c5-e7ea-4d75-a444-4c11200e7a1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81767) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 15:47:42.231081 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae614c5-e7ea-4d75-a444-4c11200e7a1b None admin] REQUEST_METHOD: `GET` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:47:42.231731 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae614c5-e7ea-4d75-a444-4c11200e7a1b None admin] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:47:42.232202 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae614c5-e7ea-4d75-a444-4c11200e7a1b None admin] PATH_INFO: `/v3/users/nova` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:47:42.240963 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ae614c5-e7ea-4d75-a444-4c11200e7a1b None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 15:47:42.242977 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ae614c5-e7ea-4d75-a444-4c11200e7a1b None admin] RBAC: Authorization granted {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 15:47:42.248452 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ae0e014-ba23-4de8-93df-797660af6477 None admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 15:47:42.256701 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application [None req-5ae614c5-e7ea-4d75-a444-4c11200e7a1b None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 22 15:47:42.256701 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 15:47:42.256701 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 15:47:42.256701 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 15:47:42.256701 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 15:47:42.256701 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 15:47:42.256701 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 15:47:42.256701 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 15:47:42.256701 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 15:47:42.256701 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 15:47:42.256701 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 15:47:42.256701 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 15:47:42.256701 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 22 15:47:42.256701 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 22 15:47:42.256701 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 22 15:47:42.256701 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 22 15:47:42.257664 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 15:47:42.257664 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 15:47:42.257664 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 22 15:47:42.257664 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 22 15:47:42.257664 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 22 15:47:42.257664 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 22 15:47:42.257664 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 15:47:42.257664 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 15:47:42.257664 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 15:47:42.257664 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 15:47:42.257664 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 15:47:42.257664 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application with Lock( Aug 22 15:47:42.257664 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 15:47:42.257664 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return self._enter() Aug 22 15:47:42.257664 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 15:47:42.257664 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 15:47:42.257664 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 15:47:42.257664 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return self.creator() Aug 22 15:47:42.258584 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 15:47:42.258584 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application created_value = creator( Aug 22 15:47:42.258584 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 22 15:47:42.258584 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 22 15:47:42.258584 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 22 15:47:42.258584 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 22 15:47:42.258584 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 22 15:47:42.258584 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 22 15:47:42.258584 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 22 15:47:42.258584 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application Aug 22 15:47:42.265995 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 344/694] 10.222.0.15 () {56 vars in 1093 bytes} [Mon Aug 22 15:47:42 2022] GET /identity/v3/users/nova => generated 82 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 15:47:42.267757 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ae614c5-e7ea-4d75-a444-4c11200e7a1b None admin] Authenticating user token {{(pid=81767) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 15:47:42.286450 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12e90894-d2df-40c7-aff4-c169d98b9dca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 15:47:42.287337 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12e90894-d2df-40c7-aff4-c169d98b9dca None admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:47:42.287451 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12e90894-d2df-40c7-aff4-c169d98b9dca None admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:47:42.287751 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12e90894-d2df-40c7-aff4-c169d98b9dca None admin] PATH_INFO: `/v3/projects/service` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:47:42.302538 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12e90894-d2df-40c7-aff4-c169d98b9dca None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 15:47:42.304074 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12e90894-d2df-40c7-aff4-c169d98b9dca None admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 15:47:42.305047 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG dbcounter [-] [81767] Writing DB stats keystone:SELECT=100 {{(pid=81767) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 15:47:42.331678 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application [None req-12e90894-d2df-40c7-aff4-c169d98b9dca None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 22 15:47:42.331678 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 15:47:42.331678 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 15:47:42.331678 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 15:47:42.331678 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 15:47:42.331678 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 15:47:42.331678 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 15:47:42.331678 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 15:47:42.331678 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 15:47:42.331678 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 15:47:42.331678 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 15:47:42.331678 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 15:47:42.331678 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 22 15:47:42.331678 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 22 15:47:42.331678 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 22 15:47:42.332623 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 22 15:47:42.332623 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 15:47:42.332623 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 15:47:42.332623 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 15:47:42.332623 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 15:47:42.332623 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 15:47:42.332623 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 15:47:42.332623 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 15:47:42.332623 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application with Lock( Aug 22 15:47:42.332623 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 15:47:42.332623 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self._enter() Aug 22 15:47:42.332623 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 15:47:42.332623 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 15:47:42.332623 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 15:47:42.332623 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self.creator() Aug 22 15:47:42.332623 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 15:47:42.332623 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application created_value = creator( Aug 22 15:47:42.333953 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 22 15:47:42.333953 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 22 15:47:42.333953 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 22 15:47:42.333953 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 22 15:47:42.333953 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 22 15:47:42.333953 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 22 15:47:42.333953 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 22 15:47:42.333953 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application Aug 22 15:47:42.333953 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 351/695] 10.222.0.15 () {56 vars in 1111 bytes} [Mon Aug 22 15:47:42 2022] GET /identity/v3/projects/service => generated 88 bytes in 230 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 15:47:42.341054 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12e90894-d2df-40c7-aff4-c169d98b9dca None admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 15:47:42.398773 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ae614c5-e7ea-4d75-a444-4c11200e7a1b None admin] Validating token access rules against request {{(pid=81767) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 15:47:42.439417 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7802f0be-fab2-49ee-89f1-fdd796dae131 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81767) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 15:47:42.440407 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7802f0be-fab2-49ee-89f1-fdd796dae131 None admin] REQUEST_METHOD: `GET` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:47:42.440932 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7802f0be-fab2-49ee-89f1-fdd796dae131 None admin] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:47:42.441448 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7802f0be-fab2-49ee-89f1-fdd796dae131 None admin] PATH_INFO: `/v3/users` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:47:42.442489 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7802f0be-fab2-49ee-89f1-fdd796dae131 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81767) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 15:47:42.443000 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7802f0be-fab2-49ee-89f1-fdd796dae131 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 15:47:42.444712 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7802f0be-fab2-49ee-89f1-fdd796dae131 None admin] RBAC: Authorization granted {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 15:47:42.471614 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 345/696] 10.222.0.15 () {56 vars in 1133 bytes} [Mon Aug 22 15:47:42 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 15:47:42.473804 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7802f0be-fab2-49ee-89f1-fdd796dae131 None admin] Authenticating user token {{(pid=81767) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 15:47:42.572166 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG dbcounter [-] [81767] Writing DB stats keystone:SELECT=100 {{(pid=81767) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 15:47:42.573758 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=100 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 15:47:42.587772 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7802f0be-fab2-49ee-89f1-fdd796dae131 None admin] Validating token access rules against request {{(pid=81767) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 15:47:42.590863 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12e90894-d2df-40c7-aff4-c169d98b9dca None admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 15:47:42.624745 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c2d117b-cb81-47bb-837d-2d08ede06e8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 15:47:42.625399 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c2d117b-cb81-47bb-837d-2d08ede06e8b None admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:47:42.625684 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c2d117b-cb81-47bb-837d-2d08ede06e8b None admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:47:42.625948 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c2d117b-cb81-47bb-837d-2d08ede06e8b None admin] PATH_INFO: `/v3/projects` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:47:42.626199 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3faf0dd-c168-4ecf-a840-a3f008a2b172 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81767) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 15:47:42.626635 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c2d117b-cb81-47bb-837d-2d08ede06e8b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81768) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 15:47:42.626979 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c2d117b-cb81-47bb-837d-2d08ede06e8b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 15:47:42.627400 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3faf0dd-c168-4ecf-a840-a3f008a2b172 None admin] REQUEST_METHOD: `GET` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:47:42.627873 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3faf0dd-c168-4ecf-a840-a3f008a2b172 None admin] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:47:42.628317 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3faf0dd-c168-4ecf-a840-a3f008a2b172 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:47:42.628420 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c2d117b-cb81-47bb-837d-2d08ede06e8b None admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 15:47:42.646965 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 352/697] 10.222.0.15 () {56 vars in 1148 bytes} [Mon Aug 22 15:47:42 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 307 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 15:47:42.648324 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e443325e-4fc9-4313-a1e6-43c8dc87b045 None None] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:47:42.648613 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e443325e-4fc9-4313-a1e6-43c8dc87b045 None None] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:47:42.648917 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e443325e-4fc9-4313-a1e6-43c8dc87b045 None None] PATH_INFO: `/` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:47:42.650787 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 353/698] 10.222.0.15 () {54 vars in 927 bytes} [Mon Aug 22 15:47:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 15:47:42.656457 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1c1f991-fc32-4584-9341-109e0867ca8e None None] REQUEST_METHOD: `POST` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:47:42.656669 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1c1f991-fc32-4584-9341-109e0867ca8e None None] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:47:42.656936 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1c1f991-fc32-4584-9341-109e0867ca8e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:47:42.677856 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.auth.core [None req-a1c1f991-fc32-4584-9341-109e0867ca8e None None] MFA Rules not processed for user `6b2199ae0fb0480aa917ada391f5ec54`. Rule list: `[]` (Enabled: `True`). {{(pid=81768) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 15:47:42.680301 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-a1c1f991-fc32-4584-9341-109e0867ca8e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 15:47:42.705951 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3faf0dd-c168-4ecf-a840-a3f008a2b172 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 15:47:42.707751 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3faf0dd-c168-4ecf-a840-a3f008a2b172 None admin] RBAC: Authorization granted {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 15:47:42.757566 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 354/699] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Aug 22 15:47:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 22 15:47:42.758580 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1c1f991-fc32-4584-9341-109e0867ca8e None None] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 15:47:42.801169 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG dbcounter [-] [81767] Writing DB stats keystone:SELECT=100 {{(pid=81767) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 15:47:42.815115 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 346/700] 10.222.0.15 () {58 vars in 1282 bytes} [Mon Aug 22 15:47:42 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 342 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 22 15:47:42.821207 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3faf0dd-c168-4ecf-a840-a3f008a2b172 None admin] Authenticating user token {{(pid=81767) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 15:47:42.848201 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=100 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 15:47:42.875482 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1c1f991-fc32-4584-9341-109e0867ca8e None None] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 15:47:42.917831 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-918818fe-5050-4981-8bba-c51c8ba0dd64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 15:47:42.918728 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-918818fe-5050-4981-8bba-c51c8ba0dd64 None admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:47:42.919154 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-918818fe-5050-4981-8bba-c51c8ba0dd64 None admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:47:42.919657 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-918818fe-5050-4981-8bba-c51c8ba0dd64 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:47:42.920588 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-918818fe-5050-4981-8bba-c51c8ba0dd64 None admin] RBAC: Adding query filter params (role.id=a5fbba1dc615490a96cde27e3a102eb6, scope.project.id=c4ee0413f9ef4d4fb1f95a660babe090, user.id=6a09d128dee9438283633e51f4834643) {{(pid=81768) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 15:47:42.921258 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-918818fe-5050-4981-8bba-c51c8ba0dd64 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 15:47:42.923068 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-918818fe-5050-4981-8bba-c51c8ba0dd64 None admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 15:47:42.934097 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 355/701] 10.222.0.15 () {56 vars in 1374 bytes} [Mon Aug 22 15:47:42 2022] GET /identity/v3/role_assignments?user.id=6a09d128dee9438283633e51f4834643&scope.project.id=c4ee0413f9ef4d4fb1f95a660babe090&role.id=a5fbba1dc615490a96cde27e3a102eb6 => generated 608 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 15:47:42.938617 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3faf0dd-c168-4ecf-a840-a3f008a2b172 None admin] Validating token access rules against request {{(pid=81767) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 15:47:42.981010 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7dd7e9d-be62-40ec-b6a2-00a109bcc775 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81767) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 15:47:42.981656 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7dd7e9d-be62-40ec-b6a2-00a109bcc775 None admin] REQUEST_METHOD: `GET` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:47:42.981935 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7dd7e9d-be62-40ec-b6a2-00a109bcc775 None admin] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:47:42.982205 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7dd7e9d-be62-40ec-b6a2-00a109bcc775 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:47:42.996713 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7dd7e9d-be62-40ec-b6a2-00a109bcc775 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 15:47:42.998239 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7dd7e9d-be62-40ec-b6a2-00a109bcc775 None admin] RBAC: Authorization granted {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 15:47:43.014203 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1358fd4c-6062-4c2c-b7df-27a3a4b13ce4 None None] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:47:43.014511 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1358fd4c-6062-4c2c-b7df-27a3a4b13ce4 None None] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:47:43.014754 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1358fd4c-6062-4c2c-b7df-27a3a4b13ce4 None None] PATH_INFO: `/` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:47:43.015953 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application [None req-e7dd7e9d-be62-40ec-b6a2-00a109bcc775 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 22 15:47:43.015953 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 15:47:43.015953 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 15:47:43.015953 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 15:47:43.015953 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 15:47:43.015953 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 15:47:43.015953 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 15:47:43.015953 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 15:47:43.015953 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 15:47:43.015953 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 15:47:43.015953 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 15:47:43.015953 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 15:47:43.015953 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 22 15:47:43.015953 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 22 15:47:43.015953 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 22 15:47:43.015953 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 22 15:47:43.016903 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 15:47:43.016903 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 15:47:43.016903 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 15:47:43.016903 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 15:47:43.016903 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 15:47:43.016903 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 15:47:43.016903 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 15:47:43.016903 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application with Lock( Aug 22 15:47:43.016903 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 15:47:43.016903 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return self._enter() Aug 22 15:47:43.016903 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 15:47:43.016903 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 15:47:43.016903 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 15:47:43.016903 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return self.creator() Aug 22 15:47:43.016903 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 15:47:43.016903 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application created_value = creator( Aug 22 15:47:43.016903 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 22 15:47:43.017844 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 22 15:47:43.017844 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 22 15:47:43.017844 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application Aug 22 15:47:43.018011 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 356/702] 10.222.0.15 () {54 vars in 927 bytes} [Mon Aug 22 15:47:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 15:47:43.020834 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 347/703] 10.222.0.15 () {56 vars in 1108 bytes} [Mon Aug 22 15:47:42 2022] GET /identity/v3/domains/Default => generated 87 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 22 15:47:43.023068 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a4d0d25-d0bc-4f3d-bee6-9ed8b49cc649 None None] REQUEST_METHOD: `POST` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:47:43.023390 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a4d0d25-d0bc-4f3d-bee6-9ed8b49cc649 None None] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:47:43.023676 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7dd7e9d-be62-40ec-b6a2-00a109bcc775 None admin] Authenticating user token {{(pid=81767) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 15:47:43.023773 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a4d0d25-d0bc-4f3d-bee6-9ed8b49cc649 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:47:43.043829 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.auth.core [None req-8a4d0d25-d0bc-4f3d-bee6-9ed8b49cc649 None None] MFA Rules not processed for user `6b2199ae0fb0480aa917ada391f5ec54`. Rule list: `[]` (Enabled: `True`). {{(pid=81768) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 15:47:43.045118 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-8a4d0d25-d0bc-4f3d-bee6-9ed8b49cc649 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 15:47:43.083610 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG dbcounter [-] [81767] Writing DB stats keystone:SELECT=100 {{(pid=81767) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 15:47:43.168001 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 357/704] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Aug 22 15:47:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 22 15:47:43.168783 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a4d0d25-d0bc-4f3d-bee6-9ed8b49cc649 None None] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 15:47:43.187111 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7dd7e9d-be62-40ec-b6a2-00a109bcc775 None admin] Validating token access rules against request {{(pid=81767) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 15:47:43.239830 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5906a2c4-7684-48cb-bf39-26748c69476e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81767) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 15:47:43.240954 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5906a2c4-7684-48cb-bf39-26748c69476e None admin] REQUEST_METHOD: `GET` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:47:43.241354 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5906a2c4-7684-48cb-bf39-26748c69476e None admin] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:47:43.241637 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5906a2c4-7684-48cb-bf39-26748c69476e None admin] PATH_INFO: `/v3/domains` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:47:43.242611 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5906a2c4-7684-48cb-bf39-26748c69476e None admin] RBAC: Adding query filter params (name=Default) {{(pid=81767) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 15:47:43.242978 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5906a2c4-7684-48cb-bf39-26748c69476e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 15:47:43.245591 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5906a2c4-7684-48cb-bf39-26748c69476e None admin] RBAC: Authorization granted {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 15:47:43.252171 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=100 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 15:47:43.264622 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 348/705] 10.222.0.15 () {56 vars in 1109 bytes} [Mon Aug 22 15:47:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 241 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 22 15:47:43.270374 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5906a2c4-7684-48cb-bf39-26748c69476e None admin] Authenticating user token {{(pid=81767) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 15:47:43.289226 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a4d0d25-d0bc-4f3d-bee6-9ed8b49cc649 None None] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 15:47:43.328018 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef3de29a-86c2-4ea9-b53b-19282ed3dcbf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 15:47:43.328666 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef3de29a-86c2-4ea9-b53b-19282ed3dcbf None admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:47:43.328937 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef3de29a-86c2-4ea9-b53b-19282ed3dcbf None admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:47:43.329202 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef3de29a-86c2-4ea9-b53b-19282ed3dcbf None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:47:43.329789 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef3de29a-86c2-4ea9-b53b-19282ed3dcbf None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 15:47:43.331166 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef3de29a-86c2-4ea9-b53b-19282ed3dcbf None admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 15:47:43.343152 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application [None req-ef3de29a-86c2-4ea9-b53b-19282ed3dcbf None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 22 15:47:43.343152 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 15:47:43.343152 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 22 15:47:43.343152 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 22 15:47:43.343152 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 22 15:47:43.343152 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 22 15:47:43.343152 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 22 15:47:43.343152 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 22 15:47:43.343152 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 22 15:47:43.343152 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application Aug 22 15:47:43.343152 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 22 15:47:43.343152 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application Aug 22 15:47:43.343152 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 15:47:43.343152 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 15:47:43.343152 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 15:47:43.343152 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 15:47:43.343152 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 15:47:43.344343 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 15:47:43.344343 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 15:47:43.344343 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 15:47:43.344343 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 15:47:43.344343 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 15:47:43.344343 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 15:47:43.344343 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 22 15:47:43.344343 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 22 15:47:43.344343 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 22 15:47:43.344343 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 22 15:47:43.344343 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 15:47:43.344343 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 15:47:43.344343 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 15:47:43.344343 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 15:47:43.344343 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 15:47:43.344343 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 15:47:43.345310 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 15:47:43.345310 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application with Lock( Aug 22 15:47:43.345310 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 15:47:43.345310 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self._enter() Aug 22 15:47:43.345310 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 15:47:43.345310 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 15:47:43.345310 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 15:47:43.345310 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self.creator() Aug 22 15:47:43.345310 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 15:47:43.345310 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application created_value = creator( Aug 22 15:47:43.345310 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 22 15:47:43.345310 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 22 15:47:43.345310 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 22 15:47:43.345310 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application Aug 22 15:47:43.345310 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 358/706] 10.222.0.15 () {56 vars in 1129 bytes} [Mon Aug 22 15:47:43 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 15:47:43.350774 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef3de29a-86c2-4ea9-b53b-19282ed3dcbf None admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 15:47:43.389628 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG dbcounter [-] [81767] Writing DB stats keystone:SELECT=100 {{(pid=81767) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 15:47:43.393129 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5906a2c4-7684-48cb-bf39-26748c69476e None admin] Validating token access rules against request {{(pid=81767) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 15:47:43.432980 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ba99232-e665-44be-a653-c3434fb8adf3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81767) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 15:47:43.433540 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ba99232-e665-44be-a653-c3434fb8adf3 None admin] REQUEST_METHOD: `GET` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:47:43.433819 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ba99232-e665-44be-a653-c3434fb8adf3 None admin] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:47:43.434126 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ba99232-e665-44be-a653-c3434fb8adf3 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:47:43.443009 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ba99232-e665-44be-a653-c3434fb8adf3 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 15:47:43.444539 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ba99232-e665-44be-a653-c3434fb8adf3 None admin] RBAC: Authorization granted {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 15:47:43.456882 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application [None req-5ba99232-e665-44be-a653-c3434fb8adf3 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 22 15:47:43.456882 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 15:47:43.456882 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 15:47:43.456882 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 15:47:43.456882 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 15:47:43.456882 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 15:47:43.456882 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 15:47:43.456882 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 15:47:43.456882 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 15:47:43.456882 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 15:47:43.456882 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 15:47:43.456882 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 15:47:43.456882 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 22 15:47:43.456882 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 22 15:47:43.456882 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 22 15:47:43.458865 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 22 15:47:43.458865 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 15:47:43.458865 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 15:47:43.458865 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 15:47:43.458865 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 15:47:43.458865 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 15:47:43.458865 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 15:47:43.458865 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 15:47:43.458865 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application with Lock( Aug 22 15:47:43.458865 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 15:47:43.458865 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return self._enter() Aug 22 15:47:43.458865 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 15:47:43.458865 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 15:47:43.458865 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 15:47:43.458865 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return self.creator() Aug 22 15:47:43.458865 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 15:47:43.458865 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application created_value = creator( Aug 22 15:47:43.460097 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 22 15:47:43.460097 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 22 15:47:43.460097 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 22 15:47:43.460097 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 22 15:47:43.460097 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 22 15:47:43.460097 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 22 15:47:43.460097 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 22 15:47:43.460097 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application Aug 22 15:47:43.460097 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 349/707] 10.222.0.15 () {56 vars in 1111 bytes} [Mon Aug 22 15:47:43 2022] GET /identity/v3/projects/service => generated 88 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 15:47:43.467295 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ba99232-e665-44be-a653-c3434fb8adf3 None admin] Authenticating user token {{(pid=81767) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 15:47:43.467450 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef3de29a-86c2-4ea9-b53b-19282ed3dcbf None admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 15:47:43.502428 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0938b74-32aa-469a-8f88-6be6a587748d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 15:47:43.503032 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0938b74-32aa-469a-8f88-6be6a587748d None admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:47:43.503331 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0938b74-32aa-469a-8f88-6be6a587748d None admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:47:43.503598 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0938b74-32aa-469a-8f88-6be6a587748d None admin] PATH_INFO: `/v3/services` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:47:43.504224 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0938b74-32aa-469a-8f88-6be6a587748d None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=81768) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 15:47:43.504517 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0938b74-32aa-469a-8f88-6be6a587748d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 15:47:43.505944 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0938b74-32aa-469a-8f88-6be6a587748d None admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 15:47:43.507262 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=100 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 15:47:43.511195 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 359/708] 10.222.0.15 () {56 vars in 1124 bytes} [Mon Aug 22 15:47:43 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 22 15:47:43.511938 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0938b74-32aa-469a-8f88-6be6a587748d None admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 15:47:43.603504 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ba99232-e665-44be-a653-c3434fb8adf3 None admin] Validating token access rules against request {{(pid=81767) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 15:47:43.648614 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57a75fbb-7192-416a-ab4a-f6f9364e9b29 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81767) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 15:47:43.649290 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57a75fbb-7192-416a-ab4a-f6f9364e9b29 None admin] REQUEST_METHOD: `GET` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:47:43.649562 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57a75fbb-7192-416a-ab4a-f6f9364e9b29 None admin] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:47:43.649927 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57a75fbb-7192-416a-ab4a-f6f9364e9b29 None admin] PATH_INFO: `/v3/projects` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:47:43.651169 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57a75fbb-7192-416a-ab4a-f6f9364e9b29 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81767) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 15:47:43.651519 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57a75fbb-7192-416a-ab4a-f6f9364e9b29 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 15:47:43.651958 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0938b74-32aa-469a-8f88-6be6a587748d None admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 15:47:43.652982 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57a75fbb-7192-416a-ab4a-f6f9364e9b29 None admin] RBAC: Authorization granted {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 15:47:43.675093 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 350/709] 10.222.0.15 () {56 vars in 1148 bytes} [Mon Aug 22 15:47:43 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 15:47:43.676100 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57a75fbb-7192-416a-ab4a-f6f9364e9b29 None admin] Authenticating user token {{(pid=81767) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 15:47:43.701221 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG dbcounter [-] [81767] Writing DB stats keystone:SELECT=100 {{(pid=81767) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 15:47:43.709859 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-535f8eb0-6782-48b4-80ca-1d461e972365 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 15:47:43.711345 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-535f8eb0-6782-48b4-80ca-1d461e972365 None admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:47:43.711902 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-535f8eb0-6782-48b4-80ca-1d461e972365 None admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:47:43.717460 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-535f8eb0-6782-48b4-80ca-1d461e972365 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:47:43.729779 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-535f8eb0-6782-48b4-80ca-1d461e972365 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 15:47:43.732442 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-535f8eb0-6782-48b4-80ca-1d461e972365 None admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 15:47:43.752256 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application [None req-535f8eb0-6782-48b4-80ca-1d461e972365 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 22 15:47:43.752256 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 15:47:43.752256 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 15:47:43.752256 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 15:47:43.752256 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 15:47:43.752256 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 15:47:43.752256 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 15:47:43.752256 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 15:47:43.752256 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 15:47:43.752256 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 15:47:43.752256 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 15:47:43.752256 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 15:47:43.752256 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 22 15:47:43.752256 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 22 15:47:43.752256 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 22 15:47:43.752256 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 22 15:47:43.753280 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 15:47:43.753280 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 15:47:43.753280 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 15:47:43.753280 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 15:47:43.753280 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 15:47:43.753280 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 15:47:43.753280 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 15:47:43.753280 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application with Lock( Aug 22 15:47:43.753280 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 15:47:43.753280 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self._enter() Aug 22 15:47:43.753280 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 15:47:43.753280 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 15:47:43.753280 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 15:47:43.753280 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self.creator() Aug 22 15:47:43.753280 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 15:47:43.753280 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application created_value = creator( Aug 22 15:47:43.753280 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 22 15:47:43.754495 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 22 15:47:43.754495 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 22 15:47:43.754495 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application Aug 22 15:47:43.758515 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 360/710] 10.222.0.15 () {56 vars in 1108 bytes} [Mon Aug 22 15:47:43 2022] GET /identity/v3/domains/Default => generated 87 bytes in 242 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 15:47:43.759495 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-535f8eb0-6782-48b4-80ca-1d461e972365 None admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 15:47:43.803778 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57a75fbb-7192-416a-ab4a-f6f9364e9b29 None admin] Validating token access rules against request {{(pid=81767) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 15:47:43.818080 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=100 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 15:47:43.843980 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1893f305-955e-4640-b016-6d15005f28cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81767) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 15:47:43.844571 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1893f305-955e-4640-b016-6d15005f28cb None admin] REQUEST_METHOD: `GET` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:47:43.844832 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1893f305-955e-4640-b016-6d15005f28cb None admin] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:47:43.845101 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1893f305-955e-4640-b016-6d15005f28cb None admin] PATH_INFO: `/v3/services` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:47:43.845793 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1893f305-955e-4640-b016-6d15005f28cb None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=81767) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 15:47:43.846088 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1893f305-955e-4640-b016-6d15005f28cb None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 15:47:43.847522 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1893f305-955e-4640-b016-6d15005f28cb None admin] RBAC: Authorization granted {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 15:47:43.859557 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 351/711] 10.222.0.15 () {56 vars in 1124 bytes} [Mon Aug 22 15:47:43 2022] GET /identity/v3/services?type=block-storage => generated 130 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 15:47:43.860643 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1893f305-955e-4640-b016-6d15005f28cb None admin] Authenticating user token {{(pid=81767) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 15:47:43.892578 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-535f8eb0-6782-48b4-80ca-1d461e972365 None admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 15:47:43.976511 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7d21700-07f1-4dfc-adfe-2fdeed8f4037 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 15:47:43.978696 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7d21700-07f1-4dfc-adfe-2fdeed8f4037 None admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:47:43.979860 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7d21700-07f1-4dfc-adfe-2fdeed8f4037 None admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:47:43.982441 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7d21700-07f1-4dfc-adfe-2fdeed8f4037 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:47:43.983526 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7d21700-07f1-4dfc-adfe-2fdeed8f4037 None admin] RBAC: Adding query filter params (role.id=9d9848cfd22e4b089db443c350e56670, scope.project.id=c4ee0413f9ef4d4fb1f95a660babe090, user.id=e4ef569e41fb46d0b721bb1c7663616f) {{(pid=81768) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 15:47:43.984712 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7d21700-07f1-4dfc-adfe-2fdeed8f4037 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 15:47:43.987449 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7d21700-07f1-4dfc-adfe-2fdeed8f4037 None admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 15:47:44.008201 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 361/712] 10.222.0.15 () {56 vars in 1374 bytes} [Mon Aug 22 15:47:43 2022] GET /identity/v3/role_assignments?user.id=e4ef569e41fb46d0b721bb1c7663616f&scope.project.id=c4ee0413f9ef4d4fb1f95a660babe090&role.id=9d9848cfd22e4b089db443c350e56670 => generated 259 bytes in 244 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 15:47:44.060071 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG dbcounter [-] [81767] Writing DB stats keystone:SELECT=100 {{(pid=81767) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 15:47:44.080409 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1893f305-955e-4640-b016-6d15005f28cb None admin] Validating token access rules against request {{(pid=81767) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 15:47:44.126151 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2063bd59-7cd2-411b-b387-245fea7f975c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81767) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 15:47:44.127553 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2063bd59-7cd2-411b-b387-245fea7f975c None admin] REQUEST_METHOD: `GET` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:47:44.128245 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2063bd59-7cd2-411b-b387-245fea7f975c None admin] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:47:44.128831 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2063bd59-7cd2-411b-b387-245fea7f975c None admin] PATH_INFO: `/v3/domains` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:47:44.129970 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2063bd59-7cd2-411b-b387-245fea7f975c None admin] RBAC: Adding query filter params (name=Default) {{(pid=81767) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 15:47:44.130580 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2063bd59-7cd2-411b-b387-245fea7f975c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 15:47:44.132842 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2063bd59-7cd2-411b-b387-245fea7f975c None admin] RBAC: Authorization granted {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 15:47:44.155105 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 352/713] 10.222.0.15 () {56 vars in 1109 bytes} [Mon Aug 22 15:47:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 295 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 15:47:44.161445 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7d21700-07f1-4dfc-adfe-2fdeed8f4037 None admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 15:47:44.286098 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7d21700-07f1-4dfc-adfe-2fdeed8f4037 None admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 15:47:44.311726 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=100 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 15:47:44.318335 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8485a021-be95-48a6-9aeb-03caadf4ccbd None None] REQUEST_METHOD: `GET` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:47:44.325604 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8485a021-be95-48a6-9aeb-03caadf4ccbd None None] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:47:44.325604 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8485a021-be95-48a6-9aeb-03caadf4ccbd None None] PATH_INFO: `/` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:47:44.325604 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 353/714] 10.222.0.15 () {54 vars in 927 bytes} [Mon Aug 22 15:47:44 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 22 15:47:44.329880 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c95a7ee-afae-4117-a7c9-c7d82f42fa0c None None] REQUEST_METHOD: `POST` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:47:44.330186 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c95a7ee-afae-4117-a7c9-c7d82f42fa0c None None] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:47:44.330449 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c95a7ee-afae-4117-a7c9-c7d82f42fa0c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:47:44.332713 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8577f61-c870-4941-8a7d-b70ed7ae1aa6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 15:47:44.333674 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8577f61-c870-4941-8a7d-b70ed7ae1aa6 None admin] REQUEST_METHOD: `POST` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:47:44.334168 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8577f61-c870-4941-8a7d-b70ed7ae1aa6 None admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:47:44.334707 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8577f61-c870-4941-8a7d-b70ed7ae1aa6 None admin] PATH_INFO: `/v3/users` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:47:44.335651 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8577f61-c870-4941-8a7d-b70ed7ae1aa6 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 15:47:44.337535 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8577f61-c870-4941-8a7d-b70ed7ae1aa6 None admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 15:47:44.365883 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.auth.core [None req-3c95a7ee-afae-4117-a7c9-c7d82f42fa0c None None] MFA Rules not processed for user `6b2199ae0fb0480aa917ada391f5ec54`. Rule list: `[]` (Enabled: `True`). {{(pid=81767) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 15:47:44.367584 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.fernet_utils [None req-3c95a7ee-afae-4117-a7c9-c7d82f42fa0c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81767) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 15:47:44.449418 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 354/715] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Aug 22 15:47:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 22 15:47:44.492437 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 362/716] 10.222.0.15 () {60 vars in 1132 bytes} [Mon Aug 22 15:47:44 2022] POST /identity/v3/users => generated 254 bytes in 331 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 22 15:47:44.785667 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c95a7ee-afae-4117-a7c9-c7d82f42fa0c None None] Authenticating user token {{(pid=81767) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 15:47:44.882793 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG dbcounter [-] [81767] Writing DB stats keystone:SELECT=100 {{(pid=81767) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 15:47:44.911083 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c95a7ee-afae-4117-a7c9-c7d82f42fa0c None None] Validating token access rules against request {{(pid=81767) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 15:47:44.954417 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b1c82f5-c823-45ce-a49f-5ce48193ffb4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81767) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 15:47:44.955067 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b1c82f5-c823-45ce-a49f-5ce48193ffb4 None admin] REQUEST_METHOD: `GET` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:47:44.955368 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b1c82f5-c823-45ce-a49f-5ce48193ffb4 None admin] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:47:44.955639 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b1c82f5-c823-45ce-a49f-5ce48193ffb4 None admin] PATH_INFO: `/v3/services/network` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:47:44.956227 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b1c82f5-c823-45ce-a49f-5ce48193ffb4 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 15:47:44.957594 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b1c82f5-c823-45ce-a49f-5ce48193ffb4 None admin] RBAC: Authorization granted {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 15:47:44.971932 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application [None req-5b1c82f5-c823-45ce-a49f-5ce48193ffb4 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Aug 22 15:47:44.971932 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 15:47:44.971932 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 22 15:47:44.971932 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 22 15:47:44.971932 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 22 15:47:44.971932 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 22 15:47:44.971932 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 22 15:47:44.971932 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 22 15:47:44.971932 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 22 15:47:44.971932 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application Aug 22 15:47:44.971932 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 22 15:47:44.971932 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application Aug 22 15:47:44.971932 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 15:47:44.971932 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 15:47:44.971932 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 15:47:44.971932 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 15:47:44.971932 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 15:47:44.972997 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 15:47:44.972997 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 15:47:44.972997 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 15:47:44.972997 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 15:47:44.972997 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 15:47:44.972997 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 15:47:44.972997 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 22 15:47:44.972997 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 22 15:47:44.972997 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 22 15:47:44.972997 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 22 15:47:44.972997 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 15:47:44.972997 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 15:47:44.972997 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 15:47:44.972997 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 15:47:44.972997 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 15:47:44.972997 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 15:47:44.973846 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 15:47:44.973846 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application with Lock( Aug 22 15:47:44.973846 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 15:47:44.973846 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return self._enter() Aug 22 15:47:44.973846 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 15:47:44.973846 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 15:47:44.973846 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 15:47:44.973846 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return self.creator() Aug 22 15:47:44.973846 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 15:47:44.973846 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application created_value = creator( Aug 22 15:47:44.973846 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 22 15:47:44.973846 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 22 15:47:44.973846 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 22 15:47:44.973846 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application Aug 22 15:47:44.975466 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 355/717] 10.222.0.15 () {56 vars in 1111 bytes} [Mon Aug 22 15:47:44 2022] GET /identity/v3/services/network => generated 88 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 15:47:44.981134 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8577f61-c870-4941-8a7d-b70ed7ae1aa6 None admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 15:47:45.111591 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8577f61-c870-4941-8a7d-b70ed7ae1aa6 None admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 15:47:45.159789 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-050a0204-ea46-40bc-b457-2e9eefa39796 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 15:47:45.160406 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-050a0204-ea46-40bc-b457-2e9eefa39796 None admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:47:45.160681 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-050a0204-ea46-40bc-b457-2e9eefa39796 None admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:47:45.160947 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-050a0204-ea46-40bc-b457-2e9eefa39796 None admin] PATH_INFO: `/v3/services` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:47:45.161579 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-050a0204-ea46-40bc-b457-2e9eefa39796 None admin] RBAC: Adding query filter params (name=network) {{(pid=81768) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 15:47:45.161874 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-050a0204-ea46-40bc-b457-2e9eefa39796 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 15:47:45.163378 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-050a0204-ea46-40bc-b457-2e9eefa39796 None admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 15:47:45.171587 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 363/718] 10.222.0.15 () {56 vars in 1112 bytes} [Mon Aug 22 15:47:44 2022] GET /identity/v3/services?name=network => generated 124 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 15:47:45.177033 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b1c82f5-c823-45ce-a49f-5ce48193ffb4 None admin] Authenticating user token {{(pid=81767) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 15:47:45.298835 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b1c82f5-c823-45ce-a49f-5ce48193ffb4 None admin] Validating token access rules against request {{(pid=81767) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 15:47:45.347950 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dc2bc8d-1bf1-43f4-b146-7a1567239ddb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81767) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 15:47:45.348643 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc2bc8d-1bf1-43f4-b146-7a1567239ddb None admin] REQUEST_METHOD: `GET` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:47:45.348921 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc2bc8d-1bf1-43f4-b146-7a1567239ddb None admin] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:47:45.349189 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc2bc8d-1bf1-43f4-b146-7a1567239ddb None admin] PATH_INFO: `/v3/services` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:47:45.350166 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dc2bc8d-1bf1-43f4-b146-7a1567239ddb None admin] RBAC: Adding query filter params (type=network) {{(pid=81767) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 15:47:45.350549 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dc2bc8d-1bf1-43f4-b146-7a1567239ddb None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 15:47:45.352010 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dc2bc8d-1bf1-43f4-b146-7a1567239ddb None admin] RBAC: Authorization granted {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 15:47:45.362795 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 356/719] 10.222.0.15 () {56 vars in 1112 bytes} [Mon Aug 22 15:47:45 2022] GET /identity/v3/services?type=network => generated 124 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 15:47:45.582419 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dfeb696-0902-4c42-a95f-1fe5ecb0c049 None None] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:47:45.582811 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dfeb696-0902-4c42-a95f-1fe5ecb0c049 None None] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:47:45.583190 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dfeb696-0902-4c42-a95f-1fe5ecb0c049 None None] PATH_INFO: `/` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:47:45.585447 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 364/720] 10.222.0.15 () {54 vars in 927 bytes} [Mon Aug 22 15:47:45 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 15:47:45.596743 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda442b9-3dfb-4855-85ce-3ee2776d8729 None None] REQUEST_METHOD: `POST` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:47:45.597406 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda442b9-3dfb-4855-85ce-3ee2776d8729 None None] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:47:45.597903 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda442b9-3dfb-4855-85ce-3ee2776d8729 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:47:45.609532 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG dbcounter [-] [81767] Writing DB stats keystone:SELECT=100 {{(pid=81767) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 15:47:45.629130 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.auth.core [None req-cda442b9-3dfb-4855-85ce-3ee2776d8729 None None] MFA Rules not processed for user `6b2199ae0fb0480aa917ada391f5ec54`. Rule list: `[]` (Enabled: `True`). {{(pid=81767) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 15:47:45.630700 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.fernet_utils [None req-cda442b9-3dfb-4855-85ce-3ee2776d8729 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81767) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 15:47:45.730292 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 357/721] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Aug 22 15:47:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 22 15:47:46.068237 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dfeb696-0902-4c42-a95f-1fe5ecb0c049 None None] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 15:47:46.108771 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 15:47:46.183564 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dfeb696-0902-4c42-a95f-1fe5ecb0c049 None None] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 15:47:46.226437 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a743e1ff-c342-4102-b0d5-b3ed8af6ce45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 15:47:46.227134 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a743e1ff-c342-4102-b0d5-b3ed8af6ce45 None admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:47:46.227438 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a743e1ff-c342-4102-b0d5-b3ed8af6ce45 None admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:47:46.227710 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a743e1ff-c342-4102-b0d5-b3ed8af6ce45 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:47:46.239587 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a743e1ff-c342-4102-b0d5-b3ed8af6ce45 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 22 15:47:46.245582 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application [None req-a743e1ff-c342-4102-b0d5-b3ed8af6ce45 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 22 15:47:46.245582 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 15:47:46.245582 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 15:47:46.245582 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 15:47:46.245582 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 15:47:46.245582 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 15:47:46.245582 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 15:47:46.245582 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 15:47:46.245582 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 15:47:46.245582 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 15:47:46.245582 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 15:47:46.245582 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 15:47:46.245582 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 22 15:47:46.245582 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 22 15:47:46.245582 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 22 15:47:46.245582 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 22 15:47:46.246529 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 22 15:47:46.246529 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 22 15:47:46.246529 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 22 15:47:46.246529 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 22 15:47:46.246529 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 15:47:46.246529 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 15:47:46.246529 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 15:47:46.246529 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 15:47:46.246529 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 15:47:46.246529 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 15:47:46.246529 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 15:47:46.246529 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application with Lock( Aug 22 15:47:46.246529 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 15:47:46.246529 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self._enter() Aug 22 15:47:46.246529 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 15:47:46.246529 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 15:47:46.247362 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 15:47:46.247362 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self.creator() Aug 22 15:47:46.247362 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 15:47:46.247362 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application created_value = creator( Aug 22 15:47:46.247362 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 22 15:47:46.247362 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 22 15:47:46.247362 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 22 15:47:46.247362 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 22 15:47:46.247362 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 22 15:47:46.247362 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 22 15:47:46.247362 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 22 15:47:46.247362 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application Aug 22 15:47:46.247362 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 365/722] 10.222.0.15 () {56 vars in 1096 bytes} [Mon Aug 22 15:47:46 2022] GET /identity/v3/roles/admin => generated 83 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 15:47:46.265400 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cda442b9-3dfb-4855-85ce-3ee2776d8729 None None] Authenticating user token {{(pid=81767) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 15:47:46.350403 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddb4c874-853d-41b0-8799-7d9b6bb1718f None None] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:47:46.350742 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddb4c874-853d-41b0-8799-7d9b6bb1718f None None] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:47:46.350978 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddb4c874-853d-41b0-8799-7d9b6bb1718f None None] PATH_INFO: `/` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:47:46.352870 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 366/723] 10.222.0.15 () {54 vars in 927 bytes} [Mon Aug 22 15:47:46 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 22 15:47:46.358639 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce28a547-4450-45a1-a8fd-19af81adf3da None None] REQUEST_METHOD: `POST` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:47:46.358921 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce28a547-4450-45a1-a8fd-19af81adf3da None None] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:47:46.359254 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce28a547-4450-45a1-a8fd-19af81adf3da None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:47:46.400480 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.auth.core [None req-ce28a547-4450-45a1-a8fd-19af81adf3da None None] MFA Rules not processed for user `6b2199ae0fb0480aa917ada391f5ec54`. Rule list: `[]` (Enabled: `True`). {{(pid=81768) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 15:47:46.401775 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-ce28a547-4450-45a1-a8fd-19af81adf3da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 15:47:46.420991 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cda442b9-3dfb-4855-85ce-3ee2776d8729 None None] Validating token access rules against request {{(pid=81767) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 15:47:46.467663 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG dbcounter [-] [81767] Writing DB stats keystone:SELECT=100 {{(pid=81767) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 15:47:46.473654 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5769103c-d654-4ff1-8745-9bcaa0f86b42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81767) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 15:47:46.474280 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5769103c-d654-4ff1-8745-9bcaa0f86b42 None admin] REQUEST_METHOD: `GET` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:47:46.474555 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5769103c-d654-4ff1-8745-9bcaa0f86b42 None admin] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:47:46.474826 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5769103c-d654-4ff1-8745-9bcaa0f86b42 None admin] PATH_INFO: `/v3/roles` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:47:46.475530 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5769103c-d654-4ff1-8745-9bcaa0f86b42 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81767) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 15:47:46.475831 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5769103c-d654-4ff1-8745-9bcaa0f86b42 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 15:47:46.477197 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5769103c-d654-4ff1-8745-9bcaa0f86b42 None admin] RBAC: Authorization granted {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 15:47:46.488497 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 358/724] 10.222.0.15 () {56 vars in 1099 bytes} [Mon Aug 22 15:47:46 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 15:47:46.500746 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5769103c-d654-4ff1-8745-9bcaa0f86b42 None admin] Authenticating user token {{(pid=81767) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 15:47:46.536737 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 367/725] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Aug 22 15:47:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 22 15:47:46.700657 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5769103c-d654-4ff1-8745-9bcaa0f86b42 None admin] Validating token access rules against request {{(pid=81767) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 15:47:46.747619 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4751ff94-194c-4a31-b933-e60cd1fba86d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81767) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 15:47:46.748311 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4751ff94-194c-4a31-b933-e60cd1fba86d None admin] REQUEST_METHOD: `GET` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:47:46.748587 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4751ff94-194c-4a31-b933-e60cd1fba86d None admin] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:47:46.748907 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4751ff94-194c-4a31-b933-e60cd1fba86d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:47:46.765485 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4751ff94-194c-4a31-b933-e60cd1fba86d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 15:47:46.767007 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4751ff94-194c-4a31-b933-e60cd1fba86d None admin] RBAC: Authorization granted {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 15:47:46.784859 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application [None req-4751ff94-194c-4a31-b933-e60cd1fba86d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 22 15:47:46.784859 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 15:47:46.784859 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 15:47:46.784859 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 15:47:46.784859 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 15:47:46.784859 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 15:47:46.784859 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 15:47:46.784859 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 15:47:46.784859 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 15:47:46.784859 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 15:47:46.784859 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 15:47:46.784859 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 15:47:46.784859 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 22 15:47:46.784859 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 22 15:47:46.784859 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 22 15:47:46.784859 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 22 15:47:46.785911 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 15:47:46.785911 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 15:47:46.785911 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 15:47:46.785911 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 15:47:46.785911 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 15:47:46.785911 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 15:47:46.785911 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 15:47:46.785911 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application with Lock( Aug 22 15:47:46.785911 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 15:47:46.785911 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return self._enter() Aug 22 15:47:46.785911 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 15:47:46.785911 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 15:47:46.785911 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 15:47:46.785911 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return self.creator() Aug 22 15:47:46.785911 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 15:47:46.785911 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application created_value = creator( Aug 22 15:47:46.785911 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 22 15:47:46.786767 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 22 15:47:46.786767 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 22 15:47:46.786767 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application Aug 22 15:47:46.788492 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 359/726] 10.222.0.15 () {56 vars in 1108 bytes} [Mon Aug 22 15:47:46 2022] GET /identity/v3/domains/Default => generated 87 bytes in 286 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 15:47:46.793553 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce28a547-4450-45a1-a8fd-19af81adf3da None None] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 15:47:46.833479 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4751ff94-194c-4a31-b933-e60cd1fba86d None admin] Authenticating user token {{(pid=81767) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 15:47:46.841145 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=100 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 15:47:46.898683 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG dbcounter [-] [81767] Writing DB stats keystone:SELECT=100 {{(pid=81767) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 15:47:46.935812 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce28a547-4450-45a1-a8fd-19af81adf3da None None] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 15:47:46.965377 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4751ff94-194c-4a31-b933-e60cd1fba86d None admin] Validating token access rules against request {{(pid=81767) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 15:47:46.989435 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77f0581b-66ba-4f00-985d-cc668c9013f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 15:47:46.990277 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77f0581b-66ba-4f00-985d-cc668c9013f9 None admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:47:46.990626 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77f0581b-66ba-4f00-985d-cc668c9013f9 None admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:47:46.990974 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77f0581b-66ba-4f00-985d-cc668c9013f9 None admin] PATH_INFO: `/v3/domains` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:47:46.991730 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77f0581b-66ba-4f00-985d-cc668c9013f9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81768) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 15:47:46.992095 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77f0581b-66ba-4f00-985d-cc668c9013f9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 15:47:46.993550 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77f0581b-66ba-4f00-985d-cc668c9013f9 None admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 15:47:47.024690 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 368/727] 10.222.0.15 () {56 vars in 1109 bytes} [Mon Aug 22 15:47:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 15:47:47.024690 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c266839-4aea-41ad-b85e-b0eec5462d39 None None] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:47:47.024690 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c266839-4aea-41ad-b85e-b0eec5462d39 None None] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:47:47.024690 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c266839-4aea-41ad-b85e-b0eec5462d39 None None] PATH_INFO: `/` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:47:47.024690 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 369/728] 10.222.0.15 () {54 vars in 927 bytes} [Mon Aug 22 15:47:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 15:47:47.028889 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c266839-4aea-41ad-b85e-b0eec5462d39 None None] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 15:47:47.142856 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=100 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 15:47:47.154428 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33cc7f8e-640c-4f8b-aecf-c3500d16587b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81767) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 15:47:47.159526 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33cc7f8e-640c-4f8b-aecf-c3500d16587b None admin] REQUEST_METHOD: `POST` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:47:47.159526 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33cc7f8e-640c-4f8b-aecf-c3500d16587b None admin] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:47:47.159526 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33cc7f8e-640c-4f8b-aecf-c3500d16587b None admin] PATH_INFO: `/v3/services` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:47:47.159526 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33cc7f8e-640c-4f8b-aecf-c3500d16587b None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 15:47:47.161541 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33cc7f8e-640c-4f8b-aecf-c3500d16587b None admin] RBAC: Authorization granted {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 15:47:47.170032 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c266839-4aea-41ad-b85e-b0eec5462d39 None None] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 15:47:47.219234 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08853e5c-70c0-4272-b752-ed130b27c809 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 15:47:47.219780 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08853e5c-70c0-4272-b752-ed130b27c809 None admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:47:47.220045 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08853e5c-70c0-4272-b752-ed130b27c809 None admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:47:47.220319 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08853e5c-70c0-4272-b752-ed130b27c809 None admin] PATH_INFO: `/v3/users/swift` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:47:47.228378 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08853e5c-70c0-4272-b752-ed130b27c809 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 15:47:47.230366 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08853e5c-70c0-4272-b752-ed130b27c809 None admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 15:47:47.242405 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application [None req-08853e5c-70c0-4272-b752-ed130b27c809 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 22 15:47:47.242405 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 15:47:47.242405 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 15:47:47.242405 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 15:47:47.242405 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 15:47:47.242405 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 15:47:47.242405 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 15:47:47.242405 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 15:47:47.242405 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 15:47:47.242405 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 15:47:47.242405 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 15:47:47.242405 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 15:47:47.242405 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 22 15:47:47.242405 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 22 15:47:47.242405 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 22 15:47:47.242405 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 22 15:47:47.245126 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 15:47:47.245126 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 15:47:47.245126 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 22 15:47:47.245126 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 22 15:47:47.245126 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 22 15:47:47.245126 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 22 15:47:47.245126 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 15:47:47.245126 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 15:47:47.245126 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 15:47:47.245126 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 15:47:47.245126 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 15:47:47.245126 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application with Lock( Aug 22 15:47:47.245126 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 15:47:47.245126 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self._enter() Aug 22 15:47:47.245126 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 15:47:47.245126 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 15:47:47.245126 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 15:47:47.245126 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self.creator() Aug 22 15:47:47.246617 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 15:47:47.246617 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application created_value = creator( Aug 22 15:47:47.246617 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 22 15:47:47.246617 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 22 15:47:47.246617 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 22 15:47:47.246617 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 22 15:47:47.246617 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 22 15:47:47.246617 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 22 15:47:47.246617 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 22 15:47:47.246617 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application Aug 22 15:47:47.248297 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 370/729] 10.222.0.15 () {56 vars in 1096 bytes} [Mon Aug 22 15:47:47 2022] GET /identity/v3/users/swift => generated 83 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 15:47:47.251048 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d63f73c2-3fcb-488f-89eb-4ed8a53a4ab1 None None] REQUEST_METHOD: `POST` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:47:47.251558 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d63f73c2-3fcb-488f-89eb-4ed8a53a4ab1 None None] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:47:47.252236 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d63f73c2-3fcb-488f-89eb-4ed8a53a4ab1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:47:47.271968 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.auth.core [None req-d63f73c2-3fcb-488f-89eb-4ed8a53a4ab1 None None] MFA Rules not processed for user `6b2199ae0fb0480aa917ada391f5ec54`. Rule list: `[]` (Enabled: `True`). {{(pid=81768) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 15:47:47.273268 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-d63f73c2-3fcb-488f-89eb-4ed8a53a4ab1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 15:47:47.341976 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 360/730] 10.222.0.15 () {60 vars in 1141 bytes} [Mon Aug 22 15:47:46 2022] POST /identity/v3/services => generated 251 bytes in 509 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 22 15:47:47.343652 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33cc7f8e-640c-4f8b-aecf-c3500d16587b None admin] Authenticating user token {{(pid=81767) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 15:47:47.384355 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 371/731] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Aug 22 15:47:47 2022] POST /identity/v3/auth/tokens => generated 903 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 22 15:47:47.386197 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8356b44-186f-4b7e-a7ee-6500ad759029 None None] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:47:47.386197 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8356b44-186f-4b7e-a7ee-6500ad759029 None None] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:47:47.386404 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8356b44-186f-4b7e-a7ee-6500ad759029 None None] PATH_INFO: `/` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:47:47.391658 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 372/732] 10.222.0.15 () {54 vars in 927 bytes} [Mon Aug 22 15:47:47 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 15:47:47.394137 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4317a37-4806-4c88-8901-aa1cd9d94712 None None] REQUEST_METHOD: `POST` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:47:47.394137 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4317a37-4806-4c88-8901-aa1cd9d94712 None None] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:47:47.394522 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4317a37-4806-4c88-8901-aa1cd9d94712 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:47:47.417243 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.auth.core [None req-f4317a37-4806-4c88-8901-aa1cd9d94712 None None] MFA Rules not processed for user `6b2199ae0fb0480aa917ada391f5ec54`. Rule list: `[]` (Enabled: `True`). {{(pid=81768) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 15:47:47.418807 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-f4317a37-4806-4c88-8901-aa1cd9d94712 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 15:47:47.492533 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33cc7f8e-640c-4f8b-aecf-c3500d16587b None admin] Validating token access rules against request {{(pid=81767) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 15:47:47.511671 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG dbcounter [-] [81767] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81767) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 15:47:47.535582 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=100 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 15:47:47.548061 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 373/733] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Aug 22 15:47:47 2022] POST /identity/v3/auth/tokens => generated 903 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 22 15:47:47.574901 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f64f71a8-e672-469f-beef-c7e56e86b95f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81767) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 15:47:47.575663 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f64f71a8-e672-469f-beef-c7e56e86b95f None admin] REQUEST_METHOD: `GET` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:47:47.575969 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f64f71a8-e672-469f-beef-c7e56e86b95f None admin] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:47:47.576259 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f64f71a8-e672-469f-beef-c7e56e86b95f None admin] PATH_INFO: `/v3/users` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:47:47.578181 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f64f71a8-e672-469f-beef-c7e56e86b95f None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81767) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 15:47:47.578181 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f64f71a8-e672-469f-beef-c7e56e86b95f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 15:47:47.578959 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f64f71a8-e672-469f-beef-c7e56e86b95f None admin] RBAC: Authorization granted {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 15:47:47.605413 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 361/734] 10.222.0.15 () {56 vars in 1135 bytes} [Mon Aug 22 15:47:47 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 262 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 15:47:47.611632 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4317a37-4806-4c88-8901-aa1cd9d94712 None None] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 15:47:47.728519 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4317a37-4806-4c88-8901-aa1cd9d94712 None None] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 15:47:47.730788 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f64f71a8-e672-469f-beef-c7e56e86b95f None admin] Authenticating user token {{(pid=81767) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 15:47:47.785944 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbf073e3-9c72-4368-a3f9-8cec266f134f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 15:47:47.786672 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf073e3-9c72-4368-a3f9-8cec266f134f None admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:47:47.787033 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf073e3-9c72-4368-a3f9-8cec266f134f None admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:47:47.787449 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf073e3-9c72-4368-a3f9-8cec266f134f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:47:47.868304 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f64f71a8-e672-469f-beef-c7e56e86b95f None admin] Validating token access rules against request {{(pid=81767) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 15:47:47.881391 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=100 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 15:47:47.890447 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbf073e3-9c72-4368-a3f9-8cec266f134f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 15:47:47.892693 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbf073e3-9c72-4368-a3f9-8cec266f134f None admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 15:47:47.917214 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a66a18a0-e378-4631-b747-8be2e02323aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81767) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 15:47:47.917849 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a66a18a0-e378-4631-b747-8be2e02323aa None admin] REQUEST_METHOD: `GET` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:47:47.918128 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a66a18a0-e378-4631-b747-8be2e02323aa None admin] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:47:47.918399 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a66a18a0-e378-4631-b747-8be2e02323aa None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:47:47.930755 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a66a18a0-e378-4631-b747-8be2e02323aa None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 22 15:47:47.936278 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application [None req-a66a18a0-e378-4631-b747-8be2e02323aa None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 22 15:47:47.936278 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 15:47:47.936278 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 15:47:47.936278 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 15:47:47.936278 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 15:47:47.936278 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 15:47:47.936278 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 15:47:47.936278 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 15:47:47.936278 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 15:47:47.936278 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 15:47:47.936278 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 15:47:47.936278 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 15:47:47.936278 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 22 15:47:47.936278 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 22 15:47:47.936278 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 22 15:47:47.936278 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 22 15:47:47.945557 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 22 15:47:47.945557 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 22 15:47:47.945557 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 22 15:47:47.945557 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 22 15:47:47.945557 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 15:47:47.945557 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 15:47:47.945557 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 15:47:47.945557 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 15:47:47.945557 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 15:47:47.945557 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 15:47:47.945557 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 15:47:47.945557 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application with Lock( Aug 22 15:47:47.945557 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 15:47:47.945557 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return self._enter() Aug 22 15:47:47.945557 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 15:47:47.945557 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 15:47:47.946489 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 15:47:47.946489 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return self.creator() Aug 22 15:47:47.946489 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 15:47:47.946489 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application created_value = creator( Aug 22 15:47:47.946489 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 22 15:47:47.946489 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 22 15:47:47.946489 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 22 15:47:47.946489 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 22 15:47:47.946489 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 22 15:47:47.946489 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 22 15:47:47.946489 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 22 15:47:47.946489 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application Aug 22 15:47:47.946489 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 362/735] 10.222.0.15 () {56 vars in 1096 bytes} [Mon Aug 22 15:47:47 2022] GET /identity/v3/roles/admin => generated 83 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 15:47:47.946489 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a66a18a0-e378-4631-b747-8be2e02323aa None admin] Authenticating user token {{(pid=81767) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 15:47:47.968199 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG dbcounter [-] [81767] Writing DB stats keystone:SELECT=100 {{(pid=81767) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 15:47:48.079448 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a66a18a0-e378-4631-b747-8be2e02323aa None admin] Validating token access rules against request {{(pid=81767) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 15:47:48.112196 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 374/736] 10.222.0.15 () {58 vars in 1282 bytes} [Mon Aug 22 15:47:47 2022] GET /identity/v3/auth/tokens => generated 903 bytes in 501 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 22 15:47:48.113038 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbf073e3-9c72-4368-a3f9-8cec266f134f None admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 15:47:48.121860 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e599256b-d378-4771-ad65-53cb802d6a5a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81767) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 15:47:48.122641 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e599256b-d378-4771-ad65-53cb802d6a5a None admin] REQUEST_METHOD: `GET` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:47:48.123018 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e599256b-d378-4771-ad65-53cb802d6a5a None admin] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:47:48.124095 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e599256b-d378-4771-ad65-53cb802d6a5a None admin] PATH_INFO: `/v3/roles/service` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:47:48.135510 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e599256b-d378-4771-ad65-53cb802d6a5a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 22 15:47:48.140565 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application [None req-e599256b-d378-4771-ad65-53cb802d6a5a None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 22 15:47:48.140565 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 15:47:48.140565 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 15:47:48.140565 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 15:47:48.140565 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 15:47:48.140565 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 15:47:48.140565 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 15:47:48.140565 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 15:47:48.140565 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 15:47:48.140565 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 15:47:48.140565 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 15:47:48.140565 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 15:47:48.140565 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 22 15:47:48.140565 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 22 15:47:48.140565 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 22 15:47:48.140565 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 22 15:47:48.141729 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 22 15:47:48.141729 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 22 15:47:48.141729 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 22 15:47:48.141729 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 22 15:47:48.141729 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 15:47:48.141729 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 15:47:48.141729 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 15:47:48.141729 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 15:47:48.141729 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 15:47:48.141729 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 15:47:48.141729 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 15:47:48.141729 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application with Lock( Aug 22 15:47:48.141729 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 15:47:48.141729 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return self._enter() Aug 22 15:47:48.141729 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 15:47:48.141729 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 15:47:48.142790 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 15:47:48.142790 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return self.creator() Aug 22 15:47:48.142790 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 15:47:48.142790 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application created_value = creator( Aug 22 15:47:48.142790 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 22 15:47:48.142790 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 22 15:47:48.142790 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 22 15:47:48.142790 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 22 15:47:48.142790 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 22 15:47:48.142790 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 22 15:47:48.142790 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 22 15:47:48.142790 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application Aug 22 15:47:48.142790 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 363/737] 10.222.0.15 () {56 vars in 1102 bytes} [Mon Aug 22 15:47:47 2022] GET /identity/v3/roles/service => generated 85 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 15:47:48.143801 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e7a591a-6bfe-4ff3-be7f-017bf54c1e6c None None] REQUEST_METHOD: `GET` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:47:48.144079 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e7a591a-6bfe-4ff3-be7f-017bf54c1e6c None None] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:47:48.144377 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e7a591a-6bfe-4ff3-be7f-017bf54c1e6c None None] PATH_INFO: `/` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:47:48.145769 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 364/738] 10.222.0.15 () {54 vars in 927 bytes} [Mon Aug 22 15:47:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 15:47:48.146309 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e7a591a-6bfe-4ff3-be7f-017bf54c1e6c None None] Authenticating user token {{(pid=81767) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 15:47:48.238133 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=100 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 15:47:48.241571 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbf073e3-9c72-4368-a3f9-8cec266f134f None admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 15:47:48.252062 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG dbcounter [-] [81767] Writing DB stats keystone:SELECT=100 {{(pid=81767) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 15:47:48.269643 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e7a591a-6bfe-4ff3-be7f-017bf54c1e6c None None] Validating token access rules against request {{(pid=81767) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 15:47:48.283894 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f9ab54b-c20b-47b8-928d-deef6b1a62e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 15:47:48.284496 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f9ab54b-c20b-47b8-928d-deef6b1a62e6 None admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:47:48.284770 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f9ab54b-c20b-47b8-928d-deef6b1a62e6 None admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:47:48.285034 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f9ab54b-c20b-47b8-928d-deef6b1a62e6 None admin] PATH_INFO: `/v3/roles` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:47:48.285688 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f9ab54b-c20b-47b8-928d-deef6b1a62e6 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81768) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 15:47:48.285984 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f9ab54b-c20b-47b8-928d-deef6b1a62e6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 15:47:48.287376 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f9ab54b-c20b-47b8-928d-deef6b1a62e6 None admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 15:47:48.296565 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 375/739] 10.222.0.15 () {56 vars in 1099 bytes} [Mon Aug 22 15:47:48 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 15:47:48.297828 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a55756e-109a-4782-a18a-7db3d7a4bbc0 None None] REQUEST_METHOD: `POST` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:47:48.298104 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a55756e-109a-4782-a18a-7db3d7a4bbc0 None None] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:47:48.298380 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a55756e-109a-4782-a18a-7db3d7a4bbc0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:47:48.318312 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.auth.core [None req-5a55756e-109a-4782-a18a-7db3d7a4bbc0 None None] MFA Rules not processed for user `6b2199ae0fb0480aa917ada391f5ec54`. Rule list: `[]` (Enabled: `True`). {{(pid=81768) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 15:47:48.319578 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-5a55756e-109a-4782-a18a-7db3d7a4bbc0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 15:47:48.324631 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d677aca-a7b5-41a2-8f56-9bf22767a63d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81767) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 15:47:48.325363 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d677aca-a7b5-41a2-8f56-9bf22767a63d None admin] REQUEST_METHOD: `GET` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:47:48.325609 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d677aca-a7b5-41a2-8f56-9bf22767a63d None admin] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:47:48.325868 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d677aca-a7b5-41a2-8f56-9bf22767a63d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:47:48.341029 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d677aca-a7b5-41a2-8f56-9bf22767a63d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 15:47:48.342760 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d677aca-a7b5-41a2-8f56-9bf22767a63d None admin] RBAC: Authorization granted {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 15:47:48.365080 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application [None req-0d677aca-a7b5-41a2-8f56-9bf22767a63d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 22 15:47:48.365080 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 15:47:48.365080 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 15:47:48.365080 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 15:47:48.365080 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 15:47:48.365080 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 15:47:48.365080 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 15:47:48.365080 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 15:47:48.365080 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 15:47:48.365080 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 15:47:48.365080 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 15:47:48.365080 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 15:47:48.365080 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 22 15:47:48.365080 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 22 15:47:48.365080 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 22 15:47:48.365080 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 22 15:47:48.366252 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 15:47:48.366252 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 15:47:48.366252 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 15:47:48.366252 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 15:47:48.366252 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 15:47:48.366252 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 15:47:48.366252 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 15:47:48.366252 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application with Lock( Aug 22 15:47:48.366252 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 15:47:48.366252 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return self._enter() Aug 22 15:47:48.366252 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 15:47:48.366252 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 15:47:48.366252 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 15:47:48.366252 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return self.creator() Aug 22 15:47:48.366252 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 15:47:48.366252 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application created_value = creator( Aug 22 15:47:48.366252 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 22 15:47:48.368841 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 22 15:47:48.368841 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 22 15:47:48.368841 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application Aug 22 15:47:48.368841 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 365/740] 10.222.0.15 () {56 vars in 1108 bytes} [Mon Aug 22 15:47:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 15:47:48.368841 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d677aca-a7b5-41a2-8f56-9bf22767a63d None admin] Authenticating user token {{(pid=81767) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 15:47:48.411669 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 376/741] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Aug 22 15:47:48 2022] POST /identity/v3/auth/tokens => generated 903 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 22 15:47:48.413391 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a55756e-109a-4782-a18a-7db3d7a4bbc0 None None] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 15:47:48.473358 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d677aca-a7b5-41a2-8f56-9bf22767a63d None admin] Validating token access rules against request {{(pid=81767) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 15:47:48.508860 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbcb9843-a871-4647-9ffd-377008b1fcd0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81767) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 15:47:48.509495 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbcb9843-a871-4647-9ffd-377008b1fcd0 None admin] REQUEST_METHOD: `GET` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:47:48.509770 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbcb9843-a871-4647-9ffd-377008b1fcd0 None admin] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:47:48.510576 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbcb9843-a871-4647-9ffd-377008b1fcd0 None admin] PATH_INFO: `/v3/roles` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:47:48.511232 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbcb9843-a871-4647-9ffd-377008b1fcd0 None admin] RBAC: Adding query filter params (name=service) {{(pid=81767) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 15:47:48.511574 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbcb9843-a871-4647-9ffd-377008b1fcd0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 15:47:48.513015 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbcb9843-a871-4647-9ffd-377008b1fcd0 None admin] RBAC: Authorization granted {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 15:47:48.515399 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG dbcounter [-] [81767] Writing DB stats keystone:SELECT=100 {{(pid=81767) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 15:47:48.521647 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=100 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 15:47:48.528044 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 366/742] 10.222.0.15 () {56 vars in 1103 bytes} [Mon Aug 22 15:47:48 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 15:47:48.528044 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbcb9843-a871-4647-9ffd-377008b1fcd0 None admin] Authenticating user token {{(pid=81767) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 15:47:48.538188 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a55756e-109a-4782-a18a-7db3d7a4bbc0 None None] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 15:47:48.581643 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ec4c3be-1016-47ba-88e9-927fa4aa15c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 15:47:48.585265 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec4c3be-1016-47ba-88e9-927fa4aa15c2 None admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:47:48.585265 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec4c3be-1016-47ba-88e9-927fa4aa15c2 None admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:47:48.585265 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec4c3be-1016-47ba-88e9-927fa4aa15c2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:47:48.596252 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ec4c3be-1016-47ba-88e9-927fa4aa15c2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 15:47:48.598162 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ec4c3be-1016-47ba-88e9-927fa4aa15c2 None admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 15:47:48.614961 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application [None req-1ec4c3be-1016-47ba-88e9-927fa4aa15c2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 22 15:47:48.614961 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 15:47:48.614961 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 15:47:48.614961 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 15:47:48.614961 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 15:47:48.614961 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 15:47:48.614961 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 15:47:48.614961 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 15:47:48.614961 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 15:47:48.614961 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 15:47:48.614961 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 15:47:48.614961 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 15:47:48.614961 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 22 15:47:48.614961 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 22 15:47:48.614961 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 22 15:47:48.614961 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 22 15:47:48.615969 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 15:47:48.615969 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 15:47:48.615969 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 15:47:48.615969 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 15:47:48.615969 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 15:47:48.615969 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 15:47:48.615969 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 15:47:48.615969 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application with Lock( Aug 22 15:47:48.615969 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 15:47:48.615969 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self._enter() Aug 22 15:47:48.615969 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 15:47:48.615969 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 15:47:48.615969 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 15:47:48.615969 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self.creator() Aug 22 15:47:48.615969 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 15:47:48.615969 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application created_value = creator( Aug 22 15:47:48.615969 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 22 15:47:48.616830 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 22 15:47:48.616830 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 22 15:47:48.616830 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application Aug 22 15:47:48.623526 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 377/743] 10.222.0.15 () {56 vars in 1108 bytes} [Mon Aug 22 15:47:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 15:47:48.624690 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ec4c3be-1016-47ba-88e9-927fa4aa15c2 None admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 15:47:48.651120 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbcb9843-a871-4647-9ffd-377008b1fcd0 None admin] Validating token access rules against request {{(pid=81767) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 15:47:48.692344 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1baafbd-d334-4219-a493-5999ced7730c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81767) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 15:47:48.693171 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1baafbd-d334-4219-a493-5999ced7730c None admin] REQUEST_METHOD: `GET` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:47:48.693279 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1baafbd-d334-4219-a493-5999ced7730c None admin] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:47:48.693506 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1baafbd-d334-4219-a493-5999ced7730c None admin] PATH_INFO: `/v3/domains` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:47:48.694162 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1baafbd-d334-4219-a493-5999ced7730c None admin] RBAC: Adding query filter params (name=Default) {{(pid=81767) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 15:47:48.694463 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1baafbd-d334-4219-a493-5999ced7730c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 15:47:48.695896 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1baafbd-d334-4219-a493-5999ced7730c None admin] RBAC: Authorization granted {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 15:47:48.711446 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 367/744] 10.222.0.15 () {56 vars in 1109 bytes} [Mon Aug 22 15:47:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 15:47:48.713093 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1baafbd-d334-4219-a493-5999ced7730c None admin] Authenticating user token {{(pid=81767) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 15:47:48.763576 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ec4c3be-1016-47ba-88e9-927fa4aa15c2 None admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 15:47:48.786939 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG dbcounter [-] [81767] Writing DB stats keystone:SELECT=100 {{(pid=81767) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 15:47:48.813796 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc743cfe-9408-4684-ac7e-2ad7f7e1cf4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 15:47:48.813796 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc743cfe-9408-4684-ac7e-2ad7f7e1cf4a None admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:47:48.815036 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc743cfe-9408-4684-ac7e-2ad7f7e1cf4a None admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:47:48.815121 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc743cfe-9408-4684-ac7e-2ad7f7e1cf4a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:47:48.829706 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc743cfe-9408-4684-ac7e-2ad7f7e1cf4a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 15:47:48.831665 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc743cfe-9408-4684-ac7e-2ad7f7e1cf4a None admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 15:47:48.835075 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=100 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 15:47:48.842351 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1baafbd-d334-4219-a493-5999ced7730c None admin] Validating token access rules against request {{(pid=81767) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 15:47:48.849693 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application [None req-dc743cfe-9408-4684-ac7e-2ad7f7e1cf4a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 22 15:47:48.849693 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 15:47:48.849693 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 15:47:48.849693 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 15:47:48.849693 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 15:47:48.849693 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 15:47:48.849693 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 15:47:48.849693 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 15:47:48.849693 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 15:47:48.849693 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 15:47:48.849693 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 15:47:48.849693 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 15:47:48.849693 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 22 15:47:48.849693 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 22 15:47:48.849693 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 22 15:47:48.849693 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 22 15:47:48.851129 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 15:47:48.851129 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 15:47:48.851129 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 15:47:48.851129 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 15:47:48.851129 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 15:47:48.851129 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 15:47:48.851129 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 15:47:48.851129 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application with Lock( Aug 22 15:47:48.851129 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 15:47:48.851129 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self._enter() Aug 22 15:47:48.851129 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 15:47:48.851129 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 15:47:48.851129 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 15:47:48.851129 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self.creator() Aug 22 15:47:48.851129 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 15:47:48.851129 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application created_value = creator( Aug 22 15:47:48.851129 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 22 15:47:48.853254 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 22 15:47:48.853254 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 22 15:47:48.853254 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application Aug 22 15:47:48.853254 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 378/745] 10.222.0.15 () {56 vars in 1108 bytes} [Mon Aug 22 15:47:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 227 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 15:47:48.853254 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc743cfe-9408-4684-ac7e-2ad7f7e1cf4a None admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 15:47:48.888214 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c900670-732c-4805-a79c-f4932bbd1345 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81767) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 15:47:48.889250 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c900670-732c-4805-a79c-f4932bbd1345 None admin] REQUEST_METHOD: `GET` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:47:48.889893 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c900670-732c-4805-a79c-f4932bbd1345 None admin] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:47:48.890387 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c900670-732c-4805-a79c-f4932bbd1345 None admin] PATH_INFO: `/v3/domains` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:47:48.891611 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c900670-732c-4805-a79c-f4932bbd1345 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81767) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 15:47:48.892145 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c900670-732c-4805-a79c-f4932bbd1345 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 15:47:48.894735 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c900670-732c-4805-a79c-f4932bbd1345 None admin] RBAC: Authorization granted {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 15:47:48.923323 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 368/746] 10.222.0.15 () {56 vars in 1109 bytes} [Mon Aug 22 15:47:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 15:47:48.923323 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c900670-732c-4805-a79c-f4932bbd1345 None admin] Authenticating user token {{(pid=81767) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 15:47:48.985913 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc743cfe-9408-4684-ac7e-2ad7f7e1cf4a None admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 15:47:49.030142 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b152abd-2c1c-41e9-bc8c-dcaa756acda3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 15:47:49.030864 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b152abd-2c1c-41e9-bc8c-dcaa756acda3 None admin] REQUEST_METHOD: `POST` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:47:49.031218 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b152abd-2c1c-41e9-bc8c-dcaa756acda3 None admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:47:49.031658 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b152abd-2c1c-41e9-bc8c-dcaa756acda3 None admin] PATH_INFO: `/v3/services` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:47:49.032377 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b152abd-2c1c-41e9-bc8c-dcaa756acda3 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 15:47:49.033885 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b152abd-2c1c-41e9-bc8c-dcaa756acda3 None admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 15:47:49.048962 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c900670-732c-4805-a79c-f4932bbd1345 None admin] Validating token access rules against request {{(pid=81767) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 15:47:49.071685 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG dbcounter [-] [81767] Writing DB stats keystone:SELECT=100 {{(pid=81767) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 15:47:49.089251 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5dc0273-e9de-46c7-afdc-5dc92d0130bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81767) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 15:47:49.089895 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5dc0273-e9de-46c7-afdc-5dc92d0130bd None admin] REQUEST_METHOD: `GET` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:47:49.090174 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5dc0273-e9de-46c7-afdc-5dc92d0130bd None admin] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:47:49.090444 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5dc0273-e9de-46c7-afdc-5dc92d0130bd None admin] PATH_INFO: `/v3/projects/service` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:47:49.097594 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5dc0273-e9de-46c7-afdc-5dc92d0130bd None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 15:47:49.099268 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5dc0273-e9de-46c7-afdc-5dc92d0130bd None admin] RBAC: Authorization granted {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 15:47:49.111944 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application [None req-a5dc0273-e9de-46c7-afdc-5dc92d0130bd None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 22 15:47:49.111944 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 15:47:49.111944 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 15:47:49.111944 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 15:47:49.111944 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 15:47:49.111944 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 15:47:49.111944 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 15:47:49.111944 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 15:47:49.111944 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 15:47:49.111944 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 15:47:49.111944 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 15:47:49.111944 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 15:47:49.111944 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 22 15:47:49.111944 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 22 15:47:49.111944 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 22 15:47:49.113129 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 22 15:47:49.113129 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 15:47:49.113129 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 15:47:49.113129 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 15:47:49.113129 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 15:47:49.113129 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 15:47:49.113129 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 15:47:49.113129 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 15:47:49.113129 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application with Lock( Aug 22 15:47:49.113129 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 15:47:49.113129 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return self._enter() Aug 22 15:47:49.113129 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 15:47:49.113129 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 15:47:49.113129 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 15:47:49.113129 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return self.creator() Aug 22 15:47:49.113129 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 15:47:49.113129 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application created_value = creator( Aug 22 15:47:49.114008 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 22 15:47:49.114008 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 22 15:47:49.114008 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 22 15:47:49.114008 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 22 15:47:49.114008 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 22 15:47:49.114008 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 22 15:47:49.114008 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 22 15:47:49.114008 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application Aug 22 15:47:49.114008 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 369/747] 10.222.0.15 () {56 vars in 1111 bytes} [Mon Aug 22 15:47:48 2022] GET /identity/v3/projects/service => generated 88 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 15:47:49.114622 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5dc0273-e9de-46c7-afdc-5dc92d0130bd None admin] Authenticating user token {{(pid=81767) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 15:47:49.128845 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 379/748] 10.222.0.15 () {60 vars in 1141 bytes} [Mon Aug 22 15:47:48 2022] POST /identity/v3/services => generated 240 bytes in 277 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 22 15:47:49.130349 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b152abd-2c1c-41e9-bc8c-dcaa756acda3 None admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 15:47:49.223763 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 15:47:49.259375 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5dc0273-e9de-46c7-afdc-5dc92d0130bd None admin] Validating token access rules against request {{(pid=81767) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 15:47:49.286376 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b152abd-2c1c-41e9-bc8c-dcaa756acda3 None admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 15:47:49.311517 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84c85b2f-ab4a-4c72-9e30-fe920b15e7a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81767) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 15:47:49.312136 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84c85b2f-ab4a-4c72-9e30-fe920b15e7a9 None admin] REQUEST_METHOD: `GET` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:47:49.312415 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84c85b2f-ab4a-4c72-9e30-fe920b15e7a9 None admin] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:47:49.312688 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84c85b2f-ab4a-4c72-9e30-fe920b15e7a9 None admin] PATH_INFO: `/v3/domains` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:47:49.313354 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84c85b2f-ab4a-4c72-9e30-fe920b15e7a9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81767) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 15:47:49.313661 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84c85b2f-ab4a-4c72-9e30-fe920b15e7a9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 15:47:49.315046 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84c85b2f-ab4a-4c72-9e30-fe920b15e7a9 None admin] RBAC: Authorization granted {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 15:47:49.335481 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 370/749] 10.222.0.15 () {56 vars in 1109 bytes} [Mon Aug 22 15:47:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 15:47:49.337308 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84c85b2f-ab4a-4c72-9e30-fe920b15e7a9 None admin] Authenticating user token {{(pid=81767) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 15:47:49.352131 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e3321f9-4b16-4a55-a226-674139feb67a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 15:47:49.352958 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e3321f9-4b16-4a55-a226-674139feb67a None admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:47:49.353116 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e3321f9-4b16-4a55-a226-674139feb67a None admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:47:49.353394 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e3321f9-4b16-4a55-a226-674139feb67a None admin] PATH_INFO: `/v3/users/nova` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:47:49.366663 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e3321f9-4b16-4a55-a226-674139feb67a None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 15:47:49.368330 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e3321f9-4b16-4a55-a226-674139feb67a None admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 15:47:49.383310 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application [None req-4e3321f9-4b16-4a55-a226-674139feb67a None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 22 15:47:49.383310 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 15:47:49.383310 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 15:47:49.383310 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 15:47:49.383310 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 15:47:49.383310 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 15:47:49.383310 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 15:47:49.383310 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 15:47:49.383310 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 15:47:49.383310 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 15:47:49.383310 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 15:47:49.383310 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 15:47:49.383310 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 22 15:47:49.383310 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 22 15:47:49.383310 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 22 15:47:49.383310 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 22 15:47:49.385534 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 15:47:49.385534 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 15:47:49.385534 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 22 15:47:49.385534 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 22 15:47:49.385534 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 22 15:47:49.385534 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 22 15:47:49.385534 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 15:47:49.385534 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 15:47:49.385534 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 15:47:49.385534 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 15:47:49.385534 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 15:47:49.385534 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application with Lock( Aug 22 15:47:49.385534 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 15:47:49.385534 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self._enter() Aug 22 15:47:49.385534 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 15:47:49.385534 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 15:47:49.385534 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 15:47:49.385534 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self.creator() Aug 22 15:47:49.386969 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 15:47:49.386969 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application created_value = creator( Aug 22 15:47:49.386969 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 22 15:47:49.386969 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 22 15:47:49.386969 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 22 15:47:49.386969 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 22 15:47:49.386969 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 22 15:47:49.386969 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 22 15:47:49.386969 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 22 15:47:49.386969 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application Aug 22 15:47:49.386969 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 380/750] 10.222.0.15 () {56 vars in 1093 bytes} [Mon Aug 22 15:47:49 2022] GET /identity/v3/users/nova => generated 82 bytes in 256 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 15:47:49.386969 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e3321f9-4b16-4a55-a226-674139feb67a None admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 15:47:49.399536 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG dbcounter [-] [81767] Writing DB stats keystone:SELECT=100 {{(pid=81767) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 15:47:49.481372 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84c85b2f-ab4a-4c72-9e30-fe920b15e7a9 None admin] Validating token access rules against request {{(pid=81767) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 15:47:49.522619 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d1b6da6-ebf0-407b-a920-3f27a3cf4efe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81767) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 15:47:49.523452 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d1b6da6-ebf0-407b-a920-3f27a3cf4efe None admin] REQUEST_METHOD: `GET` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:47:49.523818 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d1b6da6-ebf0-407b-a920-3f27a3cf4efe None admin] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:47:49.524096 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d1b6da6-ebf0-407b-a920-3f27a3cf4efe None admin] PATH_INFO: `/v3/projects` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:47:49.524852 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d1b6da6-ebf0-407b-a920-3f27a3cf4efe None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81767) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 15:47:49.525160 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d1b6da6-ebf0-407b-a920-3f27a3cf4efe None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 15:47:49.525390 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e3321f9-4b16-4a55-a226-674139feb67a None admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 15:47:49.527303 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d1b6da6-ebf0-407b-a920-3f27a3cf4efe None admin] RBAC: Authorization granted {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 15:47:49.543560 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 371/751] 10.222.0.15 () {56 vars in 1148 bytes} [Mon Aug 22 15:47:49 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 15:47:49.544594 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d1b6da6-ebf0-407b-a920-3f27a3cf4efe None admin] Authenticating user token {{(pid=81767) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 15:47:49.546129 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=100 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 15:47:49.575696 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6e365d6-6b06-49b9-a061-d0abee0fab72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 15:47:49.576772 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e365d6-6b06-49b9-a061-d0abee0fab72 None admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:47:49.577249 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e365d6-6b06-49b9-a061-d0abee0fab72 None admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:47:49.577720 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e365d6-6b06-49b9-a061-d0abee0fab72 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:47:49.586549 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6e365d6-6b06-49b9-a061-d0abee0fab72 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 15:47:49.589015 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6e365d6-6b06-49b9-a061-d0abee0fab72 None admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 15:47:49.602669 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application [None req-d6e365d6-6b06-49b9-a061-d0abee0fab72 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 22 15:47:49.602669 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 15:47:49.602669 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 15:47:49.602669 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 15:47:49.602669 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 15:47:49.602669 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 15:47:49.602669 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 15:47:49.602669 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 15:47:49.602669 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 15:47:49.602669 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 15:47:49.602669 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 15:47:49.602669 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 15:47:49.602669 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 22 15:47:49.602669 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 22 15:47:49.602669 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 22 15:47:49.602669 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 22 15:47:49.603715 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 15:47:49.603715 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 15:47:49.603715 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 22 15:47:49.603715 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 22 15:47:49.603715 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 22 15:47:49.603715 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 22 15:47:49.603715 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 15:47:49.603715 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 15:47:49.603715 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 15:47:49.603715 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 15:47:49.603715 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 15:47:49.603715 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application with Lock( Aug 22 15:47:49.603715 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 15:47:49.603715 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self._enter() Aug 22 15:47:49.603715 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 15:47:49.603715 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 15:47:49.603715 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 15:47:49.603715 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self.creator() Aug 22 15:47:49.604631 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 15:47:49.604631 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application created_value = creator( Aug 22 15:47:49.604631 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 22 15:47:49.604631 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 22 15:47:49.604631 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 22 15:47:49.604631 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 22 15:47:49.604631 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 22 15:47:49.604631 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 22 15:47:49.604631 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 22 15:47:49.604631 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application Aug 22 15:47:49.611855 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 381/752] 10.222.0.15 () {56 vars in 1117 bytes} [Mon Aug 22 15:47:49 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 15:47:49.613085 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6e365d6-6b06-49b9-a061-d0abee0fab72 None admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 15:47:49.659159 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG dbcounter [-] [81767] Writing DB stats keystone:SELECT=100 {{(pid=81767) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 15:47:49.663537 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d1b6da6-ebf0-407b-a920-3f27a3cf4efe None admin] Validating token access rules against request {{(pid=81767) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 15:47:49.701118 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b0de5df-c9f9-4f57-aaad-e93619070e20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81767) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 15:47:49.701481 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b0de5df-c9f9-4f57-aaad-e93619070e20 None admin] REQUEST_METHOD: `GET` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:47:49.701729 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b0de5df-c9f9-4f57-aaad-e93619070e20 None admin] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:47:49.702001 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b0de5df-c9f9-4f57-aaad-e93619070e20 None admin] PATH_INFO: `/v3/users` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:47:49.702755 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b0de5df-c9f9-4f57-aaad-e93619070e20 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81767) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 15:47:49.703054 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b0de5df-c9f9-4f57-aaad-e93619070e20 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 15:47:49.704527 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b0de5df-c9f9-4f57-aaad-e93619070e20 None admin] RBAC: Authorization granted {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 15:47:49.730811 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 372/753] 10.222.0.15 () {56 vars in 1133 bytes} [Mon Aug 22 15:47:49 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 15:47:49.730811 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b0de5df-c9f9-4f57-aaad-e93619070e20 None admin] Authenticating user token {{(pid=81767) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 15:47:49.738055 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6e365d6-6b06-49b9-a061-d0abee0fab72 None admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 15:47:49.782381 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-842926a4-7eaf-4fc2-b728-94c0727658e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 15:47:49.783427 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-842926a4-7eaf-4fc2-b728-94c0727658e9 None admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:47:49.783900 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-842926a4-7eaf-4fc2-b728-94c0727658e9 None admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:47:49.784398 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-842926a4-7eaf-4fc2-b728-94c0727658e9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:47:49.785336 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-842926a4-7eaf-4fc2-b728-94c0727658e9 None admin] RBAC: Adding query filter params (role.id=9d9848cfd22e4b089db443c350e56670, scope.project.id=c4ee0413f9ef4d4fb1f95a660babe090, user.id=6a09d128dee9438283633e51f4834643) {{(pid=81768) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 15:47:49.785863 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-842926a4-7eaf-4fc2-b728-94c0727658e9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 15:47:49.787582 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-842926a4-7eaf-4fc2-b728-94c0727658e9 None admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 15:47:49.794501 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 382/754] 10.222.0.15 () {56 vars in 1374 bytes} [Mon Aug 22 15:47:49 2022] GET /identity/v3/role_assignments?user.id=6a09d128dee9438283633e51f4834643&scope.project.id=c4ee0413f9ef4d4fb1f95a660babe090&role.id=9d9848cfd22e4b089db443c350e56670 => generated 259 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 15:47:49.795806 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-842926a4-7eaf-4fc2-b728-94c0727658e9 None admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 15:47:49.839219 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=100 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 15:47:49.855908 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b0de5df-c9f9-4f57-aaad-e93619070e20 None admin] Validating token access rules against request {{(pid=81767) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 15:47:49.917208 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eba6dad0-cef3-4f9e-9f5f-c7754f3d254e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81767) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 15:47:49.917975 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eba6dad0-cef3-4f9e-9f5f-c7754f3d254e None admin] REQUEST_METHOD: `GET` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:47:49.923312 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eba6dad0-cef3-4f9e-9f5f-c7754f3d254e None admin] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:47:49.923312 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eba6dad0-cef3-4f9e-9f5f-c7754f3d254e None admin] PATH_INFO: `/v3/users` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:47:49.923312 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eba6dad0-cef3-4f9e-9f5f-c7754f3d254e None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81767) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 15:47:49.923312 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eba6dad0-cef3-4f9e-9f5f-c7754f3d254e None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 15:47:49.926141 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eba6dad0-cef3-4f9e-9f5f-c7754f3d254e None admin] RBAC: Authorization granted {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 15:47:49.974982 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-842926a4-7eaf-4fc2-b728-94c0727658e9 None admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 15:47:50.021297 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-888b03d2-5234-4428-94fd-6dab96035952 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 15:47:50.021949 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-888b03d2-5234-4428-94fd-6dab96035952 None admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:47:50.022229 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-888b03d2-5234-4428-94fd-6dab96035952 None admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:47:50.022505 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-888b03d2-5234-4428-94fd-6dab96035952 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:47:50.035745 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-888b03d2-5234-4428-94fd-6dab96035952 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 15:47:50.037272 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-888b03d2-5234-4428-94fd-6dab96035952 None admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 15:47:50.055532 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application [None req-888b03d2-5234-4428-94fd-6dab96035952 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 22 15:47:50.055532 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 15:47:50.055532 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 15:47:50.055532 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 15:47:50.055532 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 15:47:50.055532 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 15:47:50.055532 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 15:47:50.055532 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 15:47:50.055532 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 15:47:50.055532 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 15:47:50.055532 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 15:47:50.055532 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 15:47:50.055532 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 22 15:47:50.055532 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 22 15:47:50.055532 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 22 15:47:50.055532 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 22 15:47:50.056728 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 15:47:50.056728 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 15:47:50.056728 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 15:47:50.056728 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 15:47:50.056728 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 15:47:50.056728 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 15:47:50.056728 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 15:47:50.056728 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application with Lock( Aug 22 15:47:50.056728 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 15:47:50.056728 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self._enter() Aug 22 15:47:50.056728 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 15:47:50.056728 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 15:47:50.056728 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 15:47:50.056728 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self.creator() Aug 22 15:47:50.056728 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 15:47:50.056728 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application created_value = creator( Aug 22 15:47:50.056728 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 22 15:47:50.058233 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 22 15:47:50.058233 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 22 15:47:50.058233 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application Aug 22 15:47:50.058233 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 383/755] 10.222.0.15 () {56 vars in 1108 bytes} [Mon Aug 22 15:47:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 262 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 15:47:50.059310 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4877445-2982-4b79-92fc-ceb7df1d5e8f None None] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:47:50.059505 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4877445-2982-4b79-92fc-ceb7df1d5e8f None None] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:47:50.060960 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4877445-2982-4b79-92fc-ceb7df1d5e8f None None] PATH_INFO: `/` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:47:50.061187 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 384/756] 10.222.0.15 () {54 vars in 927 bytes} [Mon Aug 22 15:47:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 15:47:50.067096 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29724acd-d57a-43b6-9840-d7ed90f18c02 None None] REQUEST_METHOD: `POST` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:47:50.067785 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29724acd-d57a-43b6-9840-d7ed90f18c02 None None] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:47:50.068222 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29724acd-d57a-43b6-9840-d7ed90f18c02 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:47:50.089074 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.auth.core [None req-29724acd-d57a-43b6-9840-d7ed90f18c02 None None] MFA Rules not processed for user `6b2199ae0fb0480aa917ada391f5ec54`. Rule list: `[]` (Enabled: `True`). {{(pid=81768) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 15:47:50.090570 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-29724acd-d57a-43b6-9840-d7ed90f18c02 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 15:47:50.145149 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 373/757] 10.222.0.15 () {56 vars in 1149 bytes} [Mon Aug 22 15:47:49 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 413 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 15:47:50.146917 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eba6dad0-cef3-4f9e-9f5f-c7754f3d254e None admin] Authenticating user token {{(pid=81767) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 15:47:50.170911 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG dbcounter [-] [81767] Writing DB stats keystone:SELECT=100 {{(pid=81767) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 15:47:50.179303 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 385/758] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Aug 22 15:47:50 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 22 15:47:50.180516 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29724acd-d57a-43b6-9840-d7ed90f18c02 None None] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 15:47:50.221710 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=100 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 15:47:50.341430 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29724acd-d57a-43b6-9840-d7ed90f18c02 None None] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 15:47:50.371673 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eba6dad0-cef3-4f9e-9f5f-c7754f3d254e None admin] Validating token access rules against request {{(pid=81767) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 15:47:50.400358 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19709e98-164f-4d57-892d-0d190074d96b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 15:47:50.403929 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19709e98-164f-4d57-892d-0d190074d96b None admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:47:50.404255 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19709e98-164f-4d57-892d-0d190074d96b None admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:47:50.404496 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19709e98-164f-4d57-892d-0d190074d96b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:47:50.439491 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-670d9c0f-334a-4586-a9a3-a3a253fc646d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81767) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 15:47:50.443656 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-670d9c0f-334a-4586-a9a3-a3a253fc646d None admin] REQUEST_METHOD: `GET` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:47:50.444412 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-670d9c0f-334a-4586-a9a3-a3a253fc646d None admin] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:47:50.445415 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-670d9c0f-334a-4586-a9a3-a3a253fc646d None admin] PATH_INFO: `/v3/domains` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:47:50.447891 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-670d9c0f-334a-4586-a9a3-a3a253fc646d None admin] RBAC: Adding query filter params (name=Default) {{(pid=81767) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 15:47:50.448962 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-670d9c0f-334a-4586-a9a3-a3a253fc646d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 15:47:50.452738 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-670d9c0f-334a-4586-a9a3-a3a253fc646d None admin] RBAC: Authorization granted {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 15:47:50.477020 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 374/759] 10.222.0.15 () {56 vars in 1109 bytes} [Mon Aug 22 15:47:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 331 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 15:47:50.483563 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-670d9c0f-334a-4586-a9a3-a3a253fc646d None admin] Authenticating user token {{(pid=81767) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 15:47:50.587224 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19709e98-164f-4d57-892d-0d190074d96b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 15:47:50.590747 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19709e98-164f-4d57-892d-0d190074d96b None admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 15:47:50.630048 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=100 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 15:47:50.649658 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG dbcounter [-] [81767] Writing DB stats keystone:SELECT=100 {{(pid=81767) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 15:47:50.700933 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-670d9c0f-334a-4586-a9a3-a3a253fc646d None admin] Validating token access rules against request {{(pid=81767) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 15:47:50.735440 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 386/760] 10.222.0.15 () {58 vars in 1282 bytes} [Mon Aug 22 15:47:50 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 550 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 22 15:47:50.736853 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19709e98-164f-4d57-892d-0d190074d96b None admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 15:47:50.752002 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-382c331d-4bc5-4d06-b489-3ebe0b2ab172 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81767) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 15:47:50.753045 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-382c331d-4bc5-4d06-b489-3ebe0b2ab172 None admin] REQUEST_METHOD: `GET` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:47:50.753529 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-382c331d-4bc5-4d06-b489-3ebe0b2ab172 None admin] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:47:50.753999 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-382c331d-4bc5-4d06-b489-3ebe0b2ab172 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:47:50.763056 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-382c331d-4bc5-4d06-b489-3ebe0b2ab172 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 15:47:50.765117 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-382c331d-4bc5-4d06-b489-3ebe0b2ab172 None admin] RBAC: Authorization granted {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 15:47:50.779102 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application [None req-382c331d-4bc5-4d06-b489-3ebe0b2ab172 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 22 15:47:50.779102 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 15:47:50.779102 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 15:47:50.779102 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 15:47:50.779102 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 15:47:50.779102 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 15:47:50.779102 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 15:47:50.779102 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 15:47:50.779102 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 15:47:50.779102 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 15:47:50.779102 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 15:47:50.779102 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 15:47:50.779102 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 22 15:47:50.779102 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 22 15:47:50.779102 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 22 15:47:50.780096 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 22 15:47:50.780096 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 15:47:50.780096 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 15:47:50.780096 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 15:47:50.780096 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 15:47:50.780096 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 15:47:50.780096 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 15:47:50.780096 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 15:47:50.780096 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application with Lock( Aug 22 15:47:50.780096 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 15:47:50.780096 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return self._enter() Aug 22 15:47:50.780096 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 15:47:50.780096 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 15:47:50.780096 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 15:47:50.780096 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return self.creator() Aug 22 15:47:50.780096 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 15:47:50.780096 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application created_value = creator( Aug 22 15:47:50.780989 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 22 15:47:50.780989 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 22 15:47:50.780989 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 22 15:47:50.780989 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 22 15:47:50.780989 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 22 15:47:50.780989 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 22 15:47:50.780989 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 22 15:47:50.780989 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application Aug 22 15:47:50.783607 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 375/761] 10.222.0.15 () {56 vars in 1111 bytes} [Mon Aug 22 15:47:50 2022] GET /identity/v3/projects/service => generated 88 bytes in 301 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 15:47:50.784775 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-382c331d-4bc5-4d06-b489-3ebe0b2ab172 None admin] Authenticating user token {{(pid=81767) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 15:47:50.886434 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19709e98-164f-4d57-892d-0d190074d96b None admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 15:47:50.902326 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-382c331d-4bc5-4d06-b489-3ebe0b2ab172 None admin] Validating token access rules against request {{(pid=81767) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 15:47:50.907984 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=100 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 15:47:50.932333 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29f90232-6bd4-4321-9f07-a54b9b5de2e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 15:47:50.933162 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29f90232-6bd4-4321-9f07-a54b9b5de2e2 None admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:47:50.933521 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29f90232-6bd4-4321-9f07-a54b9b5de2e2 None admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:47:50.933891 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29f90232-6bd4-4321-9f07-a54b9b5de2e2 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:47:50.934485 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29f90232-6bd4-4321-9f07-a54b9b5de2e2 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 15:47:50.936570 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29f90232-6bd4-4321-9f07-a54b9b5de2e2 None admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 15:47:50.947809 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application [None req-29f90232-6bd4-4321-9f07-a54b9b5de2e2 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 22 15:47:50.947809 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 15:47:50.947809 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 22 15:47:50.947809 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 22 15:47:50.947809 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 22 15:47:50.947809 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 22 15:47:50.947809 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 22 15:47:50.947809 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 22 15:47:50.947809 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 22 15:47:50.947809 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application Aug 22 15:47:50.947809 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 22 15:47:50.947809 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application Aug 22 15:47:50.947809 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 15:47:50.947809 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 15:47:50.947809 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 15:47:50.947809 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 15:47:50.947809 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 15:47:50.948879 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 15:47:50.948879 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 15:47:50.948879 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 15:47:50.948879 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 15:47:50.948879 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 15:47:50.948879 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 15:47:50.948879 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 22 15:47:50.948879 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 22 15:47:50.948879 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 22 15:47:50.948879 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 22 15:47:50.948879 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 15:47:50.948879 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 15:47:50.948879 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 15:47:50.948879 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 15:47:50.948879 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 15:47:50.948879 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 15:47:50.949712 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 15:47:50.949712 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application with Lock( Aug 22 15:47:50.949712 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 15:47:50.949712 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self._enter() Aug 22 15:47:50.949712 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 15:47:50.949712 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 15:47:50.949712 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 15:47:50.949712 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self.creator() Aug 22 15:47:50.949712 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 15:47:50.949712 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application created_value = creator( Aug 22 15:47:50.949712 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 22 15:47:50.949712 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 22 15:47:50.949712 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 22 15:47:50.949712 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application Aug 22 15:47:50.958485 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 387/762] 10.222.0.15 () {56 vars in 1129 bytes} [Mon Aug 22 15:47:50 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 15:47:50.958485 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29f90232-6bd4-4321-9f07-a54b9b5de2e2 None admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 15:47:50.960758 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef4b1175-3b57-49ce-bdbc-4fb991593780 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81767) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 15:47:50.961859 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef4b1175-3b57-49ce-bdbc-4fb991593780 None admin] REQUEST_METHOD: `GET` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:47:50.962297 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef4b1175-3b57-49ce-bdbc-4fb991593780 None admin] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:47:50.962771 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef4b1175-3b57-49ce-bdbc-4fb991593780 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:47:50.966458 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG dbcounter [-] [81767] Writing DB stats keystone:SELECT=100 {{(pid=81767) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 15:47:50.979886 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef4b1175-3b57-49ce-bdbc-4fb991593780 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 15:47:50.981843 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef4b1175-3b57-49ce-bdbc-4fb991593780 None admin] RBAC: Authorization granted {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 15:47:51.000213 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application [None req-ef4b1175-3b57-49ce-bdbc-4fb991593780 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 22 15:47:51.000213 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 15:47:51.000213 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 15:47:51.000213 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 15:47:51.000213 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 15:47:51.000213 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 15:47:51.000213 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 15:47:51.000213 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 15:47:51.000213 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 15:47:51.000213 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 15:47:51.000213 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 15:47:51.000213 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 15:47:51.000213 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 22 15:47:51.000213 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 22 15:47:51.000213 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 22 15:47:51.000213 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 22 15:47:51.001322 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 15:47:51.001322 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 15:47:51.001322 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 15:47:51.001322 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 15:47:51.001322 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 15:47:51.001322 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 15:47:51.001322 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 15:47:51.001322 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application with Lock( Aug 22 15:47:51.001322 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 15:47:51.001322 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return self._enter() Aug 22 15:47:51.001322 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 15:47:51.001322 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 15:47:51.001322 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 15:47:51.001322 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return self.creator() Aug 22 15:47:51.001322 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 15:47:51.001322 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application created_value = creator( Aug 22 15:47:51.001322 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 22 15:47:51.002202 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 22 15:47:51.002202 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 22 15:47:51.002202 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application Aug 22 15:47:51.004389 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 376/763] 10.222.0.15 () {56 vars in 1108 bytes} [Mon Aug 22 15:47:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 15:47:51.005829 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef4b1175-3b57-49ce-bdbc-4fb991593780 None admin] Authenticating user token {{(pid=81767) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 15:47:51.086073 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29f90232-6bd4-4321-9f07-a54b9b5de2e2 None admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 15:47:51.136595 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a70448b-5c74-43ec-b8ba-7b04c501e701 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 15:47:51.137499 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a70448b-5c74-43ec-b8ba-7b04c501e701 None admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:47:51.138026 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a70448b-5c74-43ec-b8ba-7b04c501e701 None admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:47:51.138371 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a70448b-5c74-43ec-b8ba-7b04c501e701 None admin] PATH_INFO: `/v3/projects` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:47:51.139320 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a70448b-5c74-43ec-b8ba-7b04c501e701 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81768) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 15:47:51.139705 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a70448b-5c74-43ec-b8ba-7b04c501e701 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 15:47:51.139812 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef4b1175-3b57-49ce-bdbc-4fb991593780 None admin] Validating token access rules against request {{(pid=81767) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 15:47:51.141258 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a70448b-5c74-43ec-b8ba-7b04c501e701 None admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 15:47:51.161239 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 388/764] 10.222.0.15 () {56 vars in 1148 bytes} [Mon Aug 22 15:47:50 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 15:47:51.162693 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a70448b-5c74-43ec-b8ba-7b04c501e701 None admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 15:47:51.188160 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68983f16-05ea-4ad3-ae35-3f011a9cf1b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81767) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 15:47:51.189122 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68983f16-05ea-4ad3-ae35-3f011a9cf1b2 None admin] REQUEST_METHOD: `GET` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:47:51.189611 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68983f16-05ea-4ad3-ae35-3f011a9cf1b2 None admin] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:47:51.190062 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68983f16-05ea-4ad3-ae35-3f011a9cf1b2 None admin] PATH_INFO: `/v3/services` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:47:51.190947 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68983f16-05ea-4ad3-ae35-3f011a9cf1b2 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=81767) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 15:47:51.191575 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68983f16-05ea-4ad3-ae35-3f011a9cf1b2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 15:47:51.193215 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68983f16-05ea-4ad3-ae35-3f011a9cf1b2 None admin] RBAC: Authorization granted {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 15:47:51.200174 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 377/765] 10.222.0.15 () {56 vars in 1124 bytes} [Mon Aug 22 15:47:51 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 15:47:51.206203 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68983f16-05ea-4ad3-ae35-3f011a9cf1b2 None admin] Authenticating user token {{(pid=81767) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 15:47:51.216636 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=100 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 15:47:51.267116 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG dbcounter [-] [81767] Writing DB stats keystone:SELECT=100 {{(pid=81767) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 15:47:51.281491 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a70448b-5c74-43ec-b8ba-7b04c501e701 None admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 15:47:51.329957 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68983f16-05ea-4ad3-ae35-3f011a9cf1b2 None admin] Validating token access rules against request {{(pid=81767) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 15:47:51.346578 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-687f41d1-8505-430e-9f71-9cf92db65837 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 15:47:51.347345 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-687f41d1-8505-430e-9f71-9cf92db65837 None admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:47:51.347603 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-687f41d1-8505-430e-9f71-9cf92db65837 None admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:47:51.348122 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-687f41d1-8505-430e-9f71-9cf92db65837 None admin] PATH_INFO: `/v3/domains` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:47:51.348606 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-687f41d1-8505-430e-9f71-9cf92db65837 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81768) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 15:47:51.348996 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-687f41d1-8505-430e-9f71-9cf92db65837 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 15:47:51.350477 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-687f41d1-8505-430e-9f71-9cf92db65837 None admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 15:47:51.370919 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 389/766] 10.222.0.15 () {56 vars in 1109 bytes} [Mon Aug 22 15:47:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 15:47:51.378790 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94c9e12a-30d6-4464-b502-6fd2a0420a2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81767) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 15:47:51.378991 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-687f41d1-8505-430e-9f71-9cf92db65837 None admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 15:47:51.380325 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94c9e12a-30d6-4464-b502-6fd2a0420a2d None admin] REQUEST_METHOD: `PUT` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:47:51.380833 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94c9e12a-30d6-4464-b502-6fd2a0420a2d None admin] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:47:51.381508 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94c9e12a-30d6-4464-b502-6fd2a0420a2d None admin] PATH_INFO: `/v3/projects/c4ee0413f9ef4d4fb1f95a660babe090/users/e4ef569e41fb46d0b721bb1c7663616f/roles/9d9848cfd22e4b089db443c350e56670` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:47:51.384171 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94c9e12a-30d6-4464-b502-6fd2a0420a2d None admin] RBAC: Authorizing `identity:create_grant(project_id=c4ee0413f9ef4d4fb1f95a660babe090, user_id=e4ef569e41fb46d0b721bb1c7663616f, role_id=9d9848cfd22e4b089db443c350e56670)` {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 15:47:51.386444 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94c9e12a-30d6-4464-b502-6fd2a0420a2d None admin] RBAC: Authorization granted {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 15:47:51.490522 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-687f41d1-8505-430e-9f71-9cf92db65837 None admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 15:47:51.492311 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=100 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 15:47:51.514096 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 378/767] 10.222.0.15 () {58 vars in 1439 bytes} [Mon Aug 22 15:47:51 2022] PUT /identity/v3/projects/c4ee0413f9ef4d4fb1f95a660babe090/users/e4ef569e41fb46d0b721bb1c7663616f/roles/9d9848cfd22e4b089db443c350e56670 => generated 0 bytes in 304 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 15:47:51.515975 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94c9e12a-30d6-4464-b502-6fd2a0420a2d None admin] Authenticating user token {{(pid=81767) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 15:47:51.582100 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11738f52-2c73-491f-9aff-56c72a2c5ff8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 15:47:51.585727 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11738f52-2c73-491f-9aff-56c72a2c5ff8 None admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:47:51.586076 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11738f52-2c73-491f-9aff-56c72a2c5ff8 None admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:47:51.586514 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11738f52-2c73-491f-9aff-56c72a2c5ff8 None admin] PATH_INFO: `/v3/services` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:47:51.587618 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11738f52-2c73-491f-9aff-56c72a2c5ff8 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=81768) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 15:47:51.593618 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11738f52-2c73-491f-9aff-56c72a2c5ff8 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 15:47:51.596544 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11738f52-2c73-491f-9aff-56c72a2c5ff8 None admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 15:47:51.622735 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 390/768] 10.222.0.15 () {56 vars in 1124 bytes} [Mon Aug 22 15:47:51 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 251 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 15:47:51.632784 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11738f52-2c73-491f-9aff-56c72a2c5ff8 None admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 15:47:51.772313 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94c9e12a-30d6-4464-b502-6fd2a0420a2d None admin] Validating token access rules against request {{(pid=81767) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 15:47:51.795450 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG dbcounter [-] [81767] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81767) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 15:47:51.823462 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11738f52-2c73-491f-9aff-56c72a2c5ff8 None admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 15:47:51.830433 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3333db3-6bb5-4298-99cd-56415a15f917 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81767) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 15:47:51.831081 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3333db3-6bb5-4298-99cd-56415a15f917 None admin] REQUEST_METHOD: `GET` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:47:51.831387 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3333db3-6bb5-4298-99cd-56415a15f917 None admin] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:47:51.831657 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3333db3-6bb5-4298-99cd-56415a15f917 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:47:51.841573 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3333db3-6bb5-4298-99cd-56415a15f917 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 15:47:51.843164 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3333db3-6bb5-4298-99cd-56415a15f917 None admin] RBAC: Authorization granted {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 15:47:51.857533 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application [None req-b3333db3-6bb5-4298-99cd-56415a15f917 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 22 15:47:51.857533 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 15:47:51.857533 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 15:47:51.857533 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 15:47:51.857533 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 15:47:51.857533 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 15:47:51.857533 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 15:47:51.857533 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 15:47:51.857533 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 15:47:51.857533 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 15:47:51.857533 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 15:47:51.857533 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 15:47:51.857533 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 22 15:47:51.857533 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 22 15:47:51.857533 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 22 15:47:51.858445 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 22 15:47:51.858445 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 15:47:51.858445 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 15:47:51.858445 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 15:47:51.858445 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 15:47:51.858445 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 15:47:51.858445 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 15:47:51.858445 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 15:47:51.858445 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application with Lock( Aug 22 15:47:51.858445 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 15:47:51.858445 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return self._enter() Aug 22 15:47:51.858445 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 15:47:51.858445 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 15:47:51.858445 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 15:47:51.858445 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return self.creator() Aug 22 15:47:51.858445 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 15:47:51.858445 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application created_value = creator( Aug 22 15:47:51.859354 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 22 15:47:51.859354 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 22 15:47:51.859354 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 22 15:47:51.859354 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 22 15:47:51.859354 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 22 15:47:51.859354 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 22 15:47:51.859354 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 22 15:47:51.859354 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application Aug 22 15:47:51.859354 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 379/769] 10.222.0.15 () {56 vars in 1111 bytes} [Mon Aug 22 15:47:51 2022] GET /identity/v3/projects/service => generated 88 bytes in 344 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 15:47:51.860764 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-164f425f-8572-4dbb-86e4-86c49b40e6db None None] REQUEST_METHOD: `GET` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:47:51.860907 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-164f425f-8572-4dbb-86e4-86c49b40e6db None None] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:47:51.863861 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-164f425f-8572-4dbb-86e4-86c49b40e6db None None] PATH_INFO: `/` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:47:51.863861 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 380/770] 10.222.0.15 () {54 vars in 927 bytes} [Mon Aug 22 15:47:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 15:47:51.868474 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b12362a1-428d-4657-a05b-84c5cb16d88b None None] REQUEST_METHOD: `POST` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:47:51.868746 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b12362a1-428d-4657-a05b-84c5cb16d88b None None] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:47:51.869022 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b12362a1-428d-4657-a05b-84c5cb16d88b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:47:51.890561 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.auth.core [None req-b12362a1-428d-4657-a05b-84c5cb16d88b None None] MFA Rules not processed for user `6b2199ae0fb0480aa917ada391f5ec54`. Rule list: `[]` (Enabled: `True`). {{(pid=81767) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 15:47:51.891922 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.fernet_utils [None req-b12362a1-428d-4657-a05b-84c5cb16d88b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81767) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 15:47:51.898267 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8983b687-0b1d-4641-a225-6b870edf70b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 15:47:51.899079 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8983b687-0b1d-4641-a225-6b870edf70b9 None admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:47:51.899403 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8983b687-0b1d-4641-a225-6b870edf70b9 None admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:47:51.899680 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8983b687-0b1d-4641-a225-6b870edf70b9 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:47:51.900955 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8983b687-0b1d-4641-a225-6b870edf70b9 None admin] RBAC: Adding query filter params (interface=public, service_id=ba6060a956bf4901a4e271a9832567cd, region_id=RegionOne) {{(pid=81768) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 15:47:51.901265 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8983b687-0b1d-4641-a225-6b870edf70b9 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 15:47:51.902668 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8983b687-0b1d-4641-a225-6b870edf70b9 None admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 15:47:51.911266 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 391/771] 10.222.0.15 () {56 vars in 1251 bytes} [Mon Aug 22 15:47:51 2022] GET /identity/v3/endpoints?service_id=ba6060a956bf4901a4e271a9832567cd&interface=public®ion_id=RegionOne => generated 194 bytes in 279 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 15:47:51.912183 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8983b687-0b1d-4641-a225-6b870edf70b9 None admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 15:47:51.946925 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=100 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 15:47:51.983972 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 381/772] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Aug 22 15:47:51 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 22 15:47:51.988788 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b12362a1-428d-4657-a05b-84c5cb16d88b None None] Authenticating user token {{(pid=81767) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 15:47:52.039364 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8983b687-0b1d-4641-a225-6b870edf70b9 None admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 15:47:52.088374 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1afc748b-28fe-4bc8-a09f-ae404861fed9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 15:47:52.089595 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1afc748b-28fe-4bc8-a09f-ae404861fed9 None admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:47:52.090079 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1afc748b-28fe-4bc8-a09f-ae404861fed9 None admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:47:52.090830 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1afc748b-28fe-4bc8-a09f-ae404861fed9 None admin] PATH_INFO: `/v3/projects` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:47:52.092237 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1afc748b-28fe-4bc8-a09f-ae404861fed9 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81768) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 15:47:52.093028 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1afc748b-28fe-4bc8-a09f-ae404861fed9 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 15:47:52.095021 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1afc748b-28fe-4bc8-a09f-ae404861fed9 None admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 15:47:52.113001 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG dbcounter [-] [81767] Writing DB stats keystone:SELECT=100 {{(pid=81767) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 15:47:52.114262 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 392/773] 10.222.0.15 () {56 vars in 1148 bytes} [Mon Aug 22 15:47:51 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 15:47:52.115821 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42cdc445-3b88-4d4a-a159-68b8a881f1be None None] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:47:52.116211 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42cdc445-3b88-4d4a-a159-68b8a881f1be None None] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:47:52.116558 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42cdc445-3b88-4d4a-a159-68b8a881f1be None None] PATH_INFO: `/` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:47:52.118045 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b12362a1-428d-4657-a05b-84c5cb16d88b None None] Validating token access rules against request {{(pid=81767) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 15:47:52.118934 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 393/774] 10.222.0.15 () {54 vars in 927 bytes} [Mon Aug 22 15:47:52 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 15:47:52.120009 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42cdc445-3b88-4d4a-a159-68b8a881f1be None None] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 15:47:52.173704 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae5079b6-27b9-4f51-8c60-4b4813d46458 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81767) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 15:47:52.174634 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae5079b6-27b9-4f51-8c60-4b4813d46458 None admin] REQUEST_METHOD: `GET` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:47:52.175157 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae5079b6-27b9-4f51-8c60-4b4813d46458 None admin] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:47:52.175711 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae5079b6-27b9-4f51-8c60-4b4813d46458 None admin] PATH_INFO: `/v3/services` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:47:52.176756 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae5079b6-27b9-4f51-8c60-4b4813d46458 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 15:47:52.178709 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae5079b6-27b9-4f51-8c60-4b4813d46458 None admin] RBAC: Authorization granted {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 15:47:52.195857 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 382/775] 10.222.0.15 () {56 vars in 1087 bytes} [Mon Aug 22 15:47:51 2022] GET /identity/v3/services => generated 772 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 15:47:52.195857 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ddc6752-d8b9-48ac-b2c2-fa37e0299056 None None] REQUEST_METHOD: `POST` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:47:52.195857 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ddc6752-d8b9-48ac-b2c2-fa37e0299056 None None] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:47:52.195857 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ddc6752-d8b9-48ac-b2c2-fa37e0299056 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:47:52.249204 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.auth.core [None req-2ddc6752-d8b9-48ac-b2c2-fa37e0299056 None None] MFA Rules not processed for user `6b2199ae0fb0480aa917ada391f5ec54`. Rule list: `[]` (Enabled: `True`). {{(pid=81767) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 15:47:52.250571 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.fernet_utils [None req-2ddc6752-d8b9-48ac-b2c2-fa37e0299056 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81767) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 15:47:52.311299 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=100 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 15:47:52.324607 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42cdc445-3b88-4d4a-a159-68b8a881f1be None None] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 15:47:52.373743 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 383/776] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Aug 22 15:47:52 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 22 15:47:52.374972 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ddc6752-d8b9-48ac-b2c2-fa37e0299056 None None] Authenticating user token {{(pid=81767) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 15:47:52.386789 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02a090f5-814c-49bf-9ec0-6a16ab0fada4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 15:47:52.388064 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02a090f5-814c-49bf-9ec0-6a16ab0fada4 None admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:47:52.388624 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02a090f5-814c-49bf-9ec0-6a16ab0fada4 None admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:47:52.389099 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02a090f5-814c-49bf-9ec0-6a16ab0fada4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:47:52.390045 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02a090f5-814c-49bf-9ec0-6a16ab0fada4 None admin] RBAC: Adding query filter params (role.id=a5fbba1dc615490a96cde27e3a102eb6, scope.project.id=c4ee0413f9ef4d4fb1f95a660babe090, user.id=058e76354dce412cb78b2875c88f2791) {{(pid=81768) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 15:47:52.390587 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02a090f5-814c-49bf-9ec0-6a16ab0fada4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 15:47:52.392444 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02a090f5-814c-49bf-9ec0-6a16ab0fada4 None admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 15:47:52.403757 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 394/777] 10.222.0.15 () {56 vars in 1374 bytes} [Mon Aug 22 15:47:52 2022] GET /identity/v3/role_assignments?user.id=058e76354dce412cb78b2875c88f2791&scope.project.id=c4ee0413f9ef4d4fb1f95a660babe090&role.id=a5fbba1dc615490a96cde27e3a102eb6 => generated 259 bytes in 284 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 15:47:52.491950 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG dbcounter [-] [81767] Writing DB stats keystone:SELECT=100 {{(pid=81767) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 15:47:52.500548 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ddc6752-d8b9-48ac-b2c2-fa37e0299056 None None] Validating token access rules against request {{(pid=81767) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 15:47:52.543548 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0689eec-3e2c-4004-8dbc-55a990a33368 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81767) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 15:47:52.544383 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0689eec-3e2c-4004-8dbc-55a990a33368 None admin] REQUEST_METHOD: `GET` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:47:52.544652 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0689eec-3e2c-4004-8dbc-55a990a33368 None admin] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:47:52.544921 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0689eec-3e2c-4004-8dbc-55a990a33368 None admin] PATH_INFO: `/v3/services/network` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:47:52.545890 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0689eec-3e2c-4004-8dbc-55a990a33368 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 15:47:52.548039 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0689eec-3e2c-4004-8dbc-55a990a33368 None admin] RBAC: Authorization granted {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 15:47:52.561594 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application [None req-f0689eec-3e2c-4004-8dbc-55a990a33368 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Aug 22 15:47:52.561594 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 15:47:52.561594 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 22 15:47:52.561594 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 22 15:47:52.561594 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 22 15:47:52.561594 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 22 15:47:52.561594 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 22 15:47:52.561594 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 22 15:47:52.561594 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 22 15:47:52.561594 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application Aug 22 15:47:52.561594 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 22 15:47:52.561594 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application Aug 22 15:47:52.561594 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 15:47:52.561594 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 15:47:52.561594 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 15:47:52.561594 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 15:47:52.561594 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 15:47:52.562619 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 15:47:52.562619 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 15:47:52.562619 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 15:47:52.562619 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 15:47:52.562619 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 15:47:52.562619 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 15:47:52.562619 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 22 15:47:52.562619 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 22 15:47:52.562619 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 22 15:47:52.562619 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 22 15:47:52.562619 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 15:47:52.562619 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 15:47:52.562619 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 15:47:52.562619 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 15:47:52.562619 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 15:47:52.562619 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 15:47:52.563944 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 15:47:52.563944 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application with Lock( Aug 22 15:47:52.563944 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 15:47:52.563944 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return self._enter() Aug 22 15:47:52.563944 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 15:47:52.563944 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 15:47:52.563944 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 15:47:52.563944 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return self.creator() Aug 22 15:47:52.563944 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 15:47:52.563944 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application created_value = creator( Aug 22 15:47:52.563944 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 22 15:47:52.563944 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 22 15:47:52.563944 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 22 15:47:52.563944 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application Aug 22 15:47:52.563944 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 384/778] 10.222.0.15 () {56 vars in 1111 bytes} [Mon Aug 22 15:47:52 2022] GET /identity/v3/services/network => generated 88 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 15:47:52.569999 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02a090f5-814c-49bf-9ec0-6a16ab0fada4 None admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 15:47:52.681251 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02a090f5-814c-49bf-9ec0-6a16ab0fada4 None admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 15:47:52.689714 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0689eec-3e2c-4004-8dbc-55a990a33368 None admin] Authenticating user token {{(pid=81767) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 15:47:52.723234 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-390650c3-b1c2-4cf2-a709-f5639a6efbc4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 15:47:52.723879 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-390650c3-b1c2-4cf2-a709-f5639a6efbc4 None admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:47:52.724154 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-390650c3-b1c2-4cf2-a709-f5639a6efbc4 None admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:47:52.724420 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-390650c3-b1c2-4cf2-a709-f5639a6efbc4 None admin] PATH_INFO: `/v3/services` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:47:52.725050 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-390650c3-b1c2-4cf2-a709-f5639a6efbc4 None admin] RBAC: Adding query filter params (name=network) {{(pid=81768) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 15:47:52.725349 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-390650c3-b1c2-4cf2-a709-f5639a6efbc4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 15:47:52.726930 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-390650c3-b1c2-4cf2-a709-f5639a6efbc4 None admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 15:47:52.733473 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 395/779] 10.222.0.15 () {56 vars in 1112 bytes} [Mon Aug 22 15:47:52 2022] GET /identity/v3/services?name=network => generated 124 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 15:47:52.740093 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-390650c3-b1c2-4cf2-a709-f5639a6efbc4 None admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 15:47:52.767677 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=100 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 15:47:52.891787 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-390650c3-b1c2-4cf2-a709-f5639a6efbc4 None admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 15:47:52.952693 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0689eec-3e2c-4004-8dbc-55a990a33368 None admin] Validating token access rules against request {{(pid=81767) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 15:47:52.960695 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5fa9966-a810-4540-a9e6-5d538390aa1c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 15:47:52.961324 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5fa9966-a810-4540-a9e6-5d538390aa1c None admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:47:52.961602 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5fa9966-a810-4540-a9e6-5d538390aa1c None admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:47:52.961874 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5fa9966-a810-4540-a9e6-5d538390aa1c None admin] PATH_INFO: `/v3/services` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:47:52.962530 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5fa9966-a810-4540-a9e6-5d538390aa1c None admin] RBAC: Adding query filter params (type=network) {{(pid=81768) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 15:47:52.963352 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5fa9966-a810-4540-a9e6-5d538390aa1c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 15:47:52.965094 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5fa9966-a810-4540-a9e6-5d538390aa1c None admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 15:47:52.972258 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 396/780] 10.222.0.15 () {56 vars in 1112 bytes} [Mon Aug 22 15:47:52 2022] GET /identity/v3/services?type=network => generated 350 bytes in 233 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 15:47:52.977752 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5fa9966-a810-4540-a9e6-5d538390aa1c None admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 15:47:53.015749 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4208ffc-9fd8-4e92-b8e2-496820370ea5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81767) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 15:47:53.016657 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4208ffc-9fd8-4e92-b8e2-496820370ea5 None admin] REQUEST_METHOD: `GET` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:47:53.017489 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4208ffc-9fd8-4e92-b8e2-496820370ea5 None admin] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:47:53.017489 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4208ffc-9fd8-4e92-b8e2-496820370ea5 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:47:53.041026 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4208ffc-9fd8-4e92-b8e2-496820370ea5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 22 15:47:53.047460 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application [None req-a4208ffc-9fd8-4e92-b8e2-496820370ea5 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 22 15:47:53.047460 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 15:47:53.047460 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 15:47:53.047460 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 15:47:53.047460 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 15:47:53.047460 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 15:47:53.047460 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 15:47:53.047460 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 15:47:53.047460 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 15:47:53.047460 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 15:47:53.047460 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 15:47:53.047460 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 15:47:53.047460 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 22 15:47:53.047460 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 22 15:47:53.047460 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 22 15:47:53.047460 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 22 15:47:53.048479 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 22 15:47:53.048479 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 22 15:47:53.048479 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 22 15:47:53.048479 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 22 15:47:53.048479 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 15:47:53.048479 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 15:47:53.048479 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 15:47:53.048479 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 15:47:53.048479 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 15:47:53.048479 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 15:47:53.048479 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 15:47:53.048479 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application with Lock( Aug 22 15:47:53.048479 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 15:47:53.048479 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return self._enter() Aug 22 15:47:53.048479 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 15:47:53.048479 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 15:47:53.049682 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 15:47:53.049682 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return self.creator() Aug 22 15:47:53.049682 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 15:47:53.049682 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application created_value = creator( Aug 22 15:47:53.049682 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 22 15:47:53.049682 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 22 15:47:53.049682 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 22 15:47:53.049682 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 22 15:47:53.049682 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 22 15:47:53.049682 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 22 15:47:53.049682 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 22 15:47:53.049682 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application Aug 22 15:47:53.049682 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 385/781] 10.222.0.15 () {56 vars in 1096 bytes} [Mon Aug 22 15:47:52 2022] GET /identity/v3/roles/admin => generated 83 bytes in 360 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 22 15:47:53.060459 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4208ffc-9fd8-4e92-b8e2-496820370ea5 None admin] Authenticating user token {{(pid=81767) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 15:47:53.093593 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=100 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 15:47:53.101949 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG dbcounter [-] [81767] Writing DB stats keystone:SELECT=100 {{(pid=81767) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 15:47:53.117915 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5fa9966-a810-4540-a9e6-5d538390aa1c None admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 15:47:53.159076 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d49ffe4-725c-4839-ad10-14b70c714ac7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 15:47:53.160008 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d49ffe4-725c-4839-ad10-14b70c714ac7 None admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:47:53.160522 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d49ffe4-725c-4839-ad10-14b70c714ac7 None admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:47:53.161054 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d49ffe4-725c-4839-ad10-14b70c714ac7 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:47:53.162066 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d49ffe4-725c-4839-ad10-14b70c714ac7 None admin] RBAC: Adding query filter params (interface=public, service_id=f411ca51dbe34c72b2b992b64981a196, region_id=RegionOne) {{(pid=81768) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 15:47:53.162558 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d49ffe4-725c-4839-ad10-14b70c714ac7 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 15:47:53.165063 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d49ffe4-725c-4839-ad10-14b70c714ac7 None admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 15:47:53.171887 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 397/782] 10.222.0.15 () {56 vars in 1251 bytes} [Mon Aug 22 15:47:52 2022] GET /identity/v3/endpoints?service_id=f411ca51dbe34c72b2b992b64981a196&interface=public®ion_id=RegionOne => generated 194 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 22 15:47:53.177405 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d49ffe4-725c-4839-ad10-14b70c714ac7 None admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 15:47:53.294896 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d49ffe4-725c-4839-ad10-14b70c714ac7 None admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 15:47:53.342422 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbf23019-9931-4233-98f8-2ffe184c5dfc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 15:47:53.343579 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbf23019-9931-4233-98f8-2ffe184c5dfc None admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:47:53.344228 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbf23019-9931-4233-98f8-2ffe184c5dfc None admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:47:53.345426 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbf23019-9931-4233-98f8-2ffe184c5dfc None admin] PATH_INFO: `/v3/services` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:47:53.346777 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbf23019-9931-4233-98f8-2ffe184c5dfc None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 15:47:53.348369 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4208ffc-9fd8-4e92-b8e2-496820370ea5 None admin] Validating token access rules against request {{(pid=81767) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 15:47:53.349346 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbf23019-9931-4233-98f8-2ffe184c5dfc None admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 15:47:53.357323 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 398/783] 10.222.0.15 () {56 vars in 1087 bytes} [Mon Aug 22 15:47:53 2022] GET /identity/v3/services => generated 772 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 15:47:53.415941 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79d340a4-8c67-4faa-8c50-5c14d88d6bc4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81767) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 15:47:53.416935 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79d340a4-8c67-4faa-8c50-5c14d88d6bc4 None admin] REQUEST_METHOD: `GET` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:47:53.417449 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79d340a4-8c67-4faa-8c50-5c14d88d6bc4 None admin] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:47:53.417924 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79d340a4-8c67-4faa-8c50-5c14d88d6bc4 None admin] PATH_INFO: `/v3/roles` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:47:53.418881 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79d340a4-8c67-4faa-8c50-5c14d88d6bc4 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81767) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 15:47:53.419481 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79d340a4-8c67-4faa-8c50-5c14d88d6bc4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 15:47:53.421086 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79d340a4-8c67-4faa-8c50-5c14d88d6bc4 None admin] RBAC: Authorization granted {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 15:47:53.437271 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 386/784] 10.222.0.15 () {56 vars in 1099 bytes} [Mon Aug 22 15:47:53 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 377 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 15:47:53.443657 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbf23019-9931-4233-98f8-2ffe184c5dfc None admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 15:47:53.460177 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=100 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 15:47:53.597329 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbf23019-9931-4233-98f8-2ffe184c5dfc None admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 15:47:53.647693 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3334042-5292-4f97-91ba-56b9ae7c2591 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 15:47:53.648517 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3334042-5292-4f97-91ba-56b9ae7c2591 None admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:47:53.648942 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3334042-5292-4f97-91ba-56b9ae7c2591 None admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:47:53.649359 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3334042-5292-4f97-91ba-56b9ae7c2591 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:47:53.665535 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3334042-5292-4f97-91ba-56b9ae7c2591 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 15:47:53.668567 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3334042-5292-4f97-91ba-56b9ae7c2591 None admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 15:47:53.703227 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application [None req-e3334042-5292-4f97-91ba-56b9ae7c2591 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 22 15:47:53.703227 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 15:47:53.703227 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 15:47:53.703227 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 15:47:53.703227 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 15:47:53.703227 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 15:47:53.703227 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 15:47:53.703227 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 15:47:53.703227 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 15:47:53.703227 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 15:47:53.703227 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 15:47:53.703227 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 15:47:53.703227 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 22 15:47:53.703227 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 22 15:47:53.703227 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 22 15:47:53.703227 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 22 15:47:53.704382 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 15:47:53.704382 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 15:47:53.704382 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 15:47:53.704382 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 15:47:53.704382 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 15:47:53.704382 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 15:47:53.704382 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 15:47:53.704382 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application with Lock( Aug 22 15:47:53.704382 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 15:47:53.704382 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self._enter() Aug 22 15:47:53.704382 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 15:47:53.704382 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 15:47:53.704382 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 15:47:53.704382 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self.creator() Aug 22 15:47:53.704382 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 15:47:53.704382 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application created_value = creator( Aug 22 15:47:53.704382 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 22 15:47:53.705285 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 22 15:47:53.705285 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 22 15:47:53.705285 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application Aug 22 15:47:53.715724 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 399/785] 10.222.0.15 () {56 vars in 1108 bytes} [Mon Aug 22 15:47:53 2022] GET /identity/v3/domains/Default => generated 87 bytes in 273 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 15:47:53.718114 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79d340a4-8c67-4faa-8c50-5c14d88d6bc4 None admin] Authenticating user token {{(pid=81767) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 15:47:53.856244 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG dbcounter [-] [81767] Writing DB stats keystone:SELECT=100 {{(pid=81767) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 15:47:53.880904 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79d340a4-8c67-4faa-8c50-5c14d88d6bc4 None admin] Validating token access rules against request {{(pid=81767) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 15:47:53.941524 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10df3643-ffe0-4823-a154-67e2ef9fb1f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81767) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 15:47:53.941995 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10df3643-ffe0-4823-a154-67e2ef9fb1f5 None admin] REQUEST_METHOD: `GET` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:47:53.942271 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10df3643-ffe0-4823-a154-67e2ef9fb1f5 None admin] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:47:53.942538 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10df3643-ffe0-4823-a154-67e2ef9fb1f5 None admin] PATH_INFO: `/v3/domains` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:47:53.943198 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10df3643-ffe0-4823-a154-67e2ef9fb1f5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81767) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 15:47:53.943521 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10df3643-ffe0-4823-a154-67e2ef9fb1f5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 15:47:53.944886 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10df3643-ffe0-4823-a154-67e2ef9fb1f5 None admin] RBAC: Authorization granted {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 15:47:53.963553 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 387/786] 10.222.0.15 () {56 vars in 1109 bytes} [Mon Aug 22 15:47:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 246 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 15:47:53.966753 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c748d4f-25ac-45d1-aa12-8ba1bc3d6b22 None None] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:47:53.967192 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c748d4f-25ac-45d1-aa12-8ba1bc3d6b22 None None] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:47:53.967656 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c748d4f-25ac-45d1-aa12-8ba1bc3d6b22 None None] PATH_INFO: `/` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:47:53.969306 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10df3643-ffe0-4823-a154-67e2ef9fb1f5 None admin] Authenticating user token {{(pid=81767) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 15:47:53.971125 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 400/787] 10.222.0.15 () {54 vars in 927 bytes} [Mon Aug 22 15:47:53 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 15:47:53.980862 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-730766d9-37da-4276-b9df-deea68519f28 None None] REQUEST_METHOD: `POST` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:47:53.981301 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-730766d9-37da-4276-b9df-deea68519f28 None None] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:47:53.981732 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-730766d9-37da-4276-b9df-deea68519f28 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:47:54.028875 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.auth.core [None req-730766d9-37da-4276-b9df-deea68519f28 None None] MFA Rules not processed for user `6b2199ae0fb0480aa917ada391f5ec54`. Rule list: `[]` (Enabled: `True`). {{(pid=81768) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 15:47:54.030914 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-730766d9-37da-4276-b9df-deea68519f28 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 15:47:54.159583 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=100 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 15:47:54.181455 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 401/788] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Aug 22 15:47:53 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 203 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 22 15:47:54.184483 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10df3643-ffe0-4823-a154-67e2ef9fb1f5 None admin] Validating token access rules against request {{(pid=81767) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 15:47:54.239831 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db7c75e3-dc7d-4d39-b844-1d98882d6eb6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81767) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 15:47:54.240831 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db7c75e3-dc7d-4d39-b844-1d98882d6eb6 None admin] REQUEST_METHOD: `GET` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:47:54.241317 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db7c75e3-dc7d-4d39-b844-1d98882d6eb6 None admin] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:47:54.241812 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db7c75e3-dc7d-4d39-b844-1d98882d6eb6 None admin] PATH_INFO: `/v3/users/swift` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:47:54.260836 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db7c75e3-dc7d-4d39-b844-1d98882d6eb6 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 15:47:54.262669 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db7c75e3-dc7d-4d39-b844-1d98882d6eb6 None admin] RBAC: Authorization granted {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 15:47:54.266050 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG dbcounter [-] [81767] Writing DB stats keystone:SELECT=100 {{(pid=81767) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 15:47:54.278799 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application [None req-db7c75e3-dc7d-4d39-b844-1d98882d6eb6 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 22 15:47:54.278799 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 15:47:54.278799 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 15:47:54.278799 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 15:47:54.278799 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 15:47:54.278799 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 15:47:54.278799 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 15:47:54.278799 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 15:47:54.278799 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 15:47:54.278799 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 15:47:54.278799 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 15:47:54.278799 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 15:47:54.278799 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 22 15:47:54.278799 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 22 15:47:54.278799 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 22 15:47:54.278799 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 22 15:47:54.284676 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 15:47:54.284676 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 15:47:54.284676 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 22 15:47:54.284676 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 22 15:47:54.284676 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 22 15:47:54.284676 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 22 15:47:54.284676 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 15:47:54.284676 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 15:47:54.284676 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 15:47:54.284676 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 15:47:54.284676 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 15:47:54.284676 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application with Lock( Aug 22 15:47:54.284676 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 15:47:54.284676 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return self._enter() Aug 22 15:47:54.284676 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 15:47:54.284676 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 15:47:54.284676 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 15:47:54.284676 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return self.creator() Aug 22 15:47:54.285579 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 15:47:54.285579 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application created_value = creator( Aug 22 15:47:54.285579 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 22 15:47:54.285579 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 22 15:47:54.285579 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 22 15:47:54.285579 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 22 15:47:54.285579 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 22 15:47:54.285579 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 22 15:47:54.285579 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 22 15:47:54.285579 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application Aug 22 15:47:54.285579 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 388/789] 10.222.0.15 () {56 vars in 1096 bytes} [Mon Aug 22 15:47:53 2022] GET /identity/v3/users/swift => generated 83 bytes in 312 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 22 15:47:54.288787 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-730766d9-37da-4276-b9df-deea68519f28 None None] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 15:47:54.449232 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-730766d9-37da-4276-b9df-deea68519f28 None None] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 15:47:54.474849 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db7c75e3-dc7d-4d39-b844-1d98882d6eb6 None admin] Authenticating user token {{(pid=81767) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 15:47:54.527199 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21a2bd55-ec94-4f1f-8082-86bd1dbb702f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 15:47:54.527930 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21a2bd55-ec94-4f1f-8082-86bd1dbb702f None admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:47:54.528393 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21a2bd55-ec94-4f1f-8082-86bd1dbb702f None admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:47:54.528822 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21a2bd55-ec94-4f1f-8082-86bd1dbb702f None admin] PATH_INFO: `/v3/users` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:47:54.531458 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21a2bd55-ec94-4f1f-8082-86bd1dbb702f None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81768) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 15:47:54.531932 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21a2bd55-ec94-4f1f-8082-86bd1dbb702f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 15:47:54.534134 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21a2bd55-ec94-4f1f-8082-86bd1dbb702f None admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 15:47:54.591014 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 402/790] 10.222.0.15 () {56 vars in 1135 bytes} [Mon Aug 22 15:47:54 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 302 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 15:47:54.597276 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21a2bd55-ec94-4f1f-8082-86bd1dbb702f None admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 15:47:54.753854 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db7c75e3-dc7d-4d39-b844-1d98882d6eb6 None admin] Validating token access rules against request {{(pid=81767) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 15:47:54.780836 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=100 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 15:47:54.849672 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78a83a30-5c43-45b6-8b59-b4d7f4689f3e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81767) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 15:47:54.850341 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78a83a30-5c43-45b6-8b59-b4d7f4689f3e None admin] REQUEST_METHOD: `GET` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:47:54.850622 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78a83a30-5c43-45b6-8b59-b4d7f4689f3e None admin] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:47:54.850896 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78a83a30-5c43-45b6-8b59-b4d7f4689f3e None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:47:54.877451 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78a83a30-5c43-45b6-8b59-b4d7f4689f3e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 22 15:47:54.882689 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application [None req-78a83a30-5c43-45b6-8b59-b4d7f4689f3e None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 22 15:47:54.882689 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 15:47:54.882689 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 15:47:54.882689 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 15:47:54.882689 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 15:47:54.882689 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 15:47:54.882689 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 15:47:54.882689 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 15:47:54.882689 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 15:47:54.882689 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 15:47:54.882689 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 15:47:54.882689 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 15:47:54.882689 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 22 15:47:54.882689 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 22 15:47:54.882689 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 22 15:47:54.882689 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 22 15:47:54.884588 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 22 15:47:54.884588 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 22 15:47:54.884588 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 22 15:47:54.884588 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 22 15:47:54.884588 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 15:47:54.884588 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 15:47:54.884588 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 15:47:54.884588 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 15:47:54.884588 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 15:47:54.884588 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 15:47:54.884588 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 15:47:54.884588 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application with Lock( Aug 22 15:47:54.884588 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 15:47:54.884588 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return self._enter() Aug 22 15:47:54.884588 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 15:47:54.884588 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 15:47:54.886752 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 15:47:54.886752 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return self.creator() Aug 22 15:47:54.886752 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 15:47:54.886752 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application created_value = creator( Aug 22 15:47:54.886752 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 22 15:47:54.886752 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 22 15:47:54.886752 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 22 15:47:54.886752 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 22 15:47:54.886752 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 22 15:47:54.886752 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 22 15:47:54.886752 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 22 15:47:54.886752 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application Aug 22 15:47:54.894992 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21a2bd55-ec94-4f1f-8082-86bd1dbb702f None admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 15:47:54.897050 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 389/791] 10.222.0.15 () {56 vars in 1096 bytes} [Mon Aug 22 15:47:54 2022] GET /identity/v3/roles/admin => generated 83 bytes in 423 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 15:47:54.907393 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78a83a30-5c43-45b6-8b59-b4d7f4689f3e None admin] Authenticating user token {{(pid=81767) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 15:47:54.974750 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c7ff7ce-a96b-42fe-999a-0bf8f1663411 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 15:47:54.975639 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c7ff7ce-a96b-42fe-999a-0bf8f1663411 None admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:47:54.975861 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c7ff7ce-a96b-42fe-999a-0bf8f1663411 None admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:47:54.976086 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c7ff7ce-a96b-42fe-999a-0bf8f1663411 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:47:54.997596 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c7ff7ce-a96b-42fe-999a-0bf8f1663411 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 15:47:54.999101 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c7ff7ce-a96b-42fe-999a-0bf8f1663411 None admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 15:47:55.026770 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application [None req-2c7ff7ce-a96b-42fe-999a-0bf8f1663411 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 22 15:47:55.026770 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 15:47:55.026770 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 15:47:55.026770 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 15:47:55.026770 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 15:47:55.026770 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 15:47:55.026770 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 15:47:55.026770 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 15:47:55.026770 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 15:47:55.026770 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 15:47:55.026770 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 15:47:55.026770 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 15:47:55.026770 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 22 15:47:55.026770 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 22 15:47:55.026770 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 22 15:47:55.026770 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 22 15:47:55.028877 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 15:47:55.028877 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 15:47:55.028877 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 15:47:55.028877 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 15:47:55.028877 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 15:47:55.028877 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 15:47:55.028877 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 15:47:55.028877 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application with Lock( Aug 22 15:47:55.028877 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 15:47:55.028877 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self._enter() Aug 22 15:47:55.028877 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 15:47:55.028877 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 15:47:55.028877 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 15:47:55.028877 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self.creator() Aug 22 15:47:55.028877 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 15:47:55.028877 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application created_value = creator( Aug 22 15:47:55.028877 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 22 15:47:55.030827 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 22 15:47:55.030827 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 22 15:47:55.030827 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application Aug 22 15:47:55.030827 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 403/792] 10.222.0.15 () {56 vars in 1108 bytes} [Mon Aug 22 15:47:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 432 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 15:47:55.039502 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG dbcounter [-] [81767] Writing DB stats keystone:SELECT=100 {{(pid=81767) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 15:47:55.041625 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c7ff7ce-a96b-42fe-999a-0bf8f1663411 None admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 15:47:55.131753 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78a83a30-5c43-45b6-8b59-b4d7f4689f3e None admin] Validating token access rules against request {{(pid=81767) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 15:47:55.184249 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-284e19bd-ec75-457c-a016-99964775e046 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81767) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 15:47:55.185502 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-284e19bd-ec75-457c-a016-99964775e046 None admin] REQUEST_METHOD: `GET` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:47:55.185982 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-284e19bd-ec75-457c-a016-99964775e046 None admin] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:47:55.186757 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-284e19bd-ec75-457c-a016-99964775e046 None admin] PATH_INFO: `/v3/roles` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:47:55.187974 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-284e19bd-ec75-457c-a016-99964775e046 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81767) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 15:47:55.188640 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-284e19bd-ec75-457c-a016-99964775e046 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 15:47:55.191130 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-284e19bd-ec75-457c-a016-99964775e046 None admin] RBAC: Authorization granted {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 15:47:55.199388 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c7ff7ce-a96b-42fe-999a-0bf8f1663411 None admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 15:47:55.201137 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=100 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 15:47:55.207958 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 390/793] 10.222.0.15 () {56 vars in 1099 bytes} [Mon Aug 22 15:47:54 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 301 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 15:47:55.209974 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c60c1ae5-2312-4802-8377-6f693134b24c None None] REQUEST_METHOD: `GET` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:47:55.210489 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c60c1ae5-2312-4802-8377-6f693134b24c None None] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:47:55.210914 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c60c1ae5-2312-4802-8377-6f693134b24c None None] PATH_INFO: `/` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:47:55.216750 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 391/794] 10.222.0.15 () {54 vars in 927 bytes} [Mon Aug 22 15:47:55 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 15:47:55.218447 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c60c1ae5-2312-4802-8377-6f693134b24c None None] Authenticating user token {{(pid=81767) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 15:47:55.241503 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b994cde-774d-4aa6-aceb-7f278713575d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 15:47:55.242160 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b994cde-774d-4aa6-aceb-7f278713575d None admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:47:55.242447 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b994cde-774d-4aa6-aceb-7f278713575d None admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:47:55.242719 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b994cde-774d-4aa6-aceb-7f278713575d None admin] PATH_INFO: `/v3/domains` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:47:55.243415 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b994cde-774d-4aa6-aceb-7f278713575d None admin] RBAC: Adding query filter params (name=Default) {{(pid=81768) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 15:47:55.243720 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b994cde-774d-4aa6-aceb-7f278713575d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 15:47:55.245788 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b994cde-774d-4aa6-aceb-7f278713575d None admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 15:47:55.259736 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 404/795] 10.222.0.15 () {56 vars in 1109 bytes} [Mon Aug 22 15:47:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 15:47:55.261602 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-841391f5-a3d1-4ec0-863d-d0c8a7a1e431 None None] REQUEST_METHOD: `POST` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:47:55.261891 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-841391f5-a3d1-4ec0-863d-d0c8a7a1e431 None None] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:47:55.262163 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-841391f5-a3d1-4ec0-863d-d0c8a7a1e431 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:47:55.281796 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.auth.core [None req-841391f5-a3d1-4ec0-863d-d0c8a7a1e431 None None] MFA Rules not processed for user `6b2199ae0fb0480aa917ada391f5ec54`. Rule list: `[]` (Enabled: `True`). {{(pid=81768) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 15:47:55.283026 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-841391f5-a3d1-4ec0-863d-d0c8a7a1e431 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 15:47:55.373901 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 405/796] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Aug 22 15:47:55 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 22 15:47:55.384214 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-841391f5-a3d1-4ec0-863d-d0c8a7a1e431 None None] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 15:47:55.394936 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c60c1ae5-2312-4802-8377-6f693134b24c None None] Validating token access rules against request {{(pid=81767) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 15:47:55.424648 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG dbcounter [-] [81767] Writing DB stats keystone:SELECT=100 {{(pid=81767) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 15:47:55.440524 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c62eb8f-b156-4fcd-87eb-7f2aded14b5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81767) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 15:47:55.441354 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c62eb8f-b156-4fcd-87eb-7f2aded14b5b None admin] REQUEST_METHOD: `GET` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:47:55.441725 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c62eb8f-b156-4fcd-87eb-7f2aded14b5b None admin] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:47:55.442478 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c62eb8f-b156-4fcd-87eb-7f2aded14b5b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:47:55.454782 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c62eb8f-b156-4fcd-87eb-7f2aded14b5b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 15:47:55.456408 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c62eb8f-b156-4fcd-87eb-7f2aded14b5b None admin] RBAC: Authorization granted {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 15:47:55.473197 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application [None req-0c62eb8f-b156-4fcd-87eb-7f2aded14b5b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 22 15:47:55.473197 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 15:47:55.473197 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 15:47:55.473197 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 15:47:55.473197 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 15:47:55.473197 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 15:47:55.473197 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 15:47:55.473197 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 15:47:55.473197 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 15:47:55.473197 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 15:47:55.473197 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 15:47:55.473197 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 15:47:55.473197 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 22 15:47:55.473197 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 22 15:47:55.473197 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 22 15:47:55.473197 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 22 15:47:55.474513 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 15:47:55.474513 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 15:47:55.474513 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 15:47:55.474513 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 15:47:55.474513 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 15:47:55.474513 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 15:47:55.474513 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 15:47:55.474513 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application with Lock( Aug 22 15:47:55.474513 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 15:47:55.474513 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return self._enter() Aug 22 15:47:55.474513 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 15:47:55.474513 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 15:47:55.474513 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 15:47:55.474513 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return self.creator() Aug 22 15:47:55.474513 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 15:47:55.474513 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application created_value = creator( Aug 22 15:47:55.474513 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 22 15:47:55.475589 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 22 15:47:55.475589 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 22 15:47:55.475589 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application Aug 22 15:47:55.482699 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 392/797] 10.222.0.15 () {56 vars in 1108 bytes} [Mon Aug 22 15:47:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 260 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 15:47:55.482699 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91795984-a966-4fc9-bf5d-a0ffea43c049 None None] REQUEST_METHOD: `GET` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:47:55.482699 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91795984-a966-4fc9-bf5d-a0ffea43c049 None None] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:47:55.482699 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91795984-a966-4fc9-bf5d-a0ffea43c049 None None] PATH_INFO: `/` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:47:55.485431 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 393/798] 10.222.0.15 () {54 vars in 927 bytes} [Mon Aug 22 15:47:55 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 15:47:55.486287 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91795984-a966-4fc9-bf5d-a0ffea43c049 None None] Authenticating user token {{(pid=81767) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 15:47:55.524757 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=100 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 15:47:55.537185 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-841391f5-a3d1-4ec0-863d-d0c8a7a1e431 None None] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 15:47:55.586932 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6cf2fc9-cf77-4dae-bb29-fee80fef31c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 15:47:55.588171 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6cf2fc9-cf77-4dae-bb29-fee80fef31c5 None admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:47:55.588662 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6cf2fc9-cf77-4dae-bb29-fee80fef31c5 None admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:47:55.589084 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6cf2fc9-cf77-4dae-bb29-fee80fef31c5 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:47:55.599780 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6cf2fc9-cf77-4dae-bb29-fee80fef31c5 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 15:47:55.602355 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6cf2fc9-cf77-4dae-bb29-fee80fef31c5 None admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 15:47:55.612246 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91795984-a966-4fc9-bf5d-a0ffea43c049 None None] Validating token access rules against request {{(pid=81767) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 15:47:55.618033 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application [None req-a6cf2fc9-cf77-4dae-bb29-fee80fef31c5 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 22 15:47:55.618033 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 15:47:55.618033 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 15:47:55.618033 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 15:47:55.618033 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 15:47:55.618033 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 15:47:55.618033 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 15:47:55.618033 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 15:47:55.618033 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 15:47:55.618033 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 15:47:55.618033 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 15:47:55.618033 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 15:47:55.618033 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 22 15:47:55.618033 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 22 15:47:55.618033 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 22 15:47:55.619747 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 22 15:47:55.619747 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 15:47:55.619747 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 15:47:55.619747 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 15:47:55.619747 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 15:47:55.619747 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 15:47:55.619747 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 15:47:55.619747 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 15:47:55.619747 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application with Lock( Aug 22 15:47:55.619747 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 15:47:55.619747 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self._enter() Aug 22 15:47:55.619747 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 15:47:55.619747 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 15:47:55.619747 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 15:47:55.619747 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self.creator() Aug 22 15:47:55.619747 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 15:47:55.619747 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application created_value = creator( Aug 22 15:47:55.620841 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 22 15:47:55.620841 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 22 15:47:55.620841 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 22 15:47:55.620841 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 22 15:47:55.620841 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 22 15:47:55.620841 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 22 15:47:55.620841 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 22 15:47:55.620841 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application Aug 22 15:47:55.622896 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 406/799] 10.222.0.15 () {56 vars in 1111 bytes} [Mon Aug 22 15:47:55 2022] GET /identity/v3/projects/service => generated 88 bytes in 248 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 15:47:55.624318 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-920c3550-d0b2-44f1-873c-b0c6f2fad064 None None] REQUEST_METHOD: `POST` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:47:55.624776 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-920c3550-d0b2-44f1-873c-b0c6f2fad064 None None] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:47:55.625208 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-920c3550-d0b2-44f1-873c-b0c6f2fad064 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:47:55.647611 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.auth.core [None req-920c3550-d0b2-44f1-873c-b0c6f2fad064 None None] MFA Rules not processed for user `6b2199ae0fb0480aa917ada391f5ec54`. Rule list: `[]` (Enabled: `True`). {{(pid=81768) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 15:47:55.649838 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-920c3550-d0b2-44f1-873c-b0c6f2fad064 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 15:47:55.670962 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a744955d-79ce-4370-86d1-8f187104e207 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81767) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 15:47:55.671604 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a744955d-79ce-4370-86d1-8f187104e207 None admin] REQUEST_METHOD: `GET` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:47:55.671950 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a744955d-79ce-4370-86d1-8f187104e207 None admin] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:47:55.672296 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a744955d-79ce-4370-86d1-8f187104e207 None admin] PATH_INFO: `/v3/domains` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:47:55.673214 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a744955d-79ce-4370-86d1-8f187104e207 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81767) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 15:47:55.673514 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a744955d-79ce-4370-86d1-8f187104e207 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 15:47:55.674906 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a744955d-79ce-4370-86d1-8f187104e207 None admin] RBAC: Authorization granted {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 15:47:55.692969 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 394/800] 10.222.0.15 () {56 vars in 1109 bytes} [Mon Aug 22 15:47:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 15:47:55.692969 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a744955d-79ce-4370-86d1-8f187104e207 None admin] Authenticating user token {{(pid=81767) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 15:47:55.741983 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG dbcounter [-] [81767] Writing DB stats keystone:SELECT=100 {{(pid=81767) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 15:47:55.751138 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 407/801] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Aug 22 15:47:55 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 22 15:47:55.752008 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-920c3550-d0b2-44f1-873c-b0c6f2fad064 None None] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 15:47:55.836707 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a744955d-79ce-4370-86d1-8f187104e207 None admin] Validating token access rules against request {{(pid=81767) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 15:47:55.849571 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=100 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 15:47:55.869650 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-920c3550-d0b2-44f1-873c-b0c6f2fad064 None None] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 15:47:55.876947 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-481a53c0-be5a-4548-8749-6c83820a14e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81767) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 15:47:55.877549 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-481a53c0-be5a-4548-8749-6c83820a14e3 None admin] REQUEST_METHOD: `GET` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:47:55.877846 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-481a53c0-be5a-4548-8749-6c83820a14e3 None admin] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:47:55.878122 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-481a53c0-be5a-4548-8749-6c83820a14e3 None admin] PATH_INFO: `/v3/projects` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:47:55.878803 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-481a53c0-be5a-4548-8749-6c83820a14e3 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81767) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 15:47:55.879100 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-481a53c0-be5a-4548-8749-6c83820a14e3 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 15:47:55.880569 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-481a53c0-be5a-4548-8749-6c83820a14e3 None admin] RBAC: Authorization granted {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 15:47:55.893053 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 395/802] 10.222.0.15 () {56 vars in 1148 bytes} [Mon Aug 22 15:47:55 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 15:47:55.893872 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-481a53c0-be5a-4548-8749-6c83820a14e3 None admin] Authenticating user token {{(pid=81767) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 15:47:55.912468 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52faa915-33dc-421f-9d39-e9204685ba8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 15:47:55.913361 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52faa915-33dc-421f-9d39-e9204685ba8d None admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:47:55.913813 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52faa915-33dc-421f-9d39-e9204685ba8d None admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:47:55.914269 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52faa915-33dc-421f-9d39-e9204685ba8d None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:47:55.915073 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52faa915-33dc-421f-9d39-e9204685ba8d None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 15:47:55.917250 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52faa915-33dc-421f-9d39-e9204685ba8d None admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 15:47:55.928238 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application [None req-52faa915-33dc-421f-9d39-e9204685ba8d None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 22 15:47:55.928238 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 15:47:55.928238 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 22 15:47:55.928238 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 22 15:47:55.928238 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 22 15:47:55.928238 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 22 15:47:55.928238 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 22 15:47:55.928238 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 22 15:47:55.928238 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 22 15:47:55.928238 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application Aug 22 15:47:55.928238 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 22 15:47:55.928238 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application Aug 22 15:47:55.928238 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 15:47:55.928238 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 15:47:55.928238 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 15:47:55.928238 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 15:47:55.928238 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 15:47:55.929397 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 15:47:55.929397 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 15:47:55.929397 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 15:47:55.929397 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 15:47:55.929397 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 15:47:55.929397 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 15:47:55.929397 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 22 15:47:55.929397 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 22 15:47:55.929397 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 22 15:47:55.929397 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 22 15:47:55.929397 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 15:47:55.929397 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 15:47:55.929397 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 15:47:55.929397 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 15:47:55.929397 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 15:47:55.929397 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 15:47:55.930298 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 15:47:55.930298 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application with Lock( Aug 22 15:47:55.930298 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 15:47:55.930298 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self._enter() Aug 22 15:47:55.930298 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 15:47:55.930298 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 15:47:55.930298 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 15:47:55.930298 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self.creator() Aug 22 15:47:55.930298 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 15:47:55.930298 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application created_value = creator( Aug 22 15:47:55.930298 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 22 15:47:55.930298 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 22 15:47:55.930298 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 22 15:47:55.930298 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application Aug 22 15:47:55.933034 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 408/803] 10.222.0.15 () {56 vars in 1129 bytes} [Mon Aug 22 15:47:55 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 15:47:55.934567 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19cafb21-4151-48b5-850b-93e069b8ca52 None None] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:47:55.934932 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19cafb21-4151-48b5-850b-93e069b8ca52 None None] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:47:55.935124 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19cafb21-4151-48b5-850b-93e069b8ca52 None None] PATH_INFO: `/` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:47:55.944287 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 409/804] 10.222.0.15 () {54 vars in 927 bytes} [Mon Aug 22 15:47:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 15:47:55.944287 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19cafb21-4151-48b5-850b-93e069b8ca52 None None] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 15:47:56.038020 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG dbcounter [-] [81767] Writing DB stats keystone:SELECT=100 {{(pid=81767) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 15:47:56.048111 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-481a53c0-be5a-4548-8749-6c83820a14e3 None admin] Validating token access rules against request {{(pid=81767) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 15:47:56.074745 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19cafb21-4151-48b5-850b-93e069b8ca52 None None] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 15:47:56.091197 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53b49890-2bd6-4f5f-a9b8-03142be8f66d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81767) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 15:47:56.091969 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53b49890-2bd6-4f5f-a9b8-03142be8f66d None admin] REQUEST_METHOD: `GET` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:47:56.092249 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53b49890-2bd6-4f5f-a9b8-03142be8f66d None admin] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:47:56.092559 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53b49890-2bd6-4f5f-a9b8-03142be8f66d None admin] PATH_INFO: `/v3/users/nova` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:47:56.100830 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53b49890-2bd6-4f5f-a9b8-03142be8f66d None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 15:47:56.102198 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53b49890-2bd6-4f5f-a9b8-03142be8f66d None admin] RBAC: Authorization granted {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 15:47:56.111961 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-939fec41-f306-46e6-b6db-a7ec53e039b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 15:47:56.112602 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-939fec41-f306-46e6-b6db-a7ec53e039b6 None admin] REQUEST_METHOD: `PUT` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:47:56.112874 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-939fec41-f306-46e6-b6db-a7ec53e039b6 None admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:47:56.113178 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-939fec41-f306-46e6-b6db-a7ec53e039b6 None admin] PATH_INFO: `/v3/projects/c4ee0413f9ef4d4fb1f95a660babe090/users/6a09d128dee9438283633e51f4834643/roles/9d9848cfd22e4b089db443c350e56670` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:47:56.113766 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application [None req-53b49890-2bd6-4f5f-a9b8-03142be8f66d None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 22 15:47:56.113766 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 15:47:56.113766 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 15:47:56.113766 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 15:47:56.113766 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 15:47:56.113766 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 15:47:56.113766 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 15:47:56.113766 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 15:47:56.113766 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 15:47:56.113766 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 15:47:56.113766 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 15:47:56.113766 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 15:47:56.113766 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 22 15:47:56.113766 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 22 15:47:56.113766 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 22 15:47:56.113766 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 22 15:47:56.114806 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 15:47:56.114806 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 15:47:56.114806 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 22 15:47:56.114806 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 22 15:47:56.114806 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 22 15:47:56.114806 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 22 15:47:56.114806 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 15:47:56.114806 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 15:47:56.114806 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 15:47:56.114806 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 15:47:56.114806 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 15:47:56.114806 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application with Lock( Aug 22 15:47:56.114806 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 15:47:56.114806 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return self._enter() Aug 22 15:47:56.114806 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 15:47:56.114806 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 15:47:56.114806 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 15:47:56.114806 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return self.creator() Aug 22 15:47:56.115820 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 15:47:56.115820 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application created_value = creator( Aug 22 15:47:56.115820 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 22 15:47:56.115820 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 22 15:47:56.115820 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 22 15:47:56.115820 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 22 15:47:56.115820 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 22 15:47:56.115820 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 22 15:47:56.115820 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 22 15:47:56.115820 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application Aug 22 15:47:56.116337 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-939fec41-f306-46e6-b6db-a7ec53e039b6 None admin] RBAC: Authorizing `identity:create_grant(project_id=c4ee0413f9ef4d4fb1f95a660babe090, user_id=6a09d128dee9438283633e51f4834643, role_id=9d9848cfd22e4b089db443c350e56670)` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 15:47:56.116407 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 396/805] 10.222.0.15 () {56 vars in 1093 bytes} [Mon Aug 22 15:47:55 2022] GET /identity/v3/users/nova => generated 82 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 15:47:56.116407 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53b49890-2bd6-4f5f-a9b8-03142be8f66d None admin] Authenticating user token {{(pid=81767) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 15:47:56.116905 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-939fec41-f306-46e6-b6db-a7ec53e039b6 None admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 15:47:56.222785 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53b49890-2bd6-4f5f-a9b8-03142be8f66d None admin] Validating token access rules against request {{(pid=81767) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 15:47:56.256210 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e29c4b3-2582-4b81-8639-2922cf301a86 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81767) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 15:47:56.256812 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e29c4b3-2582-4b81-8639-2922cf301a86 None admin] REQUEST_METHOD: `GET` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:47:56.257098 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e29c4b3-2582-4b81-8639-2922cf301a86 None admin] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:47:56.257366 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e29c4b3-2582-4b81-8639-2922cf301a86 None admin] PATH_INFO: `/v3/services` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:47:56.257998 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e29c4b3-2582-4b81-8639-2922cf301a86 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=81767) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 15:47:56.258300 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e29c4b3-2582-4b81-8639-2922cf301a86 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 15:47:56.260193 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e29c4b3-2582-4b81-8639-2922cf301a86 None admin] RBAC: Authorization granted {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 15:47:56.263663 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 410/806] 10.222.0.15 () {58 vars in 1439 bytes} [Mon Aug 22 15:47:55 2022] PUT /identity/v3/projects/c4ee0413f9ef4d4fb1f95a660babe090/users/6a09d128dee9438283633e51f4834643/roles/9d9848cfd22e4b089db443c350e56670 => generated 0 bytes in 327 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 15:47:56.270013 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c16b62d4-18fa-417f-aa34-efb07365b57e None None] REQUEST_METHOD: `POST` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:47:56.270165 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 397/807] 10.222.0.15 () {56 vars in 1124 bytes} [Mon Aug 22 15:47:56 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 15:47:56.270255 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c16b62d4-18fa-417f-aa34-efb07365b57e None None] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:47:56.270255 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c16b62d4-18fa-417f-aa34-efb07365b57e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:47:56.270411 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e29c4b3-2582-4b81-8639-2922cf301a86 None admin] Authenticating user token {{(pid=81767) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 15:47:56.281354 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 15:47:56.287500 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.auth.core [None req-c16b62d4-18fa-417f-aa34-efb07365b57e None None] MFA Rules not processed for user `6b2199ae0fb0480aa917ada391f5ec54`. Rule list: `[]` (Enabled: `True`). {{(pid=81768) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 15:47:56.288903 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-c16b62d4-18fa-417f-aa34-efb07365b57e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 15:47:56.296290 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG dbcounter [-] [81767] Writing DB stats keystone:SELECT=100 {{(pid=81767) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 15:47:56.392279 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 411/808] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Aug 22 15:47:56 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 22 15:47:56.393479 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c16b62d4-18fa-417f-aa34-efb07365b57e None None] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 15:47:56.406700 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e29c4b3-2582-4b81-8639-2922cf301a86 None admin] Validating token access rules against request {{(pid=81767) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 15:47:56.454746 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dc3ae02-ab0d-43c5-a1d3-e479c58976c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81767) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 15:47:56.455514 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc3ae02-ab0d-43c5-a1d3-e479c58976c4 None admin] REQUEST_METHOD: `GET` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:47:56.455772 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc3ae02-ab0d-43c5-a1d3-e479c58976c4 None admin] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:47:56.456091 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc3ae02-ab0d-43c5-a1d3-e479c58976c4 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:47:56.467803 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dc3ae02-ab0d-43c5-a1d3-e479c58976c4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 22 15:47:56.473177 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application [None req-6dc3ae02-ab0d-43c5-a1d3-e479c58976c4 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 22 15:47:56.473177 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 15:47:56.473177 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 15:47:56.473177 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 15:47:56.473177 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 15:47:56.473177 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 15:47:56.473177 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 15:47:56.473177 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 15:47:56.473177 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 15:47:56.473177 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 15:47:56.473177 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 15:47:56.473177 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 15:47:56.473177 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 22 15:47:56.473177 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 22 15:47:56.473177 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 22 15:47:56.473177 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 22 15:47:56.474255 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 22 15:47:56.474255 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 22 15:47:56.474255 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 22 15:47:56.474255 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 22 15:47:56.474255 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 15:47:56.474255 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 15:47:56.474255 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 15:47:56.474255 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 15:47:56.474255 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 15:47:56.474255 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 15:47:56.474255 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 15:47:56.474255 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application with Lock( Aug 22 15:47:56.474255 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 15:47:56.474255 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return self._enter() Aug 22 15:47:56.474255 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 15:47:56.474255 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 15:47:56.475822 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 15:47:56.475822 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return self.creator() Aug 22 15:47:56.475822 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 15:47:56.475822 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application created_value = creator( Aug 22 15:47:56.475822 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 22 15:47:56.475822 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 22 15:47:56.475822 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 22 15:47:56.475822 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 22 15:47:56.475822 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 22 15:47:56.475822 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 22 15:47:56.475822 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 22 15:47:56.475822 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application Aug 22 15:47:56.475822 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 398/809] 10.222.0.15 () {56 vars in 1102 bytes} [Mon Aug 22 15:47:56 2022] GET /identity/v3/roles/service => generated 85 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 15:47:56.475822 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dc3ae02-ab0d-43c5-a1d3-e479c58976c4 None admin] Authenticating user token {{(pid=81767) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 15:47:56.537773 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c16b62d4-18fa-417f-aa34-efb07365b57e None None] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 15:47:56.573987 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG dbcounter [-] [81767] Writing DB stats keystone:SELECT=100 {{(pid=81767) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 15:47:56.592980 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70032f04-e9bc-4776-9f64-7284d722d3b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 15:47:56.593999 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70032f04-e9bc-4776-9f64-7284d722d3b4 None admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:47:56.594782 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70032f04-e9bc-4776-9f64-7284d722d3b4 None admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:47:56.595241 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70032f04-e9bc-4776-9f64-7284d722d3b4 None admin] PATH_INFO: `/v3/users` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:47:56.596395 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70032f04-e9bc-4776-9f64-7284d722d3b4 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81768) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 15:47:56.596890 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70032f04-e9bc-4776-9f64-7284d722d3b4 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 15:47:56.599461 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70032f04-e9bc-4776-9f64-7284d722d3b4 None admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 15:47:56.600408 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dc3ae02-ab0d-43c5-a1d3-e479c58976c4 None admin] Validating token access rules against request {{(pid=81767) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 15:47:56.604184 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=100 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 15:47:56.629132 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 412/810] 10.222.0.15 () {56 vars in 1133 bytes} [Mon Aug 22 15:47:56 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 15:47:56.629132 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70032f04-e9bc-4776-9f64-7284d722d3b4 None admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 15:47:56.649903 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2f16152-58f2-4992-b999-f149806b41e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81767) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 15:47:56.649903 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2f16152-58f2-4992-b999-f149806b41e2 None admin] REQUEST_METHOD: `GET` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:47:56.650182 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2f16152-58f2-4992-b999-f149806b41e2 None admin] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:47:56.650429 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2f16152-58f2-4992-b999-f149806b41e2 None admin] PATH_INFO: `/v3/services` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:47:56.651059 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2f16152-58f2-4992-b999-f149806b41e2 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=81767) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 15:47:56.651386 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2f16152-58f2-4992-b999-f149806b41e2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 15:47:56.652771 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2f16152-58f2-4992-b999-f149806b41e2 None admin] RBAC: Authorization granted {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 15:47:56.664753 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 399/811] 10.222.0.15 () {56 vars in 1124 bytes} [Mon Aug 22 15:47:56 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 15:47:56.664753 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2f16152-58f2-4992-b999-f149806b41e2 None admin] Authenticating user token {{(pid=81767) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 15:47:56.751770 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70032f04-e9bc-4776-9f64-7284d722d3b4 None admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 15:47:56.785928 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2f16152-58f2-4992-b999-f149806b41e2 None admin] Validating token access rules against request {{(pid=81767) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 15:47:56.814412 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a490a21-2349-4246-aac9-525ea0970e98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 15:47:56.815547 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a490a21-2349-4246-aac9-525ea0970e98 None admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:47:56.816038 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a490a21-2349-4246-aac9-525ea0970e98 None admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:47:56.816522 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a490a21-2349-4246-aac9-525ea0970e98 None admin] PATH_INFO: `/v3/roles` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:47:56.817728 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a490a21-2349-4246-aac9-525ea0970e98 None admin] RBAC: Adding query filter params (name=service) {{(pid=81768) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 15:47:56.818262 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a490a21-2349-4246-aac9-525ea0970e98 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 15:47:56.820884 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a490a21-2349-4246-aac9-525ea0970e98 None admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 15:47:56.827935 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c88f18bc-d85e-4cde-9635-e08735eb1bf5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81767) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 15:47:56.828547 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c88f18bc-d85e-4cde-9635-e08735eb1bf5 None admin] REQUEST_METHOD: `GET` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:47:56.828824 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c88f18bc-d85e-4cde-9635-e08735eb1bf5 None admin] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:47:56.829091 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c88f18bc-d85e-4cde-9635-e08735eb1bf5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:47:56.840793 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 413/812] 10.222.0.15 () {56 vars in 1103 bytes} [Mon Aug 22 15:47:56 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 15:47:56.840793 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a490a21-2349-4246-aac9-525ea0970e98 None admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 15:47:56.847437 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG dbcounter [-] [81767] Writing DB stats keystone:SELECT=100 {{(pid=81767) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 15:47:56.921705 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=100 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 15:47:56.924678 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c88f18bc-d85e-4cde-9635-e08735eb1bf5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 15:47:56.926378 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c88f18bc-d85e-4cde-9635-e08735eb1bf5 None admin] RBAC: Authorization granted {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 15:47:56.994623 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a490a21-2349-4246-aac9-525ea0970e98 None admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 15:47:57.065245 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88a90237-a576-4a8e-b4c3-8e1dd5a37228 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 15:47:57.065871 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a90237-a576-4a8e-b4c3-8e1dd5a37228 None admin] REQUEST_METHOD: `POST` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:47:57.066367 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a90237-a576-4a8e-b4c3-8e1dd5a37228 None admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:47:57.066577 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a90237-a576-4a8e-b4c3-8e1dd5a37228 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:47:57.067249 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88a90237-a576-4a8e-b4c3-8e1dd5a37228 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 15:47:57.068681 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88a90237-a576-4a8e-b4c3-8e1dd5a37228 None admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 15:47:57.086871 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 400/813] 10.222.0.15 () {58 vars in 1282 bytes} [Mon Aug 22 15:47:56 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 426 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 22 15:47:57.087848 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c88f18bc-d85e-4cde-9635-e08735eb1bf5 None admin] Authenticating user token {{(pid=81767) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 15:47:57.139267 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG dbcounter [-] [81767] Writing DB stats keystone:SELECT=100 {{(pid=81767) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 15:47:57.201551 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c88f18bc-d85e-4cde-9635-e08735eb1bf5 None admin] Validating token access rules against request {{(pid=81767) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 15:47:57.238210 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-186b9ca8-c7f6-47ab-bcd3-722712c7dba9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81767) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 15:47:57.239030 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-186b9ca8-c7f6-47ab-bcd3-722712c7dba9 None admin] REQUEST_METHOD: `GET` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:47:57.239407 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-186b9ca8-c7f6-47ab-bcd3-722712c7dba9 None admin] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:47:57.239782 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-186b9ca8-c7f6-47ab-bcd3-722712c7dba9 None admin] PATH_INFO: `/v3/services/network` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:47:57.240479 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-186b9ca8-c7f6-47ab-bcd3-722712c7dba9 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 15:47:57.241948 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-186b9ca8-c7f6-47ab-bcd3-722712c7dba9 None admin] RBAC: Authorization granted {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 15:47:57.253376 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application [None req-186b9ca8-c7f6-47ab-bcd3-722712c7dba9 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Aug 22 15:47:57.253376 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 15:47:57.253376 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 22 15:47:57.253376 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 22 15:47:57.253376 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 22 15:47:57.253376 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 22 15:47:57.253376 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 22 15:47:57.253376 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 22 15:47:57.253376 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 22 15:47:57.253376 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application Aug 22 15:47:57.253376 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 22 15:47:57.253376 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application Aug 22 15:47:57.253376 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 15:47:57.253376 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 15:47:57.253376 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 15:47:57.253376 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 15:47:57.253376 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 15:47:57.254691 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 15:47:57.254691 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 15:47:57.254691 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 15:47:57.254691 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 15:47:57.254691 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 15:47:57.254691 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 15:47:57.254691 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 22 15:47:57.254691 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 22 15:47:57.254691 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 22 15:47:57.254691 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 22 15:47:57.254691 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 15:47:57.254691 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 15:47:57.254691 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 15:47:57.254691 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 15:47:57.254691 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 15:47:57.254691 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 15:47:57.256081 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 15:47:57.256081 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application with Lock( Aug 22 15:47:57.256081 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 15:47:57.256081 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return self._enter() Aug 22 15:47:57.256081 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 15:47:57.256081 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 15:47:57.256081 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 15:47:57.256081 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return self.creator() Aug 22 15:47:57.256081 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 15:47:57.256081 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application created_value = creator( Aug 22 15:47:57.256081 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 22 15:47:57.256081 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 22 15:47:57.256081 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 22 15:47:57.256081 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application Aug 22 15:47:57.257055 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 401/814] 10.222.0.15 () {56 vars in 1111 bytes} [Mon Aug 22 15:47:57 2022] GET /identity/v3/services/network => generated 88 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 15:47:57.257217 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-186b9ca8-c7f6-47ab-bcd3-722712c7dba9 None admin] Authenticating user token {{(pid=81767) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 15:47:57.272657 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 414/815] 10.222.0.15 () {60 vars in 1144 bytes} [Mon Aug 22 15:47:56 2022] POST /identity/v3/endpoints => generated 346 bytes in 435 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 22 15:47:57.274063 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88a90237-a576-4a8e-b4c3-8e1dd5a37228 None admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 15:47:57.403361 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-186b9ca8-c7f6-47ab-bcd3-722712c7dba9 None admin] Validating token access rules against request {{(pid=81767) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 15:47:57.412589 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG dbcounter [-] [81767] Writing DB stats keystone:SELECT=100 {{(pid=81767) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 15:47:57.449880 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88a90237-a576-4a8e-b4c3-8e1dd5a37228 None admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 15:47:57.462043 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d49d284c-9f64-4b8a-a679-c08280452205 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81767) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 15:47:57.462937 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d49d284c-9f64-4b8a-a679-c08280452205 None admin] REQUEST_METHOD: `GET` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:47:57.463229 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 15:47:57.463463 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d49d284c-9f64-4b8a-a679-c08280452205 None admin] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:47:57.463702 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d49d284c-9f64-4b8a-a679-c08280452205 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:47:57.475913 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d49d284c-9f64-4b8a-a679-c08280452205 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 15:47:57.477401 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d49d284c-9f64-4b8a-a679-c08280452205 None admin] RBAC: Authorization granted {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 15:47:57.496438 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application [None req-d49d284c-9f64-4b8a-a679-c08280452205 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 22 15:47:57.496438 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 15:47:57.496438 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 15:47:57.496438 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 15:47:57.496438 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 15:47:57.496438 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 15:47:57.496438 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 15:47:57.496438 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 15:47:57.496438 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 15:47:57.496438 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 15:47:57.496438 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 15:47:57.496438 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 15:47:57.496438 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 22 15:47:57.496438 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 22 15:47:57.496438 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 22 15:47:57.496438 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 22 15:47:57.497449 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 15:47:57.497449 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 15:47:57.497449 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 15:47:57.497449 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 15:47:57.497449 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 15:47:57.497449 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 15:47:57.497449 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 15:47:57.497449 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application with Lock( Aug 22 15:47:57.497449 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 15:47:57.497449 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return self._enter() Aug 22 15:47:57.497449 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 15:47:57.497449 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 15:47:57.497449 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 15:47:57.497449 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return self.creator() Aug 22 15:47:57.497449 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 15:47:57.497449 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application created_value = creator( Aug 22 15:47:57.497449 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 22 15:47:57.498830 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 22 15:47:57.498830 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 22 15:47:57.498830 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application Aug 22 15:47:57.498830 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 402/816] 10.222.0.15 () {56 vars in 1108 bytes} [Mon Aug 22 15:47:57 2022] GET /identity/v3/domains/Default => generated 87 bytes in 241 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 15:47:57.499051 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3ff1098-f5d6-47b3-9480-926a38006152 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 15:47:57.499155 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d49d284c-9f64-4b8a-a679-c08280452205 None admin] Authenticating user token {{(pid=81767) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 15:47:57.499227 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ff1098-f5d6-47b3-9480-926a38006152 None admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:47:57.499587 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ff1098-f5d6-47b3-9480-926a38006152 None admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:47:57.499920 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ff1098-f5d6-47b3-9480-926a38006152 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:47:57.634478 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3ff1098-f5d6-47b3-9480-926a38006152 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 15:47:57.636077 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3ff1098-f5d6-47b3-9480-926a38006152 None admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 15:47:57.646154 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d49d284c-9f64-4b8a-a679-c08280452205 None admin] Validating token access rules against request {{(pid=81767) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 15:47:57.651315 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application [None req-d3ff1098-f5d6-47b3-9480-926a38006152 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 22 15:47:57.651315 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 15:47:57.651315 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 15:47:57.651315 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 15:47:57.651315 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 15:47:57.651315 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 15:47:57.651315 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 15:47:57.651315 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 15:47:57.651315 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 15:47:57.651315 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 15:47:57.651315 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 15:47:57.651315 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 15:47:57.651315 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 22 15:47:57.651315 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 22 15:47:57.651315 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 22 15:47:57.651315 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 22 15:47:57.652238 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 15:47:57.652238 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 15:47:57.652238 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 15:47:57.652238 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 15:47:57.652238 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 15:47:57.652238 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 15:47:57.652238 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 15:47:57.652238 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application with Lock( Aug 22 15:47:57.652238 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 15:47:57.652238 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self._enter() Aug 22 15:47:57.652238 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 15:47:57.652238 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 15:47:57.652238 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 15:47:57.652238 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self.creator() Aug 22 15:47:57.652238 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 15:47:57.652238 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application created_value = creator( Aug 22 15:47:57.652238 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 22 15:47:57.653693 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 22 15:47:57.653693 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 22 15:47:57.653693 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application Aug 22 15:47:57.653693 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 415/817] 10.222.0.15 () {56 vars in 1108 bytes} [Mon Aug 22 15:47:57 2022] GET /identity/v3/domains/Default => generated 87 bytes in 379 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 15:47:57.653693 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3ff1098-f5d6-47b3-9480-926a38006152 None admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 15:47:57.697426 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e36976c0-1395-43e9-ad2c-eb8807ceb89e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81767) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 15:47:57.698408 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e36976c0-1395-43e9-ad2c-eb8807ceb89e None admin] REQUEST_METHOD: `GET` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:47:57.698881 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e36976c0-1395-43e9-ad2c-eb8807ceb89e None admin] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:47:57.699397 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e36976c0-1395-43e9-ad2c-eb8807ceb89e None admin] PATH_INFO: `/v3/services` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:47:57.700224 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e36976c0-1395-43e9-ad2c-eb8807ceb89e None admin] RBAC: Adding query filter params (name=network) {{(pid=81767) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 15:47:57.700732 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e36976c0-1395-43e9-ad2c-eb8807ceb89e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 15:47:57.702875 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e36976c0-1395-43e9-ad2c-eb8807ceb89e None admin] RBAC: Authorization granted {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 15:47:57.710942 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 403/818] 10.222.0.15 () {56 vars in 1112 bytes} [Mon Aug 22 15:47:57 2022] GET /identity/v3/services?name=network => generated 124 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 15:47:57.712609 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e36976c0-1395-43e9-ad2c-eb8807ceb89e None admin] Authenticating user token {{(pid=81767) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 15:47:57.743702 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG dbcounter [-] [81767] Writing DB stats keystone:SELECT=100 {{(pid=81767) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 15:47:57.773903 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3ff1098-f5d6-47b3-9480-926a38006152 None admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 15:47:57.822871 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42ff0847-8914-47ce-93bf-6219d5c5ba35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 15:47:57.824486 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42ff0847-8914-47ce-93bf-6219d5c5ba35 None admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:47:57.825063 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42ff0847-8914-47ce-93bf-6219d5c5ba35 None admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:47:57.825853 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42ff0847-8914-47ce-93bf-6219d5c5ba35 None admin] PATH_INFO: `/v3/domains` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:47:57.827006 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42ff0847-8914-47ce-93bf-6219d5c5ba35 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81768) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 15:47:57.827530 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42ff0847-8914-47ce-93bf-6219d5c5ba35 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 15:47:57.829689 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42ff0847-8914-47ce-93bf-6219d5c5ba35 None admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 15:47:57.850876 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 416/819] 10.222.0.15 () {56 vars in 1109 bytes} [Mon Aug 22 15:47:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 15:47:57.850876 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42ff0847-8914-47ce-93bf-6219d5c5ba35 None admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 15:47:57.854377 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e36976c0-1395-43e9-ad2c-eb8807ceb89e None admin] Validating token access rules against request {{(pid=81767) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 15:47:57.895647 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=100 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 15:47:57.903645 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc148335-4f33-43e6-8832-61ea0d0298f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81767) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 15:47:57.904634 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc148335-4f33-43e6-8832-61ea0d0298f7 None admin] REQUEST_METHOD: `GET` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:47:57.905110 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc148335-4f33-43e6-8832-61ea0d0298f7 None admin] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:47:57.905573 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc148335-4f33-43e6-8832-61ea0d0298f7 None admin] PATH_INFO: `/v3/domains` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:47:57.906448 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc148335-4f33-43e6-8832-61ea0d0298f7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81767) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 15:47:57.906991 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc148335-4f33-43e6-8832-61ea0d0298f7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 15:47:57.908551 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc148335-4f33-43e6-8832-61ea0d0298f7 None admin] RBAC: Authorization granted {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 15:47:57.922213 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 404/820] 10.222.0.15 () {56 vars in 1109 bytes} [Mon Aug 22 15:47:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 15:47:57.923416 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc148335-4f33-43e6-8832-61ea0d0298f7 None admin] Authenticating user token {{(pid=81767) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 15:47:57.970159 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42ff0847-8914-47ce-93bf-6219d5c5ba35 None admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 15:47:58.009444 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc5c748d-366b-49ba-bcb8-afc264e75d85 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 15:47:58.009862 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc5c748d-366b-49ba-bcb8-afc264e75d85 None admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:47:58.010140 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc5c748d-366b-49ba-bcb8-afc264e75d85 None admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:47:58.010411 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc5c748d-366b-49ba-bcb8-afc264e75d85 None admin] PATH_INFO: `/v3/services` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:47:58.011074 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc5c748d-366b-49ba-bcb8-afc264e75d85 None admin] RBAC: Adding query filter params (type=network) {{(pid=81768) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 15:47:58.011394 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc5c748d-366b-49ba-bcb8-afc264e75d85 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 15:47:58.015950 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc5c748d-366b-49ba-bcb8-afc264e75d85 None admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 15:47:58.016951 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG dbcounter [-] [81767] Writing DB stats keystone:SELECT=100 {{(pid=81767) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 15:47:58.023732 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 417/821] 10.222.0.15 () {56 vars in 1112 bytes} [Mon Aug 22 15:47:57 2022] GET /identity/v3/services?type=network => generated 350 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 15:47:58.024901 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc5c748d-366b-49ba-bcb8-afc264e75d85 None admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 15:47:58.049590 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc148335-4f33-43e6-8832-61ea0d0298f7 None admin] Validating token access rules against request {{(pid=81767) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 15:47:58.089699 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2272baae-10fa-41b1-9f05-d987a5fa759e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81767) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 15:47:58.090392 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2272baae-10fa-41b1-9f05-d987a5fa759e None admin] REQUEST_METHOD: `GET` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:47:58.090664 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2272baae-10fa-41b1-9f05-d987a5fa759e None admin] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:47:58.090960 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2272baae-10fa-41b1-9f05-d987a5fa759e None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:47:58.101630 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2272baae-10fa-41b1-9f05-d987a5fa759e None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 15:47:58.101630 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2272baae-10fa-41b1-9f05-d987a5fa759e None admin] RBAC: Authorization granted {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 15:47:58.116676 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application [None req-2272baae-10fa-41b1-9f05-d987a5fa759e None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 22 15:47:58.116676 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 15:47:58.116676 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 15:47:58.116676 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 15:47:58.116676 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 15:47:58.116676 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 15:47:58.116676 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 15:47:58.116676 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 15:47:58.116676 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 15:47:58.116676 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 15:47:58.116676 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 15:47:58.116676 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 15:47:58.116676 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 22 15:47:58.116676 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 22 15:47:58.116676 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 22 15:47:58.116676 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 22 15:47:58.117618 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 15:47:58.117618 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 15:47:58.117618 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 22 15:47:58.117618 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 22 15:47:58.117618 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 22 15:47:58.117618 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 22 15:47:58.117618 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 15:47:58.117618 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 15:47:58.117618 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 15:47:58.117618 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 15:47:58.117618 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 15:47:58.117618 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application with Lock( Aug 22 15:47:58.117618 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 15:47:58.117618 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return self._enter() Aug 22 15:47:58.117618 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 15:47:58.117618 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 15:47:58.117618 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 15:47:58.117618 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return self.creator() Aug 22 15:47:58.118531 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 15:47:58.118531 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application created_value = creator( Aug 22 15:47:58.118531 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 22 15:47:58.118531 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 22 15:47:58.118531 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 22 15:47:58.118531 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 22 15:47:58.118531 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 22 15:47:58.118531 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 22 15:47:58.118531 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 22 15:47:58.118531 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application Aug 22 15:47:58.123540 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 405/822] 10.222.0.15 () {56 vars in 1117 bytes} [Mon Aug 22 15:47:57 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 15:47:58.124433 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2272baae-10fa-41b1-9f05-d987a5fa759e None admin] Authenticating user token {{(pid=81767) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 15:47:58.129724 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=100 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 15:47:58.146116 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc5c748d-366b-49ba-bcb8-afc264e75d85 None admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 15:47:58.185442 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7c321e8-ace7-40f8-a7c3-27ae39c1d7d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 15:47:58.186568 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7c321e8-ace7-40f8-a7c3-27ae39c1d7d0 None admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:47:58.187034 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7c321e8-ace7-40f8-a7c3-27ae39c1d7d0 None admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:47:58.187578 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7c321e8-ace7-40f8-a7c3-27ae39c1d7d0 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:47:58.195584 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7c321e8-ace7-40f8-a7c3-27ae39c1d7d0 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 15:47:58.197406 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7c321e8-ace7-40f8-a7c3-27ae39c1d7d0 None admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 15:47:58.209575 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application [None req-c7c321e8-ace7-40f8-a7c3-27ae39c1d7d0 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 22 15:47:58.209575 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 15:47:58.209575 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 15:47:58.209575 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 15:47:58.209575 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 15:47:58.209575 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 15:47:58.209575 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 15:47:58.209575 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 15:47:58.209575 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 15:47:58.209575 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 15:47:58.209575 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 15:47:58.209575 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 15:47:58.209575 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 22 15:47:58.209575 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 22 15:47:58.209575 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 22 15:47:58.210569 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 22 15:47:58.210569 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 15:47:58.210569 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 15:47:58.210569 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 15:47:58.210569 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 15:47:58.210569 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 15:47:58.210569 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 15:47:58.210569 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 15:47:58.210569 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application with Lock( Aug 22 15:47:58.210569 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 15:47:58.210569 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self._enter() Aug 22 15:47:58.210569 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 15:47:58.210569 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 15:47:58.210569 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 15:47:58.210569 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self.creator() Aug 22 15:47:58.210569 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 15:47:58.210569 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application created_value = creator( Aug 22 15:47:58.211465 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 22 15:47:58.211465 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 22 15:47:58.211465 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 22 15:47:58.211465 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 22 15:47:58.211465 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 22 15:47:58.211465 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 22 15:47:58.211465 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 22 15:47:58.211465 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application Aug 22 15:47:58.214539 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 418/823] 10.222.0.15 () {56 vars in 1111 bytes} [Mon Aug 22 15:47:58 2022] GET /identity/v3/projects/service => generated 88 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 15:47:58.215672 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7c321e8-ace7-40f8-a7c3-27ae39c1d7d0 None admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 15:47:58.238147 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2272baae-10fa-41b1-9f05-d987a5fa759e None admin] Validating token access rules against request {{(pid=81767) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 15:47:58.274983 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c45bc0d-da92-4d15-b260-6de8c3d79bd2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81767) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 15:47:58.276217 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c45bc0d-da92-4d15-b260-6de8c3d79bd2 None admin] REQUEST_METHOD: `POST` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:47:58.276488 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c45bc0d-da92-4d15-b260-6de8c3d79bd2 None admin] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:47:58.276754 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c45bc0d-da92-4d15-b260-6de8c3d79bd2 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:47:58.277474 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c45bc0d-da92-4d15-b260-6de8c3d79bd2 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 15:47:58.278948 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c45bc0d-da92-4d15-b260-6de8c3d79bd2 None admin] RBAC: Authorization granted {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 15:47:58.288887 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG dbcounter [-] [81767] Writing DB stats keystone:SELECT=100 {{(pid=81767) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 15:47:58.330406 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7c321e8-ace7-40f8-a7c3-27ae39c1d7d0 None admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 15:47:58.367421 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93af89bd-eb9f-474e-bc20-53679a4ed7e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 15:47:58.368029 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93af89bd-eb9f-474e-bc20-53679a4ed7e7 None admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:47:58.368305 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93af89bd-eb9f-474e-bc20-53679a4ed7e7 None admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:47:58.368569 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93af89bd-eb9f-474e-bc20-53679a4ed7e7 None admin] PATH_INFO: `/v3/users` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:47:58.369315 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93af89bd-eb9f-474e-bc20-53679a4ed7e7 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81768) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 15:47:58.369612 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93af89bd-eb9f-474e-bc20-53679a4ed7e7 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 15:47:58.371024 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93af89bd-eb9f-474e-bc20-53679a4ed7e7 None admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 15:47:58.390103 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 419/824] 10.222.0.15 () {56 vars in 1149 bytes} [Mon Aug 22 15:47:58 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 15:47:58.390947 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93af89bd-eb9f-474e-bc20-53679a4ed7e7 None admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 15:47:58.393915 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=100 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 15:47:58.452819 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 406/825] 10.222.0.15 () {60 vars in 1144 bytes} [Mon Aug 22 15:47:58 2022] POST /identity/v3/endpoints => generated 337 bytes in 327 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 22 15:47:58.452819 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08cbb570-247f-4b74-b481-84f66b3f668e None None] REQUEST_METHOD: `GET` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:47:58.452819 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08cbb570-247f-4b74-b481-84f66b3f668e None None] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:47:58.452819 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08cbb570-247f-4b74-b481-84f66b3f668e None None] PATH_INFO: `/` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:47:58.454896 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 407/826] 10.222.0.15 () {54 vars in 927 bytes} [Mon Aug 22 15:47:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 15:47:58.459809 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08cbb570-247f-4b74-b481-84f66b3f668e None None] Authenticating user token {{(pid=81767) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 15:47:58.526800 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93af89bd-eb9f-474e-bc20-53679a4ed7e7 None admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 15:47:58.576687 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a35f244-1e06-48c0-b63f-e5fa861ff894 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 15:47:58.577446 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a35f244-1e06-48c0-b63f-e5fa861ff894 None admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:47:58.577792 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a35f244-1e06-48c0-b63f-e5fa861ff894 None admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:47:58.578068 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a35f244-1e06-48c0-b63f-e5fa861ff894 None admin] PATH_INFO: `/v3/projects` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:47:58.578872 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a35f244-1e06-48c0-b63f-e5fa861ff894 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81768) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 15:47:58.579186 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a35f244-1e06-48c0-b63f-e5fa861ff894 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 15:47:58.581024 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a35f244-1e06-48c0-b63f-e5fa861ff894 None admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 15:47:58.596178 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 420/827] 10.222.0.15 () {56 vars in 1148 bytes} [Mon Aug 22 15:47:58 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 15:47:58.597632 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3dddaf6-a2ad-4a61-9a51-da59993846ae None None] REQUEST_METHOD: `POST` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:47:58.597920 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3dddaf6-a2ad-4a61-9a51-da59993846ae None None] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:47:58.598191 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3dddaf6-a2ad-4a61-9a51-da59993846ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:47:58.600275 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08cbb570-247f-4b74-b481-84f66b3f668e None None] Validating token access rules against request {{(pid=81767) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 15:47:58.618488 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.auth.core [None req-d3dddaf6-a2ad-4a61-9a51-da59993846ae None None] MFA Rules not processed for user `6b2199ae0fb0480aa917ada391f5ec54`. Rule list: `[]` (Enabled: `True`). {{(pid=81768) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 15:47:58.619932 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-d3dddaf6-a2ad-4a61-9a51-da59993846ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 15:47:58.638897 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26536500-5e5a-4cd2-8365-d3fe16f6741a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81767) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 15:47:58.639993 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26536500-5e5a-4cd2-8365-d3fe16f6741a None admin] REQUEST_METHOD: `GET` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:47:58.640464 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26536500-5e5a-4cd2-8365-d3fe16f6741a None admin] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:47:58.640922 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26536500-5e5a-4cd2-8365-d3fe16f6741a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:47:58.654900 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26536500-5e5a-4cd2-8365-d3fe16f6741a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 15:47:58.656878 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26536500-5e5a-4cd2-8365-d3fe16f6741a None admin] RBAC: Authorization granted {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 15:47:58.672352 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application [None req-26536500-5e5a-4cd2-8365-d3fe16f6741a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 22 15:47:58.672352 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 15:47:58.672352 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 15:47:58.672352 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 15:47:58.672352 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 15:47:58.672352 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 15:47:58.672352 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 15:47:58.672352 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 15:47:58.672352 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 15:47:58.672352 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 15:47:58.672352 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 15:47:58.672352 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 15:47:58.672352 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 22 15:47:58.672352 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 22 15:47:58.672352 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 22 15:47:58.672352 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 22 15:47:58.673306 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 15:47:58.673306 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 15:47:58.673306 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 15:47:58.673306 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 15:47:58.673306 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 15:47:58.673306 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 15:47:58.673306 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 15:47:58.673306 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application with Lock( Aug 22 15:47:58.673306 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 15:47:58.673306 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return self._enter() Aug 22 15:47:58.673306 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 15:47:58.673306 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 15:47:58.673306 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 15:47:58.673306 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return self.creator() Aug 22 15:47:58.673306 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 15:47:58.673306 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application created_value = creator( Aug 22 15:47:58.673306 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 22 15:47:58.674168 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 22 15:47:58.674168 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 22 15:47:58.674168 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application Aug 22 15:47:58.677114 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 408/828] 10.222.0.15 () {56 vars in 1108 bytes} [Mon Aug 22 15:47:58 2022] GET /identity/v3/domains/Default => generated 87 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 15:47:58.678132 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26536500-5e5a-4cd2-8365-d3fe16f6741a None admin] Authenticating user token {{(pid=81767) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 15:47:58.687614 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=100 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 15:47:58.702710 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 421/829] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Aug 22 15:47:58 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 22 15:47:58.703638 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3dddaf6-a2ad-4a61-9a51-da59993846ae None None] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 15:47:58.736378 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG dbcounter [-] [81767] Writing DB stats keystone:INSERT=1,keystone:SELECT=99 {{(pid=81767) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 15:47:58.801365 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26536500-5e5a-4cd2-8365-d3fe16f6741a None admin] Validating token access rules against request {{(pid=81767) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 15:47:58.835575 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3dddaf6-a2ad-4a61-9a51-da59993846ae None None] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 15:47:58.846897 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-294d527d-00ea-4e49-aa97-902cae0f278d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81767) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 15:47:58.848102 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-294d527d-00ea-4e49-aa97-902cae0f278d None admin] REQUEST_METHOD: `GET` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:47:58.848102 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-294d527d-00ea-4e49-aa97-902cae0f278d None admin] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:47:58.848253 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-294d527d-00ea-4e49-aa97-902cae0f278d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:47:58.849571 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-294d527d-00ea-4e49-aa97-902cae0f278d None admin] RBAC: Adding query filter params (role.id=9d9848cfd22e4b089db443c350e56670, scope.project.id=c4ee0413f9ef4d4fb1f95a660babe090, user.id=e4ef569e41fb46d0b721bb1c7663616f) {{(pid=81767) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 15:47:58.849571 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-294d527d-00ea-4e49-aa97-902cae0f278d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 15:47:58.851536 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-294d527d-00ea-4e49-aa97-902cae0f278d None admin] RBAC: Authorization granted {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 15:47:58.861970 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 409/830] 10.222.0.15 () {56 vars in 1374 bytes} [Mon Aug 22 15:47:58 2022] GET /identity/v3/role_assignments?user.id=e4ef569e41fb46d0b721bb1c7663616f&scope.project.id=c4ee0413f9ef4d4fb1f95a660babe090&role.id=9d9848cfd22e4b089db443c350e56670 => generated 608 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 15:47:58.891838 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97f1e477-6e69-4c23-920e-5dfeae8e9c80 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 15:47:58.892796 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97f1e477-6e69-4c23-920e-5dfeae8e9c80 None admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:47:58.893254 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97f1e477-6e69-4c23-920e-5dfeae8e9c80 None admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:47:58.893725 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97f1e477-6e69-4c23-920e-5dfeae8e9c80 None admin] PATH_INFO: `/v3/domains` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:47:58.894725 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97f1e477-6e69-4c23-920e-5dfeae8e9c80 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81768) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 15:47:58.895192 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97f1e477-6e69-4c23-920e-5dfeae8e9c80 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 15:47:58.897087 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97f1e477-6e69-4c23-920e-5dfeae8e9c80 None admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 15:47:58.911612 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 422/831] 10.222.0.15 () {56 vars in 1109 bytes} [Mon Aug 22 15:47:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 15:47:58.920228 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-294d527d-00ea-4e49-aa97-902cae0f278d None admin] Authenticating user token {{(pid=81767) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 15:47:59.002474 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97f1e477-6e69-4c23-920e-5dfeae8e9c80 None admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 15:47:59.057041 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-294d527d-00ea-4e49-aa97-902cae0f278d None admin] Validating token access rules against request {{(pid=81767) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 15:47:59.074541 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG dbcounter [-] [81767] Writing DB stats keystone:SELECT=100 {{(pid=81767) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 15:47:59.086986 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=100 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 15:47:59.112941 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbb51d22-9ca4-4b36-b435-1fb791ca4138 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81767) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 15:47:59.113638 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb51d22-9ca4-4b36-b435-1fb791ca4138 None admin] REQUEST_METHOD: `GET` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:47:59.113912 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb51d22-9ca4-4b36-b435-1fb791ca4138 None admin] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:47:59.114232 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb51d22-9ca4-4b36-b435-1fb791ca4138 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:47:59.125004 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbb51d22-9ca4-4b36-b435-1fb791ca4138 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 15:47:59.126459 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbb51d22-9ca4-4b36-b435-1fb791ca4138 None admin] RBAC: Authorization granted {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 15:47:59.150176 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application [None req-fbb51d22-9ca4-4b36-b435-1fb791ca4138 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 22 15:47:59.150176 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 15:47:59.150176 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 15:47:59.150176 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 15:47:59.150176 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 15:47:59.150176 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 15:47:59.150176 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 15:47:59.150176 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 15:47:59.150176 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 15:47:59.150176 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 15:47:59.150176 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 15:47:59.150176 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 15:47:59.150176 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 22 15:47:59.150176 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 22 15:47:59.150176 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 22 15:47:59.152270 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 22 15:47:59.152270 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 15:47:59.152270 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 15:47:59.152270 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 15:47:59.152270 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 15:47:59.152270 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 15:47:59.152270 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 15:47:59.152270 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 15:47:59.152270 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application with Lock( Aug 22 15:47:59.152270 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 15:47:59.152270 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return self._enter() Aug 22 15:47:59.152270 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 15:47:59.152270 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 15:47:59.152270 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 15:47:59.152270 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return self.creator() Aug 22 15:47:59.152270 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 15:47:59.152270 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application created_value = creator( Aug 22 15:47:59.153934 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 22 15:47:59.153934 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 22 15:47:59.153934 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 22 15:47:59.153934 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 22 15:47:59.153934 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 22 15:47:59.153934 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 22 15:47:59.153934 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 22 15:47:59.153934 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application Aug 22 15:47:59.153934 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 410/832] 10.222.0.15 () {56 vars in 1111 bytes} [Mon Aug 22 15:47:58 2022] GET /identity/v3/projects/service => generated 88 bytes in 236 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 15:47:59.158969 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbb51d22-9ca4-4b36-b435-1fb791ca4138 None admin] Authenticating user token {{(pid=81767) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 15:47:59.176225 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97f1e477-6e69-4c23-920e-5dfeae8e9c80 None admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 15:47:59.216400 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2423dc67-9c22-4221-af4f-dedaa50f6edf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 15:47:59.217425 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2423dc67-9c22-4221-af4f-dedaa50f6edf None admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:47:59.217856 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2423dc67-9c22-4221-af4f-dedaa50f6edf None admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:47:59.218269 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2423dc67-9c22-4221-af4f-dedaa50f6edf None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:47:59.230951 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2423dc67-9c22-4221-af4f-dedaa50f6edf None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 22 15:47:59.236620 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application [None req-2423dc67-9c22-4221-af4f-dedaa50f6edf None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 22 15:47:59.236620 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 15:47:59.236620 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 15:47:59.236620 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 15:47:59.236620 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 15:47:59.236620 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 15:47:59.236620 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 15:47:59.236620 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 15:47:59.236620 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 15:47:59.236620 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 15:47:59.236620 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 15:47:59.236620 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 15:47:59.236620 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 22 15:47:59.236620 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 22 15:47:59.236620 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 22 15:47:59.236620 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 22 15:47:59.237846 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 22 15:47:59.237846 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 22 15:47:59.237846 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 22 15:47:59.237846 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 22 15:47:59.237846 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 15:47:59.237846 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 15:47:59.237846 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 15:47:59.237846 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 15:47:59.237846 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 15:47:59.237846 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 15:47:59.237846 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 15:47:59.237846 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application with Lock( Aug 22 15:47:59.237846 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 15:47:59.237846 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self._enter() Aug 22 15:47:59.237846 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 15:47:59.237846 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 15:47:59.239134 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 15:47:59.239134 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self.creator() Aug 22 15:47:59.239134 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 15:47:59.239134 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application created_value = creator( Aug 22 15:47:59.239134 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 22 15:47:59.239134 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 22 15:47:59.239134 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 22 15:47:59.239134 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 22 15:47:59.239134 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 22 15:47:59.239134 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 22 15:47:59.239134 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 22 15:47:59.239134 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application Aug 22 15:47:59.241856 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 423/833] 10.222.0.15 () {56 vars in 1096 bytes} [Mon Aug 22 15:47:59 2022] GET /identity/v3/roles/admin => generated 83 bytes in 240 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 22 15:47:59.249647 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2423dc67-9c22-4221-af4f-dedaa50f6edf None admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 15:47:59.319947 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbb51d22-9ca4-4b36-b435-1fb791ca4138 None admin] Validating token access rules against request {{(pid=81767) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 15:47:59.380042 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b6d73ca-8cdb-4f50-96cb-47099e64a4bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81767) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 15:47:59.380868 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b6d73ca-8cdb-4f50-96cb-47099e64a4bd None admin] REQUEST_METHOD: `GET` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:47:59.381377 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b6d73ca-8cdb-4f50-96cb-47099e64a4bd None admin] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:47:59.381704 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b6d73ca-8cdb-4f50-96cb-47099e64a4bd None admin] PATH_INFO: `/v3/projects` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:47:59.382733 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b6d73ca-8cdb-4f50-96cb-47099e64a4bd None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81767) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 15:47:59.383052 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b6d73ca-8cdb-4f50-96cb-47099e64a4bd None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 15:47:59.384647 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b6d73ca-8cdb-4f50-96cb-47099e64a4bd None admin] RBAC: Authorization granted {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 15:47:59.402384 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2423dc67-9c22-4221-af4f-dedaa50f6edf None admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 15:47:59.407752 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 411/834] 10.222.0.15 () {56 vars in 1148 bytes} [Mon Aug 22 15:47:59 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 250 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 22 15:47:59.408803 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b6d73ca-8cdb-4f50-96cb-47099e64a4bd None admin] Authenticating user token {{(pid=81767) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 15:47:59.416932 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=100 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 15:47:59.446565 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG dbcounter [-] [81767] Writing DB stats keystone:SELECT=100 {{(pid=81767) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 15:47:59.451950 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6901cd0-b520-46b0-aa3b-b0d93906292e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 15:47:59.452560 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6901cd0-b520-46b0-aa3b-b0d93906292e None admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:47:59.452836 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6901cd0-b520-46b0-aa3b-b0d93906292e None admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:47:59.453106 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6901cd0-b520-46b0-aa3b-b0d93906292e None admin] PATH_INFO: `/v3/roles` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:47:59.453781 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6901cd0-b520-46b0-aa3b-b0d93906292e None admin] RBAC: Adding query filter params (name=admin) {{(pid=81768) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 15:47:59.454077 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6901cd0-b520-46b0-aa3b-b0d93906292e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 15:47:59.456746 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6901cd0-b520-46b0-aa3b-b0d93906292e None admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 15:47:59.471382 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 424/835] 10.222.0.15 () {56 vars in 1099 bytes} [Mon Aug 22 15:47:59 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 15:47:59.474355 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cfec253-42da-48f7-ba09-76713fa7e535 None None] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:47:59.474631 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cfec253-42da-48f7-ba09-76713fa7e535 None None] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:47:59.474910 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cfec253-42da-48f7-ba09-76713fa7e535 None None] PATH_INFO: `/` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:47:59.476676 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 425/836] 10.222.0.15 () {54 vars in 927 bytes} [Mon Aug 22 15:47:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 15:47:59.477736 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cfec253-42da-48f7-ba09-76713fa7e535 None None] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 15:47:59.550812 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b6d73ca-8cdb-4f50-96cb-47099e64a4bd None admin] Validating token access rules against request {{(pid=81767) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 15:47:59.600159 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b7ca67c-ed06-43c3-b6ed-1991137c7552 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81767) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 15:47:59.600727 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b7ca67c-ed06-43c3-b6ed-1991137c7552 None admin] REQUEST_METHOD: `PUT` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:47:59.601011 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b7ca67c-ed06-43c3-b6ed-1991137c7552 None admin] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:47:59.601280 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b7ca67c-ed06-43c3-b6ed-1991137c7552 None admin] PATH_INFO: `/v3/projects/c4ee0413f9ef4d4fb1f95a660babe090/users/058e76354dce412cb78b2875c88f2791/roles/a5fbba1dc615490a96cde27e3a102eb6` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:47:59.618123 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cfec253-42da-48f7-ba09-76713fa7e535 None None] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 15:47:59.621644 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b7ca67c-ed06-43c3-b6ed-1991137c7552 None admin] RBAC: Authorizing `identity:create_grant(project_id=c4ee0413f9ef4d4fb1f95a660babe090, user_id=058e76354dce412cb78b2875c88f2791, role_id=a5fbba1dc615490a96cde27e3a102eb6)` {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 15:47:59.623109 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b7ca67c-ed06-43c3-b6ed-1991137c7552 None admin] RBAC: Authorization granted {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 15:47:59.654418 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f666bbc7-120b-42a5-8323-8f576b165ec4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 15:47:59.655046 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f666bbc7-120b-42a5-8323-8f576b165ec4 None admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:47:59.655354 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f666bbc7-120b-42a5-8323-8f576b165ec4 None admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:47:59.655630 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f666bbc7-120b-42a5-8323-8f576b165ec4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:47:59.668428 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f666bbc7-120b-42a5-8323-8f576b165ec4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 15:47:59.669901 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f666bbc7-120b-42a5-8323-8f576b165ec4 None admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 15:47:59.685145 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application [None req-f666bbc7-120b-42a5-8323-8f576b165ec4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 22 15:47:59.685145 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 15:47:59.685145 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 15:47:59.685145 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 15:47:59.685145 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 15:47:59.685145 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 15:47:59.685145 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 15:47:59.685145 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 15:47:59.685145 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 15:47:59.685145 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 15:47:59.685145 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 15:47:59.685145 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 15:47:59.685145 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 22 15:47:59.685145 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 22 15:47:59.685145 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 22 15:47:59.685145 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 22 15:47:59.693324 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 15:47:59.693324 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 15:47:59.693324 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 15:47:59.693324 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 15:47:59.693324 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 15:47:59.693324 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 15:47:59.693324 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 15:47:59.693324 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application with Lock( Aug 22 15:47:59.693324 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 15:47:59.693324 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self._enter() Aug 22 15:47:59.693324 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 15:47:59.693324 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 15:47:59.693324 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 15:47:59.693324 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self.creator() Aug 22 15:47:59.693324 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 15:47:59.693324 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application created_value = creator( Aug 22 15:47:59.693324 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 22 15:47:59.694300 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 22 15:47:59.694300 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 22 15:47:59.694300 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application Aug 22 15:47:59.694300 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 426/837] 10.222.0.15 () {56 vars in 1108 bytes} [Mon Aug 22 15:47:59 2022] GET /identity/v3/domains/Default => generated 87 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 15:47:59.694300 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb6f56a2-e98b-4e18-ad5d-192039ae66e2 None None] REQUEST_METHOD: `POST` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:47:59.694300 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb6f56a2-e98b-4e18-ad5d-192039ae66e2 None None] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:47:59.694300 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb6f56a2-e98b-4e18-ad5d-192039ae66e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:47:59.706544 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.auth.core [None req-cb6f56a2-e98b-4e18-ad5d-192039ae66e2 None None] MFA Rules not processed for user `6b2199ae0fb0480aa917ada391f5ec54`. Rule list: `[]` (Enabled: `True`). {{(pid=81768) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 15:47:59.707779 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-cb6f56a2-e98b-4e18-ad5d-192039ae66e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 15:47:59.726031 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=100 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 15:47:59.781052 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 427/838] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Aug 22 15:47:59 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 22 15:47:59.782079 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb6f56a2-e98b-4e18-ad5d-192039ae66e2 None None] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 15:47:59.797526 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 412/839] 10.222.0.15 () {58 vars in 1439 bytes} [Mon Aug 22 15:47:59 2022] PUT /identity/v3/projects/c4ee0413f9ef4d4fb1f95a660babe090/users/058e76354dce412cb78b2875c88f2791/roles/a5fbba1dc615490a96cde27e3a102eb6 => generated 0 bytes in 389 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 15:47:59.912625 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb6f56a2-e98b-4e18-ad5d-192039ae66e2 None None] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 15:47:59.960857 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7fd80c5-4b36-4500-933a-6423214ef1e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 15:47:59.961932 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7fd80c5-4b36-4500-933a-6423214ef1e8 None admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:47:59.962378 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7fd80c5-4b36-4500-933a-6423214ef1e8 None admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:47:59.962847 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7fd80c5-4b36-4500-933a-6423214ef1e8 None admin] PATH_INFO: `/v3/domains` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:47:59.963749 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7fd80c5-4b36-4500-933a-6423214ef1e8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81768) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 15:47:59.964219 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7fd80c5-4b36-4500-933a-6423214ef1e8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 15:47:59.965852 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7fd80c5-4b36-4500-933a-6423214ef1e8 None admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 15:47:59.982161 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 428/840] 10.222.0.15 () {56 vars in 1109 bytes} [Mon Aug 22 15:47:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 15:47:59.988342 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b7ca67c-ed06-43c3-b6ed-1991137c7552 None admin] Authenticating user token {{(pid=81767) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 15:48:00.081808 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7fd80c5-4b36-4500-933a-6423214ef1e8 None admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 15:48:00.082787 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG dbcounter [-] [81767] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81767) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 15:48:00.099106 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=100 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 15:48:00.105223 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b7ca67c-ed06-43c3-b6ed-1991137c7552 None admin] Validating token access rules against request {{(pid=81767) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 15:48:00.145535 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f902a987-f23d-4232-9d80-7b15a25f9833 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81767) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 15:48:00.146172 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f902a987-f23d-4232-9d80-7b15a25f9833 None admin] REQUEST_METHOD: `GET` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:48:00.146466 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f902a987-f23d-4232-9d80-7b15a25f9833 None admin] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:48:00.146753 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f902a987-f23d-4232-9d80-7b15a25f9833 None admin] PATH_INFO: `/v3/users/swift` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:48:00.155817 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f902a987-f23d-4232-9d80-7b15a25f9833 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 15:48:00.157290 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f902a987-f23d-4232-9d80-7b15a25f9833 None admin] RBAC: Authorization granted {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 15:48:00.171132 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application [None req-f902a987-f23d-4232-9d80-7b15a25f9833 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 22 15:48:00.171132 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 15:48:00.171132 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 15:48:00.171132 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 15:48:00.171132 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 15:48:00.171132 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 15:48:00.171132 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 15:48:00.171132 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 15:48:00.171132 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 15:48:00.171132 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 15:48:00.171132 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 15:48:00.171132 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 15:48:00.171132 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 22 15:48:00.171132 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 22 15:48:00.171132 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 22 15:48:00.171132 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 22 15:48:00.172138 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 15:48:00.172138 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 15:48:00.172138 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 22 15:48:00.172138 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 22 15:48:00.172138 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 22 15:48:00.172138 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 22 15:48:00.172138 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 15:48:00.172138 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 15:48:00.172138 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 15:48:00.172138 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 15:48:00.172138 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 15:48:00.172138 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application with Lock( Aug 22 15:48:00.172138 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 15:48:00.172138 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return self._enter() Aug 22 15:48:00.172138 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 15:48:00.172138 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 15:48:00.172138 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 15:48:00.172138 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return self.creator() Aug 22 15:48:00.173113 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 15:48:00.173113 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application created_value = creator( Aug 22 15:48:00.173113 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 22 15:48:00.173113 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 22 15:48:00.173113 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 22 15:48:00.173113 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 22 15:48:00.173113 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 22 15:48:00.173113 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 22 15:48:00.173113 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 22 15:48:00.173113 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application Aug 22 15:48:00.177442 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 413/841] 10.222.0.15 () {56 vars in 1096 bytes} [Mon Aug 22 15:47:59 2022] GET /identity/v3/users/swift => generated 83 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 15:48:00.178441 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f902a987-f23d-4232-9d80-7b15a25f9833 None admin] Authenticating user token {{(pid=81767) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 15:48:00.209872 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7fd80c5-4b36-4500-933a-6423214ef1e8 None admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 15:48:00.247908 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a88d6cb-d3d8-481f-8953-ad082f9de46f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 15:48:00.248538 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a88d6cb-d3d8-481f-8953-ad082f9de46f None admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:48:00.248862 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a88d6cb-d3d8-481f-8953-ad082f9de46f None admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:48:00.249137 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a88d6cb-d3d8-481f-8953-ad082f9de46f None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:48:00.249721 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a88d6cb-d3d8-481f-8953-ad082f9de46f None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 15:48:00.251101 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a88d6cb-d3d8-481f-8953-ad082f9de46f None admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 15:48:00.263492 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application [None req-8a88d6cb-d3d8-481f-8953-ad082f9de46f None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 22 15:48:00.263492 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 15:48:00.263492 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 22 15:48:00.263492 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 22 15:48:00.263492 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 22 15:48:00.263492 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 22 15:48:00.263492 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 22 15:48:00.263492 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 22 15:48:00.263492 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 22 15:48:00.263492 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application Aug 22 15:48:00.263492 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 22 15:48:00.263492 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application Aug 22 15:48:00.263492 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 15:48:00.263492 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 15:48:00.263492 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 15:48:00.263492 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 15:48:00.263492 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 15:48:00.278164 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 15:48:00.278164 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 15:48:00.278164 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 15:48:00.278164 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 15:48:00.278164 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 15:48:00.278164 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 15:48:00.278164 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 22 15:48:00.278164 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 22 15:48:00.278164 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 22 15:48:00.278164 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 22 15:48:00.278164 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 15:48:00.278164 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 15:48:00.278164 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 15:48:00.278164 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 15:48:00.278164 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 15:48:00.278164 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 15:48:00.280290 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 15:48:00.280290 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application with Lock( Aug 22 15:48:00.280290 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 15:48:00.280290 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self._enter() Aug 22 15:48:00.280290 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 15:48:00.280290 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 15:48:00.280290 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 15:48:00.280290 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self.creator() Aug 22 15:48:00.280290 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 15:48:00.280290 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application created_value = creator( Aug 22 15:48:00.280290 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 22 15:48:00.280290 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 22 15:48:00.280290 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 22 15:48:00.280290 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application Aug 22 15:48:00.280290 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 429/842] 10.222.0.15 () {56 vars in 1114 bytes} [Mon Aug 22 15:48:00 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 22 15:48:00.282166 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a88d6cb-d3d8-481f-8953-ad082f9de46f None admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 15:48:00.288185 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f902a987-f23d-4232-9d80-7b15a25f9833 None admin] Validating token access rules against request {{(pid=81767) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 15:48:00.326796 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3eef737-66d1-4a7b-bf28-525c648c45df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81767) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 15:48:00.327510 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3eef737-66d1-4a7b-bf28-525c648c45df None admin] REQUEST_METHOD: `GET` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:48:00.327796 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3eef737-66d1-4a7b-bf28-525c648c45df None admin] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:48:00.328162 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3eef737-66d1-4a7b-bf28-525c648c45df None admin] PATH_INFO: `/v3/users` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:48:00.329117 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3eef737-66d1-4a7b-bf28-525c648c45df None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81767) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 15:48:00.329594 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3eef737-66d1-4a7b-bf28-525c648c45df None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 15:48:00.332595 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3eef737-66d1-4a7b-bf28-525c648c45df None admin] RBAC: Authorization granted {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 15:48:00.353367 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG dbcounter [-] [81767] Writing DB stats keystone:SELECT=100 {{(pid=81767) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 15:48:00.371718 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=100 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 15:48:00.378737 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 414/843] 10.222.0.15 () {56 vars in 1135 bytes} [Mon Aug 22 15:48:00 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 15:48:00.381328 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3eef737-66d1-4a7b-bf28-525c648c45df None admin] Authenticating user token {{(pid=81767) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 15:48:00.412714 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a88d6cb-d3d8-481f-8953-ad082f9de46f None admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 15:48:00.464801 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31c65e07-89c6-4904-a195-1b6491876d0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 15:48:00.466071 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31c65e07-89c6-4904-a195-1b6491876d0b None admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:48:00.466809 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31c65e07-89c6-4904-a195-1b6491876d0b None admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:48:00.467336 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31c65e07-89c6-4904-a195-1b6491876d0b None admin] PATH_INFO: `/v3/services` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:48:00.468843 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31c65e07-89c6-4904-a195-1b6491876d0b None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=81768) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 15:48:00.469353 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31c65e07-89c6-4904-a195-1b6491876d0b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 15:48:00.470936 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31c65e07-89c6-4904-a195-1b6491876d0b None admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 15:48:00.478890 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 430/844] 10.222.0.15 () {56 vars in 1114 bytes} [Mon Aug 22 15:48:00 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 22 15:48:00.485395 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31c65e07-89c6-4904-a195-1b6491876d0b None admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 15:48:00.524857 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3eef737-66d1-4a7b-bf28-525c648c45df None admin] Validating token access rules against request {{(pid=81767) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 15:48:00.578541 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ae096ce-9c71-4ddd-970c-803225c8ab44 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81767) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 15:48:00.579310 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ae096ce-9c71-4ddd-970c-803225c8ab44 None admin] REQUEST_METHOD: `GET` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:48:00.579599 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ae096ce-9c71-4ddd-970c-803225c8ab44 None admin] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:48:00.579877 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ae096ce-9c71-4ddd-970c-803225c8ab44 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:48:00.624868 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31c65e07-89c6-4904-a195-1b6491876d0b None admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 15:48:00.675738 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG dbcounter [-] [81767] Writing DB stats keystone:SELECT=100 {{(pid=81767) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 15:48:00.681757 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ae096ce-9c71-4ddd-970c-803225c8ab44 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 15:48:00.682683 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3b3b75d-c90d-4272-9c89-2e3f41b94d11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 15:48:00.683627 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ae096ce-9c71-4ddd-970c-803225c8ab44 None admin] RBAC: Authorization granted {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 15:48:00.687775 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3b3b75d-c90d-4272-9c89-2e3f41b94d11 None admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:48:00.687775 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3b3b75d-c90d-4272-9c89-2e3f41b94d11 None admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:48:00.687775 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3b3b75d-c90d-4272-9c89-2e3f41b94d11 None admin] PATH_INFO: `/v3/services` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:48:00.687775 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3b3b75d-c90d-4272-9c89-2e3f41b94d11 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=81768) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 15:48:00.687775 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3b3b75d-c90d-4272-9c89-2e3f41b94d11 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 15:48:00.688683 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3b3b75d-c90d-4272-9c89-2e3f41b94d11 None admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 15:48:00.696176 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 431/845] 10.222.0.15 () {56 vars in 1114 bytes} [Mon Aug 22 15:48:00 2022] GET /identity/v3/services?type=volumev3 => generated 125 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 22 15:48:00.826975 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 415/846] 10.222.0.15 () {58 vars in 1282 bytes} [Mon Aug 22 15:48:00 2022] GET /identity/v3/auth/tokens => generated 1161 bytes in 446 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 22 15:48:00.836255 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3b3b75d-c90d-4272-9c89-2e3f41b94d11 None admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 15:48:00.842037 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=100 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 15:48:00.964990 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3b3b75d-c90d-4272-9c89-2e3f41b94d11 None admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 15:48:01.007474 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05344494-fa2d-4c51-9cd1-89e98e78bbf0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 15:48:01.008500 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05344494-fa2d-4c51-9cd1-89e98e78bbf0 None admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:48:01.008992 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05344494-fa2d-4c51-9cd1-89e98e78bbf0 None admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:48:01.009547 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05344494-fa2d-4c51-9cd1-89e98e78bbf0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:48:01.016528 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fe46753-1a27-4837-a952-1043ffec9768 None None] REQUEST_METHOD: `GET` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:48:01.017434 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fe46753-1a27-4837-a952-1043ffec9768 None None] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:48:01.017937 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fe46753-1a27-4837-a952-1043ffec9768 None None] PATH_INFO: `/` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:48:01.019835 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 416/847] 10.222.0.15 () {54 vars in 927 bytes} [Mon Aug 22 15:48:01 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 22 15:48:01.027874 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05344494-fa2d-4c51-9cd1-89e98e78bbf0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 15:48:01.028484 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fe8b29f-0407-480f-ad95-d04f1985c95d None None] REQUEST_METHOD: `POST` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:48:01.028994 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fe8b29f-0407-480f-ad95-d04f1985c95d None None] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:48:01.029346 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05344494-fa2d-4c51-9cd1-89e98e78bbf0 None admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 15:48:01.029738 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fe8b29f-0407-480f-ad95-d04f1985c95d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:48:01.044804 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application [None req-05344494-fa2d-4c51-9cd1-89e98e78bbf0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 22 15:48:01.044804 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 15:48:01.044804 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 15:48:01.044804 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 15:48:01.044804 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 15:48:01.044804 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 15:48:01.044804 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 15:48:01.044804 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 15:48:01.044804 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 15:48:01.044804 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 15:48:01.044804 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 15:48:01.044804 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 15:48:01.044804 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 22 15:48:01.044804 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 22 15:48:01.044804 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 22 15:48:01.044804 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 22 15:48:01.046066 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 15:48:01.046066 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 15:48:01.046066 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 15:48:01.046066 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 15:48:01.046066 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 15:48:01.046066 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 15:48:01.046066 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 15:48:01.046066 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application with Lock( Aug 22 15:48:01.046066 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 15:48:01.046066 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self._enter() Aug 22 15:48:01.046066 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 15:48:01.046066 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 15:48:01.046066 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 15:48:01.046066 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self.creator() Aug 22 15:48:01.046066 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 15:48:01.046066 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application created_value = creator( Aug 22 15:48:01.046066 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 22 15:48:01.047782 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 22 15:48:01.047782 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 22 15:48:01.047782 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application Aug 22 15:48:01.047782 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 432/848] 10.222.0.15 () {56 vars in 1108 bytes} [Mon Aug 22 15:48:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 15:48:01.051670 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05344494-fa2d-4c51-9cd1-89e98e78bbf0 None admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 15:48:01.053773 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.auth.core [None req-9fe8b29f-0407-480f-ad95-d04f1985c95d None None] MFA Rules not processed for user `6b2199ae0fb0480aa917ada391f5ec54`. Rule list: `[]` (Enabled: `True`). {{(pid=81767) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 15:48:01.055431 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.fernet_utils [None req-9fe8b29f-0407-480f-ad95-d04f1985c95d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81767) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 15:48:01.115662 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=100 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 15:48:01.139586 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 417/849] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Aug 22 15:48:01 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 22 15:48:01.170182 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05344494-fa2d-4c51-9cd1-89e98e78bbf0 None admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 15:48:01.209610 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf1021c7-2f5b-4680-a33c-895ae896f4b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 15:48:01.210603 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf1021c7-2f5b-4680-a33c-895ae896f4b8 None admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:48:01.211082 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf1021c7-2f5b-4680-a33c-895ae896f4b8 None admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:48:01.211687 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf1021c7-2f5b-4680-a33c-895ae896f4b8 None admin] PATH_INFO: `/v3/domains` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:48:01.212820 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf1021c7-2f5b-4680-a33c-895ae896f4b8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81768) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 15:48:01.213340 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf1021c7-2f5b-4680-a33c-895ae896f4b8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 15:48:01.214983 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf1021c7-2f5b-4680-a33c-895ae896f4b8 None admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 15:48:01.230365 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 433/850] 10.222.0.15 () {56 vars in 1109 bytes} [Mon Aug 22 15:48:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 15:48:01.236725 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fe8b29f-0407-480f-ad95-d04f1985c95d None None] Authenticating user token {{(pid=81767) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 15:48:01.273750 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG dbcounter [-] [81767] Writing DB stats keystone:SELECT=100 {{(pid=81767) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 15:48:01.398900 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fe8b29f-0407-480f-ad95-d04f1985c95d None None] Validating token access rules against request {{(pid=81767) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 15:48:01.439416 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5b0e221-0595-4d7a-8955-48cfaeb21948 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81767) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 15:48:01.440451 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5b0e221-0595-4d7a-8955-48cfaeb21948 None admin] REQUEST_METHOD: `GET` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:48:01.441121 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5b0e221-0595-4d7a-8955-48cfaeb21948 None admin] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:48:01.441599 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5b0e221-0595-4d7a-8955-48cfaeb21948 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:48:01.449501 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5b0e221-0595-4d7a-8955-48cfaeb21948 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 15:48:01.451621 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5b0e221-0595-4d7a-8955-48cfaeb21948 None admin] RBAC: Authorization granted {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 15:48:01.465494 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application [None req-a5b0e221-0595-4d7a-8955-48cfaeb21948 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 22 15:48:01.465494 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 15:48:01.465494 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 15:48:01.465494 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 15:48:01.465494 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 15:48:01.465494 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 15:48:01.465494 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 15:48:01.465494 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 15:48:01.465494 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 15:48:01.465494 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 15:48:01.465494 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 15:48:01.465494 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 15:48:01.465494 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 22 15:48:01.465494 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 22 15:48:01.465494 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 22 15:48:01.466443 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 22 15:48:01.466443 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 15:48:01.466443 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 15:48:01.466443 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 15:48:01.466443 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 15:48:01.466443 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 15:48:01.466443 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 15:48:01.466443 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 15:48:01.466443 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application with Lock( Aug 22 15:48:01.466443 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 15:48:01.466443 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return self._enter() Aug 22 15:48:01.466443 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 15:48:01.466443 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 15:48:01.466443 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 15:48:01.466443 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return self.creator() Aug 22 15:48:01.466443 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 15:48:01.466443 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application created_value = creator( Aug 22 15:48:01.467400 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 22 15:48:01.467400 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 22 15:48:01.467400 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 22 15:48:01.467400 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 22 15:48:01.467400 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 22 15:48:01.467400 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 22 15:48:01.467400 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 22 15:48:01.467400 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application Aug 22 15:48:01.469418 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 418/851] 10.222.0.15 () {56 vars in 1111 bytes} [Mon Aug 22 15:48:01 2022] GET /identity/v3/projects/service => generated 88 bytes in 233 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 15:48:01.475969 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf1021c7-2f5b-4680-a33c-895ae896f4b8 None admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 15:48:01.490648 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5b0e221-0595-4d7a-8955-48cfaeb21948 None admin] Authenticating user token {{(pid=81767) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 15:48:01.618384 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG dbcounter [-] [81767] Writing DB stats keystone:SELECT=100 {{(pid=81767) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 15:48:01.632692 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf1021c7-2f5b-4680-a33c-895ae896f4b8 None admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 15:48:01.634969 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5b0e221-0595-4d7a-8955-48cfaeb21948 None admin] Validating token access rules against request {{(pid=81767) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 15:48:01.657777 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=100 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 15:48:01.682543 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e80a5b1-f10c-4656-a372-0a676998d32c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81767) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 15:48:01.682848 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c65915e1-69fa-438c-953d-cb9ac9d076b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 15:48:01.683600 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e80a5b1-f10c-4656-a372-0a676998d32c None admin] REQUEST_METHOD: `GET` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:48:01.683996 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e80a5b1-f10c-4656-a372-0a676998d32c None admin] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:48:01.684318 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c65915e1-69fa-438c-953d-cb9ac9d076b1 None admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:48:01.684429 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e80a5b1-f10c-4656-a372-0a676998d32c None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:48:01.684837 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c65915e1-69fa-438c-953d-cb9ac9d076b1 None admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:48:01.685258 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e80a5b1-f10c-4656-a372-0a676998d32c None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 15:48:01.685474 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c65915e1-69fa-438c-953d-cb9ac9d076b1 None admin] PATH_INFO: `/v3/projects` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:48:01.686401 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c65915e1-69fa-438c-953d-cb9ac9d076b1 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81768) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 15:48:01.686950 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c65915e1-69fa-438c-953d-cb9ac9d076b1 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 15:48:01.687064 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e80a5b1-f10c-4656-a372-0a676998d32c None admin] RBAC: Authorization granted {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 15:48:01.688930 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c65915e1-69fa-438c-953d-cb9ac9d076b1 None admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 15:48:01.698463 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application [None req-8e80a5b1-f10c-4656-a372-0a676998d32c None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 22 15:48:01.698463 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 15:48:01.698463 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 22 15:48:01.698463 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 22 15:48:01.698463 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 22 15:48:01.698463 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 22 15:48:01.698463 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 22 15:48:01.698463 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 22 15:48:01.698463 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 22 15:48:01.698463 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application Aug 22 15:48:01.698463 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 22 15:48:01.698463 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application Aug 22 15:48:01.698463 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 15:48:01.698463 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 15:48:01.698463 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 15:48:01.698463 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 15:48:01.699741 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 15:48:01.699741 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 15:48:01.699741 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 15:48:01.699741 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 15:48:01.699741 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 15:48:01.699741 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 15:48:01.699741 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 15:48:01.699741 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 22 15:48:01.699741 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 22 15:48:01.699741 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 22 15:48:01.699741 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 22 15:48:01.699741 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 15:48:01.699741 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 15:48:01.699741 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 15:48:01.699741 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 15:48:01.699741 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 15:48:01.699741 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 15:48:01.700792 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 15:48:01.700792 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application with Lock( Aug 22 15:48:01.700792 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 15:48:01.700792 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return self._enter() Aug 22 15:48:01.700792 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 15:48:01.700792 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 15:48:01.700792 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 15:48:01.700792 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return self.creator() Aug 22 15:48:01.700792 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 15:48:01.700792 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application created_value = creator( Aug 22 15:48:01.700792 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 22 15:48:01.700792 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 22 15:48:01.700792 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 22 15:48:01.700792 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application Aug 22 15:48:01.707436 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 419/852] 10.222.0.15 () {56 vars in 1132 bytes} [Mon Aug 22 15:48:01 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 15:48:01.708870 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 434/853] 10.222.0.15 () {56 vars in 1148 bytes} [Mon Aug 22 15:48:01 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 233 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 15:48:01.709603 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e80a5b1-f10c-4656-a372-0a676998d32c None admin] Authenticating user token {{(pid=81767) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 15:48:01.715704 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c65915e1-69fa-438c-953d-cb9ac9d076b1 None admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 15:48:01.836442 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e80a5b1-f10c-4656-a372-0a676998d32c None admin] Validating token access rules against request {{(pid=81767) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 15:48:01.839635 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c65915e1-69fa-438c-953d-cb9ac9d076b1 None admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 15:48:01.878107 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21897c00-eac4-40aa-91f0-441694083db7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81767) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 15:48:01.878829 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21897c00-eac4-40aa-91f0-441694083db7 None admin] REQUEST_METHOD: `GET` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:48:01.879223 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21897c00-eac4-40aa-91f0-441694083db7 None admin] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:48:01.879605 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21897c00-eac4-40aa-91f0-441694083db7 None admin] PATH_INFO: `/v3/services` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:48:01.880421 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21897c00-eac4-40aa-91f0-441694083db7 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=81767) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 15:48:01.880814 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21897c00-eac4-40aa-91f0-441694083db7 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 15:48:01.881424 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f5d91ba-d87a-49d2-b1cc-5e4112bdd691 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 15:48:01.882199 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f5d91ba-d87a-49d2-b1cc-5e4112bdd691 None admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:48:01.882432 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21897c00-eac4-40aa-91f0-441694083db7 None admin] RBAC: Authorization granted {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 15:48:01.882780 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f5d91ba-d87a-49d2-b1cc-5e4112bdd691 None admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:48:01.883209 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f5d91ba-d87a-49d2-b1cc-5e4112bdd691 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:48:01.884210 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f5d91ba-d87a-49d2-b1cc-5e4112bdd691 None admin] RBAC: Adding query filter params (role.id=9d9848cfd22e4b089db443c350e56670, scope.project.id=c4ee0413f9ef4d4fb1f95a660babe090, user.id=6a09d128dee9438283633e51f4834643) {{(pid=81768) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 15:48:01.884744 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f5d91ba-d87a-49d2-b1cc-5e4112bdd691 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 15:48:01.886542 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f5d91ba-d87a-49d2-b1cc-5e4112bdd691 None admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 15:48:01.887734 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 420/854] 10.222.0.15 () {56 vars in 1126 bytes} [Mon Aug 22 15:48:01 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 22 15:48:01.899326 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-542fcc6f-156f-4ea1-9f34-4dfd2176c843 None None] REQUEST_METHOD: `GET` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:48:01.899326 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-542fcc6f-156f-4ea1-9f34-4dfd2176c843 None None] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:48:01.899326 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-542fcc6f-156f-4ea1-9f34-4dfd2176c843 None None] PATH_INFO: `/` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:48:01.899706 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 435/855] 10.222.0.15 () {56 vars in 1374 bytes} [Mon Aug 22 15:48:01 2022] GET /identity/v3/role_assignments?user.id=6a09d128dee9438283633e51f4834643&scope.project.id=c4ee0413f9ef4d4fb1f95a660babe090&role.id=9d9848cfd22e4b089db443c350e56670 => generated 608 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 22 15:48:01.899706 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f5d91ba-d87a-49d2-b1cc-5e4112bdd691 None admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 15:48:01.899836 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 421/856] 10.222.0.15 () {54 vars in 927 bytes} [Mon Aug 22 15:48:01 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 15:48:01.902844 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bacdff26-fd9a-4fec-af68-4f57d2076082 None None] REQUEST_METHOD: `POST` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:48:01.903112 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bacdff26-fd9a-4fec-af68-4f57d2076082 None None] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:48:01.903436 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bacdff26-fd9a-4fec-af68-4f57d2076082 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:48:01.934777 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.auth.core [None req-bacdff26-fd9a-4fec-af68-4f57d2076082 None None] MFA Rules not processed for user `6b2199ae0fb0480aa917ada391f5ec54`. Rule list: `[]` (Enabled: `True`). {{(pid=81767) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 15:48:01.936558 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.fernet_utils [None req-bacdff26-fd9a-4fec-af68-4f57d2076082 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81767) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 15:48:01.955698 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG dbcounter [-] [81767] Writing DB stats keystone:SELECT=100 {{(pid=81767) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 15:48:01.970130 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=100 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 15:48:02.043447 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 422/857] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Aug 22 15:48:01 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 22 15:48:02.051945 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f5d91ba-d87a-49d2-b1cc-5e4112bdd691 None admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 15:48:02.095700 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68c80102-1688-4c79-a7dc-df63bfbb482a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 15:48:02.096306 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c80102-1688-4c79-a7dc-df63bfbb482a None admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:48:02.096583 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c80102-1688-4c79-a7dc-df63bfbb482a None admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:48:02.096850 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c80102-1688-4c79-a7dc-df63bfbb482a None admin] PATH_INFO: `/v3/services` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:48:02.097482 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68c80102-1688-4c79-a7dc-df63bfbb482a None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=81768) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 15:48:02.097782 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68c80102-1688-4c79-a7dc-df63bfbb482a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 15:48:02.099144 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68c80102-1688-4c79-a7dc-df63bfbb482a None admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 15:48:02.104517 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 436/858] 10.222.0.15 () {56 vars in 1126 bytes} [Mon Aug 22 15:48:01 2022] GET /identity/v3/services?type=compute_legacy => generated 131 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 15:48:02.330734 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bacdff26-fd9a-4fec-af68-4f57d2076082 None None] Authenticating user token {{(pid=81767) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 15:48:02.465950 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bacdff26-fd9a-4fec-af68-4f57d2076082 None None] Validating token access rules against request {{(pid=81767) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 15:48:02.506343 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-225d1282-ff28-4401-bfab-4f4737751be1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81767) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 15:48:02.506978 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-225d1282-ff28-4401-bfab-4f4737751be1 None admin] REQUEST_METHOD: `GET` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:48:02.507251 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-225d1282-ff28-4401-bfab-4f4737751be1 None admin] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:48:02.507555 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-225d1282-ff28-4401-bfab-4f4737751be1 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:48:02.520650 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-225d1282-ff28-4401-bfab-4f4737751be1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 22 15:48:02.526359 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application [None req-225d1282-ff28-4401-bfab-4f4737751be1 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 22 15:48:02.526359 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 15:48:02.526359 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 15:48:02.526359 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 15:48:02.526359 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 15:48:02.526359 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 15:48:02.526359 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 15:48:02.526359 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 15:48:02.526359 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 15:48:02.526359 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 15:48:02.526359 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 15:48:02.526359 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 15:48:02.526359 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 22 15:48:02.526359 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 22 15:48:02.526359 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 22 15:48:02.526359 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 22 15:48:02.527359 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 22 15:48:02.527359 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 22 15:48:02.527359 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 22 15:48:02.527359 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 22 15:48:02.527359 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 15:48:02.527359 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 15:48:02.527359 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 15:48:02.527359 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 15:48:02.527359 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 15:48:02.527359 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 15:48:02.527359 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 15:48:02.527359 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application with Lock( Aug 22 15:48:02.527359 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 15:48:02.527359 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return self._enter() Aug 22 15:48:02.527359 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 15:48:02.527359 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 15:48:02.528225 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 15:48:02.528225 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return self.creator() Aug 22 15:48:02.528225 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 15:48:02.528225 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application created_value = creator( Aug 22 15:48:02.528225 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 22 15:48:02.528225 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 22 15:48:02.528225 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 22 15:48:02.528225 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 22 15:48:02.528225 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 22 15:48:02.528225 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 22 15:48:02.528225 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 22 15:48:02.528225 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application Aug 22 15:48:02.531051 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 423/859] 10.222.0.15 () {56 vars in 1102 bytes} [Mon Aug 22 15:48:02 2022] GET /identity/v3/roles/service => generated 85 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 15:48:02.541340 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68c80102-1688-4c79-a7dc-df63bfbb482a None admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 15:48:02.657855 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68c80102-1688-4c79-a7dc-df63bfbb482a None admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 15:48:02.660383 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=100 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 15:48:02.695897 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c6fb194-4ede-4960-8e23-70aac8155da3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 15:48:02.696507 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c6fb194-4ede-4960-8e23-70aac8155da3 None admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:48:02.696786 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c6fb194-4ede-4960-8e23-70aac8155da3 None admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:48:02.697598 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c6fb194-4ede-4960-8e23-70aac8155da3 None admin] PATH_INFO: `/v3/roles` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:48:02.698269 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c6fb194-4ede-4960-8e23-70aac8155da3 None admin] RBAC: Adding query filter params (name=service) {{(pid=81768) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 15:48:02.698754 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c6fb194-4ede-4960-8e23-70aac8155da3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 15:48:02.701866 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c6fb194-4ede-4960-8e23-70aac8155da3 None admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 15:48:02.709875 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 437/860] 10.222.0.15 () {56 vars in 1103 bytes} [Mon Aug 22 15:48:02 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 15:48:02.716114 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-225d1282-ff28-4401-bfab-4f4737751be1 None admin] Authenticating user token {{(pid=81767) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 15:48:02.729609 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG dbcounter [-] [81767] Writing DB stats keystone:SELECT=100 {{(pid=81767) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 15:48:02.801080 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dce89dd1-062a-426b-a9e8-4774abad1678 None None] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:48:02.801411 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dce89dd1-062a-426b-a9e8-4774abad1678 None None] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:48:02.801794 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dce89dd1-062a-426b-a9e8-4774abad1678 None None] PATH_INFO: `/` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:48:02.803382 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 438/861] 10.222.0.15 () {54 vars in 927 bytes} [Mon Aug 22 15:48:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 15:48:02.810007 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec5bc407-e73e-441d-99f2-d614c023f28b None None] REQUEST_METHOD: `POST` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:48:02.810589 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec5bc407-e73e-441d-99f2-d614c023f28b None None] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:48:02.811084 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec5bc407-e73e-441d-99f2-d614c023f28b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:48:02.832083 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.auth.core [None req-ec5bc407-e73e-441d-99f2-d614c023f28b None None] MFA Rules not processed for user `6b2199ae0fb0480aa917ada391f5ec54`. Rule list: `[]` (Enabled: `True`). {{(pid=81768) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 15:48:02.833709 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-ec5bc407-e73e-441d-99f2-d614c023f28b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 15:48:02.842711 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-225d1282-ff28-4401-bfab-4f4737751be1 None admin] Validating token access rules against request {{(pid=81767) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 15:48:02.883421 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f625dc7c-9458-4571-b916-d77d71034980 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81767) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 15:48:02.884053 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f625dc7c-9458-4571-b916-d77d71034980 None admin] REQUEST_METHOD: `GET` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:48:02.884329 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f625dc7c-9458-4571-b916-d77d71034980 None admin] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:48:02.884625 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f625dc7c-9458-4571-b916-d77d71034980 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:48:02.895593 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f625dc7c-9458-4571-b916-d77d71034980 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 15:48:02.897119 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f625dc7c-9458-4571-b916-d77d71034980 None admin] RBAC: Authorization granted {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 15:48:02.913166 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application [None req-f625dc7c-9458-4571-b916-d77d71034980 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 22 15:48:02.913166 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 15:48:02.913166 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 15:48:02.913166 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 15:48:02.913166 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 15:48:02.913166 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 15:48:02.913166 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 15:48:02.913166 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 15:48:02.913166 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 15:48:02.913166 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 15:48:02.913166 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 15:48:02.913166 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 15:48:02.913166 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 22 15:48:02.913166 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 22 15:48:02.913166 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 22 15:48:02.913166 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 22 15:48:02.914127 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 15:48:02.914127 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 15:48:02.914127 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 15:48:02.914127 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 15:48:02.914127 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 15:48:02.914127 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 15:48:02.914127 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 15:48:02.914127 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application with Lock( Aug 22 15:48:02.914127 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 15:48:02.914127 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return self._enter() Aug 22 15:48:02.914127 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 15:48:02.914127 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 15:48:02.914127 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 15:48:02.914127 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return self.creator() Aug 22 15:48:02.914127 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 15:48:02.914127 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application created_value = creator( Aug 22 15:48:02.914127 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 22 15:48:02.915013 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 22 15:48:02.915013 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 22 15:48:02.915013 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application Aug 22 15:48:02.915013 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 424/862] 10.222.0.15 () {56 vars in 1108 bytes} [Mon Aug 22 15:48:02 2022] GET /identity/v3/domains/Default => generated 87 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 15:48:02.919161 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 439/863] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Aug 22 15:48:02 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 22 15:48:02.925192 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f625dc7c-9458-4571-b916-d77d71034980 None admin] Authenticating user token {{(pid=81767) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 15:48:03.027036 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG dbcounter [-] [81767] Writing DB stats keystone:SELECT=100 {{(pid=81767) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 15:48:03.100883 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f625dc7c-9458-4571-b916-d77d71034980 None admin] Validating token access rules against request {{(pid=81767) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 15:48:03.145124 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39a1badc-37e5-4493-b21b-778a127fa5b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81767) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 15:48:03.146191 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a1badc-37e5-4493-b21b-778a127fa5b6 None admin] REQUEST_METHOD: `GET` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:48:03.146680 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a1badc-37e5-4493-b21b-778a127fa5b6 None admin] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:48:03.147376 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a1badc-37e5-4493-b21b-778a127fa5b6 None admin] PATH_INFO: `/v3/domains` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:48:03.148338 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39a1badc-37e5-4493-b21b-778a127fa5b6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81767) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 15:48:03.148858 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39a1badc-37e5-4493-b21b-778a127fa5b6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 15:48:03.150951 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39a1badc-37e5-4493-b21b-778a127fa5b6 None admin] RBAC: Authorization granted {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 15:48:03.165347 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 425/864] 10.222.0.15 () {56 vars in 1109 bytes} [Mon Aug 22 15:48:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 241 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 15:48:03.175301 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec5bc407-e73e-441d-99f2-d614c023f28b None None] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 15:48:03.249271 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39a1badc-37e5-4493-b21b-778a127fa5b6 None admin] Authenticating user token {{(pid=81767) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 15:48:03.283884 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=100 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 15:48:03.292137 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec5bc407-e73e-441d-99f2-d614c023f28b None None] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 15:48:03.350537 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6ee2835-1d3a-4851-997c-a5f6976083aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 15:48:03.351546 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6ee2835-1d3a-4851-997c-a5f6976083aa None admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:48:03.351973 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6ee2835-1d3a-4851-997c-a5f6976083aa None admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:48:03.352385 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6ee2835-1d3a-4851-997c-a5f6976083aa None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:48:03.369966 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6ee2835-1d3a-4851-997c-a5f6976083aa None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 15:48:03.372544 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6ee2835-1d3a-4851-997c-a5f6976083aa None admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 15:48:03.390791 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application [None req-b6ee2835-1d3a-4851-997c-a5f6976083aa None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 22 15:48:03.390791 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 15:48:03.390791 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 15:48:03.390791 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 15:48:03.390791 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 15:48:03.390791 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 15:48:03.390791 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 15:48:03.390791 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 15:48:03.390791 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 15:48:03.390791 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 15:48:03.390791 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 15:48:03.390791 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 15:48:03.390791 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 22 15:48:03.390791 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 22 15:48:03.390791 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 22 15:48:03.390791 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 22 15:48:03.391959 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 15:48:03.391959 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 15:48:03.391959 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 22 15:48:03.391959 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 22 15:48:03.391959 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 22 15:48:03.391959 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 22 15:48:03.391959 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 15:48:03.391959 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 15:48:03.391959 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 15:48:03.391959 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 15:48:03.391959 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 15:48:03.391959 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application with Lock( Aug 22 15:48:03.391959 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 15:48:03.391959 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self._enter() Aug 22 15:48:03.391959 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 15:48:03.391959 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 15:48:03.391959 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 15:48:03.391959 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self.creator() Aug 22 15:48:03.392873 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 15:48:03.392873 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application created_value = creator( Aug 22 15:48:03.392873 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 22 15:48:03.392873 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 22 15:48:03.392873 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 22 15:48:03.392873 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 22 15:48:03.392873 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 22 15:48:03.392873 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 22 15:48:03.392873 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 22 15:48:03.392873 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application Aug 22 15:48:03.393403 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39a1badc-37e5-4493-b21b-778a127fa5b6 None admin] Validating token access rules against request {{(pid=81767) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 15:48:03.400996 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 440/865] 10.222.0.15 () {56 vars in 1117 bytes} [Mon Aug 22 15:48:03 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 229 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 15:48:03.402104 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6ee2835-1d3a-4851-997c-a5f6976083aa None admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 15:48:03.418842 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG dbcounter [-] [81767] Writing DB stats keystone:SELECT=100 {{(pid=81767) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 15:48:03.443726 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4901b418-b0a0-44d1-91d0-d033ceaa530d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81767) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 15:48:03.444257 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4901b418-b0a0-44d1-91d0-d033ceaa530d None admin] REQUEST_METHOD: `POST` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:48:03.444533 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4901b418-b0a0-44d1-91d0-d033ceaa530d None admin] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:48:03.444801 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4901b418-b0a0-44d1-91d0-d033ceaa530d None admin] PATH_INFO: `/v3/services` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:48:03.445453 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4901b418-b0a0-44d1-91d0-d033ceaa530d None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 15:48:03.446822 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4901b418-b0a0-44d1-91d0-d033ceaa530d None admin] RBAC: Authorization granted {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 15:48:03.545612 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6ee2835-1d3a-4851-997c-a5f6976083aa None admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 15:48:03.595154 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f08f3c5-4ff2-4404-891c-45baf074b316 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 15:48:03.596044 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f08f3c5-4ff2-4404-891c-45baf074b316 None admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:48:03.596501 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f08f3c5-4ff2-4404-891c-45baf074b316 None admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:48:03.596919 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f08f3c5-4ff2-4404-891c-45baf074b316 None admin] PATH_INFO: `/v3/users` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:48:03.598056 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f08f3c5-4ff2-4404-891c-45baf074b316 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81768) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 15:48:03.598547 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f08f3c5-4ff2-4404-891c-45baf074b316 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 15:48:03.600201 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f08f3c5-4ff2-4404-891c-45baf074b316 None admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 15:48:03.611449 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 426/866] 10.222.0.15 () {60 vars in 1141 bytes} [Mon Aug 22 15:48:03 2022] POST /identity/v3/services => generated 251 bytes in 363 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 22 15:48:03.631482 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 441/867] 10.222.0.15 () {56 vars in 1149 bytes} [Mon Aug 22 15:48:03 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 22 15:48:03.638422 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4901b418-b0a0-44d1-91d0-d033ceaa530d None admin] Authenticating user token {{(pid=81767) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 15:48:03.776702 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4901b418-b0a0-44d1-91d0-d033ceaa530d None admin] Validating token access rules against request {{(pid=81767) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 15:48:03.820423 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-925acc4c-8be0-47cf-ab18-5d77255251ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81767) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 15:48:03.821046 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-925acc4c-8be0-47cf-ab18-5d77255251ec None admin] REQUEST_METHOD: `GET` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:48:03.821320 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-925acc4c-8be0-47cf-ab18-5d77255251ec None admin] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:48:03.821589 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-925acc4c-8be0-47cf-ab18-5d77255251ec None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:48:03.876937 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG dbcounter [-] [81767] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81767) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 15:48:03.905488 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-925acc4c-8be0-47cf-ab18-5d77255251ec None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 15:48:03.906943 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-925acc4c-8be0-47cf-ab18-5d77255251ec None admin] RBAC: Authorization granted {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 15:48:04.033607 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 427/868] 10.222.0.15 () {58 vars in 1282 bytes} [Mon Aug 22 15:48:03 2022] GET /identity/v3/auth/tokens => generated 1262 bytes in 397 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 22 15:48:04.039266 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f08f3c5-4ff2-4404-891c-45baf074b316 None admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 15:48:04.057927 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=100 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 15:48:04.088062 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c672713-ca61-4d80-9c17-9992b6d9bb6e None None] REQUEST_METHOD: `GET` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:48:04.088386 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c672713-ca61-4d80-9c17-9992b6d9bb6e None None] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:48:04.088668 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c672713-ca61-4d80-9c17-9992b6d9bb6e None None] PATH_INFO: `/` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:48:04.090253 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 428/869] 10.222.0.15 () {54 vars in 927 bytes} [Mon Aug 22 15:48:04 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 22 15:48:04.096755 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3e40b94-8500-4c5b-a10d-048995fcb79f None None] REQUEST_METHOD: `POST` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:48:04.097042 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3e40b94-8500-4c5b-a10d-048995fcb79f None None] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:48:04.097307 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3e40b94-8500-4c5b-a10d-048995fcb79f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:48:04.122595 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.auth.core [None req-c3e40b94-8500-4c5b-a10d-048995fcb79f None None] MFA Rules not processed for user `6b2199ae0fb0480aa917ada391f5ec54`. Rule list: `[]` (Enabled: `True`). {{(pid=81767) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 15:48:04.123849 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.fernet_utils [None req-c3e40b94-8500-4c5b-a10d-048995fcb79f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81767) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 15:48:04.163545 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f08f3c5-4ff2-4404-891c-45baf074b316 None admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 15:48:04.218392 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 429/870] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Aug 22 15:48:04 2022] POST /identity/v3/auth/tokens => generated 1262 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 22 15:48:04.220302 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e1debad-4628-4b7b-b61d-3ebcfcc32063 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 15:48:04.221748 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e1debad-4628-4b7b-b61d-3ebcfcc32063 None admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:48:04.222488 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e1debad-4628-4b7b-b61d-3ebcfcc32063 None admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:48:04.223199 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e1debad-4628-4b7b-b61d-3ebcfcc32063 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:48:04.242447 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e1debad-4628-4b7b-b61d-3ebcfcc32063 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 15:48:04.246100 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e1debad-4628-4b7b-b61d-3ebcfcc32063 None admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 15:48:04.276347 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application [None req-2e1debad-4628-4b7b-b61d-3ebcfcc32063 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 22 15:48:04.276347 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 15:48:04.276347 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 15:48:04.276347 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 15:48:04.276347 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 15:48:04.276347 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 15:48:04.276347 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 15:48:04.276347 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 15:48:04.276347 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 15:48:04.276347 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 15:48:04.276347 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 15:48:04.276347 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 15:48:04.276347 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 22 15:48:04.276347 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 22 15:48:04.276347 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 22 15:48:04.276347 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 22 15:48:04.278161 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 15:48:04.278161 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 15:48:04.278161 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 15:48:04.278161 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 15:48:04.278161 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 15:48:04.278161 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 15:48:04.278161 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 15:48:04.278161 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application with Lock( Aug 22 15:48:04.278161 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 15:48:04.278161 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self._enter() Aug 22 15:48:04.278161 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 15:48:04.278161 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 15:48:04.278161 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 15:48:04.278161 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self.creator() Aug 22 15:48:04.278161 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 15:48:04.278161 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application created_value = creator( Aug 22 15:48:04.278161 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 22 15:48:04.279955 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 22 15:48:04.279955 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 22 15:48:04.279955 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application Aug 22 15:48:04.287782 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 442/871] 10.222.0.15 () {56 vars in 1108 bytes} [Mon Aug 22 15:48:04 2022] GET /identity/v3/domains/Default => generated 87 bytes in 249 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 15:48:04.293664 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3e40b94-8500-4c5b-a10d-048995fcb79f None None] Authenticating user token {{(pid=81767) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 15:48:04.300712 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG dbcounter [-] [81767] Writing DB stats keystone:SELECT=100 {{(pid=81767) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 15:48:04.344807 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c93839-32e1-4221-96de-673f816d31cf None None] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:48:04.345086 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c93839-32e1-4221-96de-673f816d31cf None None] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:48:04.345345 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c93839-32e1-4221-96de-673f816d31cf None None] PATH_INFO: `/` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:48:04.346864 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 443/872] 10.222.0.15 () {54 vars in 927 bytes} [Mon Aug 22 15:48:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 15:48:04.353036 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f19d19ce-d44c-4e09-9834-c052f5fcdce8 None None] REQUEST_METHOD: `POST` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:48:04.353833 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f19d19ce-d44c-4e09-9834-c052f5fcdce8 None None] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:48:04.354092 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f19d19ce-d44c-4e09-9834-c052f5fcdce8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:48:04.378557 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.auth.core [None req-f19d19ce-d44c-4e09-9834-c052f5fcdce8 None None] MFA Rules not processed for user `6b2199ae0fb0480aa917ada391f5ec54`. Rule list: `[]` (Enabled: `True`). {{(pid=81768) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 15:48:04.379913 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-f19d19ce-d44c-4e09-9834-c052f5fcdce8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 15:48:04.473478 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=100 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 15:48:04.478972 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 444/873] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Aug 22 15:48:04 2022] POST /identity/v3/auth/tokens => generated 1262 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 22 15:48:04.491799 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3e40b94-8500-4c5b-a10d-048995fcb79f None None] Validating token access rules against request {{(pid=81767) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 15:48:04.531185 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cdf899b-4da2-42f0-a6de-5c8be964f260 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81767) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 15:48:04.532154 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cdf899b-4da2-42f0-a6de-5c8be964f260 None admin] REQUEST_METHOD: `GET` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:48:04.532636 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cdf899b-4da2-42f0-a6de-5c8be964f260 None admin] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:48:04.533098 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cdf899b-4da2-42f0-a6de-5c8be964f260 None admin] PATH_INFO: `/v3/domains` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:48:04.533945 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cdf899b-4da2-42f0-a6de-5c8be964f260 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81767) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 15:48:04.534429 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cdf899b-4da2-42f0-a6de-5c8be964f260 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 15:48:04.536043 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cdf899b-4da2-42f0-a6de-5c8be964f260 None admin] RBAC: Authorization granted {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 15:48:04.537501 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f19d19ce-d44c-4e09-9834-c052f5fcdce8 None None] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 15:48:04.552736 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 430/874] 10.222.0.15 () {56 vars in 1109 bytes} [Mon Aug 22 15:48:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 260 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 22 15:48:04.559415 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cdf899b-4da2-42f0-a6de-5c8be964f260 None admin] Authenticating user token {{(pid=81767) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 15:48:04.642076 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG dbcounter [-] [81767] Writing DB stats keystone:SELECT=100 {{(pid=81767) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 15:48:04.684059 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f19d19ce-d44c-4e09-9834-c052f5fcdce8 None None] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 15:48:04.696132 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cdf899b-4da2-42f0-a6de-5c8be964f260 None admin] Validating token access rules against request {{(pid=81767) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 15:48:04.727744 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fdf0fc4-af98-4b27-9bb9-e99191cdca6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 15:48:04.728381 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fdf0fc4-af98-4b27-9bb9-e99191cdca6b None admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:48:04.728723 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fdf0fc4-af98-4b27-9bb9-e99191cdca6b None admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:48:04.729012 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fdf0fc4-af98-4b27-9bb9-e99191cdca6b None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:48:04.729636 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fdf0fc4-af98-4b27-9bb9-e99191cdca6b None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 15:48:04.732061 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fdf0fc4-af98-4b27-9bb9-e99191cdca6b None admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 15:48:04.742044 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a92ea6d1-818c-4498-be70-d3f9c0afb87e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81767) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 15:48:04.742977 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a92ea6d1-818c-4498-be70-d3f9c0afb87e None admin] REQUEST_METHOD: `GET` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:48:04.743469 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a92ea6d1-818c-4498-be70-d3f9c0afb87e None admin] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:48:04.743788 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application [None req-9fdf0fc4-af98-4b27-9bb9-e99191cdca6b None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 22 15:48:04.743788 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 15:48:04.743788 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 22 15:48:04.743788 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 22 15:48:04.743788 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 22 15:48:04.743788 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 22 15:48:04.743788 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 22 15:48:04.743788 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 22 15:48:04.743788 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 22 15:48:04.743788 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application Aug 22 15:48:04.743788 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 22 15:48:04.743788 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application Aug 22 15:48:04.743788 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 15:48:04.743788 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 15:48:04.743788 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 15:48:04.743788 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 15:48:04.743788 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 15:48:04.744737 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 15:48:04.744737 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 15:48:04.744737 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 15:48:04.744737 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 15:48:04.744737 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 15:48:04.744737 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 15:48:04.744737 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 22 15:48:04.744737 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 22 15:48:04.744737 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 22 15:48:04.744737 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 22 15:48:04.744737 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 15:48:04.744737 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 15:48:04.744737 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 15:48:04.744737 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 15:48:04.744737 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 15:48:04.744737 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 15:48:04.745563 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 15:48:04.745563 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application with Lock( Aug 22 15:48:04.745563 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 15:48:04.745563 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self._enter() Aug 22 15:48:04.745563 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 15:48:04.745563 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 15:48:04.745563 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 15:48:04.745563 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self.creator() Aug 22 15:48:04.745563 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 15:48:04.745563 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application created_value = creator( Aug 22 15:48:04.745563 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 22 15:48:04.745563 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 22 15:48:04.745563 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 22 15:48:04.745563 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application Aug 22 15:48:04.747211 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a92ea6d1-818c-4498-be70-d3f9c0afb87e None admin] PATH_INFO: `/v3/projects/service` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:48:04.750888 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 445/875] 10.222.0.15 () {56 vars in 1126 bytes} [Mon Aug 22 15:48:04 2022] GET /identity/v3/services/object-store => generated 93 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 22 15:48:04.751527 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fdf0fc4-af98-4b27-9bb9-e99191cdca6b None admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 15:48:04.757247 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a92ea6d1-818c-4498-be70-d3f9c0afb87e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 15:48:04.759009 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a92ea6d1-818c-4498-be70-d3f9c0afb87e None admin] RBAC: Authorization granted {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 15:48:04.775475 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application [None req-a92ea6d1-818c-4498-be70-d3f9c0afb87e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 22 15:48:04.775475 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 15:48:04.775475 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 15:48:04.775475 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 15:48:04.775475 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 15:48:04.775475 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 15:48:04.775475 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 15:48:04.775475 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 15:48:04.775475 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 15:48:04.775475 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 15:48:04.775475 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 15:48:04.775475 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 15:48:04.775475 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 22 15:48:04.775475 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 22 15:48:04.775475 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 22 15:48:04.776835 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 22 15:48:04.776835 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 15:48:04.776835 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 15:48:04.776835 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 15:48:04.776835 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 15:48:04.776835 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 15:48:04.776835 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 15:48:04.776835 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 15:48:04.776835 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application with Lock( Aug 22 15:48:04.776835 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 15:48:04.776835 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return self._enter() Aug 22 15:48:04.776835 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 15:48:04.776835 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 15:48:04.776835 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 15:48:04.776835 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return self.creator() Aug 22 15:48:04.776835 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 15:48:04.776835 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application created_value = creator( Aug 22 15:48:04.777957 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 22 15:48:04.777957 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 22 15:48:04.777957 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 22 15:48:04.777957 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 22 15:48:04.777957 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 22 15:48:04.777957 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 22 15:48:04.777957 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 22 15:48:04.777957 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application Aug 22 15:48:04.782283 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 431/876] 10.222.0.15 () {56 vars in 1111 bytes} [Mon Aug 22 15:48:04 2022] GET /identity/v3/projects/service => generated 88 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 15:48:04.783208 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a92ea6d1-818c-4498-be70-d3f9c0afb87e None admin] Authenticating user token {{(pid=81767) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 15:48:04.838792 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=100 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 15:48:04.896147 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fdf0fc4-af98-4b27-9bb9-e99191cdca6b None admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 15:48:04.911211 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a92ea6d1-818c-4498-be70-d3f9c0afb87e None admin] Validating token access rules against request {{(pid=81767) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 15:48:04.940275 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG dbcounter [-] [81767] Writing DB stats keystone:SELECT=100 {{(pid=81767) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 15:48:04.944790 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c6548fe-ad01-42ce-8576-74af830f0255 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 15:48:04.945889 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c6548fe-ad01-42ce-8576-74af830f0255 None admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:48:04.946378 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c6548fe-ad01-42ce-8576-74af830f0255 None admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:48:04.947075 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c6548fe-ad01-42ce-8576-74af830f0255 None admin] PATH_INFO: `/v3/services` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:48:04.947926 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c6548fe-ad01-42ce-8576-74af830f0255 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81768) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 15:48:04.948698 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c6548fe-ad01-42ce-8576-74af830f0255 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 15:48:04.955172 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c6548fe-ad01-42ce-8576-74af830f0255 None admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 15:48:04.955303 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49229b51-49a3-4689-baf0-9a36174893e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81767) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 15:48:04.955303 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49229b51-49a3-4689-baf0-9a36174893e8 None admin] REQUEST_METHOD: `GET` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:48:04.955303 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49229b51-49a3-4689-baf0-9a36174893e8 None admin] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:48:04.955303 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49229b51-49a3-4689-baf0-9a36174893e8 None admin] PATH_INFO: `/v3/projects` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:48:04.956040 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49229b51-49a3-4689-baf0-9a36174893e8 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81767) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 15:48:04.956466 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 446/877] 10.222.0.15 () {56 vars in 1122 bytes} [Mon Aug 22 15:48:04 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 15:48:04.961687 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49229b51-49a3-4689-baf0-9a36174893e8 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 15:48:04.963392 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c6548fe-ad01-42ce-8576-74af830f0255 None admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 15:48:04.963717 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49229b51-49a3-4689-baf0-9a36174893e8 None admin] RBAC: Authorization granted {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 15:48:04.980495 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 432/878] 10.222.0.15 () {56 vars in 1148 bytes} [Mon Aug 22 15:48:04 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 15:48:04.981391 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49229b51-49a3-4689-baf0-9a36174893e8 None admin] Authenticating user token {{(pid=81767) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 15:48:05.096548 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49229b51-49a3-4689-baf0-9a36174893e8 None admin] Validating token access rules against request {{(pid=81767) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 15:48:05.096930 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c6548fe-ad01-42ce-8576-74af830f0255 None admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 15:48:05.121902 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=100 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 15:48:05.132665 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e5cf02f-7b78-468d-bb3c-b549ac543f74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 15:48:05.133272 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e5cf02f-7b78-468d-bb3c-b549ac543f74 None admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:48:05.133551 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e5cf02f-7b78-468d-bb3c-b549ac543f74 None admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:48:05.133838 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e5cf02f-7b78-468d-bb3c-b549ac543f74 None admin] PATH_INFO: `/v3/services` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:48:05.134439 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e5cf02f-7b78-468d-bb3c-b549ac543f74 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81768) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 15:48:05.134743 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e5cf02f-7b78-468d-bb3c-b549ac543f74 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 15:48:05.136143 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e5cf02f-7b78-468d-bb3c-b549ac543f74 None admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 15:48:05.151305 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 447/879] 10.222.0.15 () {56 vars in 1122 bytes} [Mon Aug 22 15:48:04 2022] GET /identity/v3/services?type=object-store => generated 129 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 15:48:05.151305 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e5cf02f-7b78-468d-bb3c-b549ac543f74 None admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 15:48:05.153814 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4ef0785-776b-48bb-ae85-f4ee42a4aa4f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81767) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 15:48:05.154498 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4ef0785-776b-48bb-ae85-f4ee42a4aa4f None admin] REQUEST_METHOD: `POST` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:48:05.154778 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4ef0785-776b-48bb-ae85-f4ee42a4aa4f None admin] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:48:05.155047 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4ef0785-776b-48bb-ae85-f4ee42a4aa4f None admin] PATH_INFO: `/v3/services` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:48:05.155735 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4ef0785-776b-48bb-ae85-f4ee42a4aa4f None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 15:48:05.157142 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4ef0785-776b-48bb-ae85-f4ee42a4aa4f None admin] RBAC: Authorization granted {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 15:48:05.265833 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e5cf02f-7b78-468d-bb3c-b549ac543f74 None admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 15:48:05.304356 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0b4635e-fda0-4328-be90-a849d45a4928 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 15:48:05.305022 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0b4635e-fda0-4328-be90-a849d45a4928 None admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:48:05.305302 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0b4635e-fda0-4328-be90-a849d45a4928 None admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:48:05.305567 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0b4635e-fda0-4328-be90-a849d45a4928 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:48:05.306291 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0b4635e-fda0-4328-be90-a849d45a4928 None admin] RBAC: Adding query filter params (role.id=a5fbba1dc615490a96cde27e3a102eb6, scope.project.id=c4ee0413f9ef4d4fb1f95a660babe090, user.id=058e76354dce412cb78b2875c88f2791) {{(pid=81768) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 15:48:05.306596 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0b4635e-fda0-4328-be90-a849d45a4928 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 15:48:05.308327 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0b4635e-fda0-4328-be90-a849d45a4928 None admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 15:48:05.314991 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 448/880] 10.222.0.15 () {56 vars in 1374 bytes} [Mon Aug 22 15:48:05 2022] GET /identity/v3/role_assignments?user.id=058e76354dce412cb78b2875c88f2791&scope.project.id=c4ee0413f9ef4d4fb1f95a660babe090&role.id=a5fbba1dc615490a96cde27e3a102eb6 => generated 608 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 15:48:05.380428 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 433/881] 10.222.0.15 () {60 vars in 1141 bytes} [Mon Aug 22 15:48:04 2022] POST /identity/v3/services => generated 269 bytes in 399 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 22 15:48:05.888383 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c6fbd24-5283-459a-b4ee-b4110c67e2b7 None None] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:48:05.888766 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c6fbd24-5283-459a-b4ee-b4110c67e2b7 None None] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:48:05.889269 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c6fbd24-5283-459a-b4ee-b4110c67e2b7 None None] PATH_INFO: `/` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:48:05.891194 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 449/882] 10.222.0.15 () {54 vars in 927 bytes} [Mon Aug 22 15:48:05 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 15:48:05.897477 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c40fee-2e72-4a9c-8586-1d6a894d199c None None] REQUEST_METHOD: `POST` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:48:05.898116 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c40fee-2e72-4a9c-8586-1d6a894d199c None None] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:48:05.898631 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c40fee-2e72-4a9c-8586-1d6a894d199c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:48:05.918762 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.auth.core [None req-d6c40fee-2e72-4a9c-8586-1d6a894d199c None None] MFA Rules not processed for user `6b2199ae0fb0480aa917ada391f5ec54`. Rule list: `[]` (Enabled: `True`). {{(pid=81767) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 15:48:05.920333 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.fernet_utils [None req-d6c40fee-2e72-4a9c-8586-1d6a894d199c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81767) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 15:48:05.970981 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG dbcounter [-] [81767] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81767) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 15:48:06.015071 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 434/883] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Aug 22 15:48:05 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 22 15:48:06.389003 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c6fbd24-5283-459a-b4ee-b4110c67e2b7 None None] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 15:48:06.486472 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=100 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 15:48:06.547188 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c6fbd24-5283-459a-b4ee-b4110c67e2b7 None None] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 15:48:06.590459 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdcaba4c-7ede-4490-9c0d-959e48177575 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 15:48:06.590994 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdcaba4c-7ede-4490-9c0d-959e48177575 None admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:48:06.591268 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdcaba4c-7ede-4490-9c0d-959e48177575 None admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:48:06.591564 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdcaba4c-7ede-4490-9c0d-959e48177575 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:48:06.592154 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdcaba4c-7ede-4490-9c0d-959e48177575 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 15:48:06.593549 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdcaba4c-7ede-4490-9c0d-959e48177575 None admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 15:48:06.604330 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application [None req-fdcaba4c-7ede-4490-9c0d-959e48177575 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 22 15:48:06.604330 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 15:48:06.604330 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 22 15:48:06.604330 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 22 15:48:06.604330 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 22 15:48:06.604330 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 22 15:48:06.604330 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 22 15:48:06.604330 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 22 15:48:06.604330 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 22 15:48:06.604330 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application Aug 22 15:48:06.604330 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 22 15:48:06.604330 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application Aug 22 15:48:06.604330 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 15:48:06.604330 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 15:48:06.604330 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 15:48:06.604330 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 15:48:06.604330 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 15:48:06.606323 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 15:48:06.606323 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 15:48:06.606323 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 15:48:06.606323 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 15:48:06.606323 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 15:48:06.606323 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 15:48:06.606323 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 22 15:48:06.606323 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 22 15:48:06.606323 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 22 15:48:06.606323 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 22 15:48:06.606323 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 15:48:06.606323 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 15:48:06.606323 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 15:48:06.606323 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 15:48:06.606323 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 15:48:06.606323 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 15:48:06.608072 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 15:48:06.608072 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application with Lock( Aug 22 15:48:06.608072 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 15:48:06.608072 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self._enter() Aug 22 15:48:06.608072 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 15:48:06.608072 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 15:48:06.608072 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 15:48:06.608072 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self.creator() Aug 22 15:48:06.608072 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 15:48:06.608072 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application created_value = creator( Aug 22 15:48:06.608072 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 22 15:48:06.608072 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 22 15:48:06.608072 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 22 15:48:06.608072 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application Aug 22 15:48:06.608072 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 450/884] 10.222.0.15 () {56 vars in 1114 bytes} [Mon Aug 22 15:48:06 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 15:48:06.621426 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6c40fee-2e72-4a9c-8586-1d6a894d199c None None] Authenticating user token {{(pid=81767) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 15:48:06.787332 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6c40fee-2e72-4a9c-8586-1d6a894d199c None None] Validating token access rules against request {{(pid=81767) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 15:48:06.824495 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ae4d94c-6827-4508-ae9c-42834c82171f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81767) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 15:48:06.825107 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ae4d94c-6827-4508-ae9c-42834c82171f None admin] REQUEST_METHOD: `GET` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:48:06.825383 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ae4d94c-6827-4508-ae9c-42834c82171f None admin] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:48:06.825652 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ae4d94c-6827-4508-ae9c-42834c82171f None admin] PATH_INFO: `/v3/services` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:48:06.826271 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ae4d94c-6827-4508-ae9c-42834c82171f None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=81767) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 15:48:06.826594 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ae4d94c-6827-4508-ae9c-42834c82171f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 15:48:06.828083 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ae4d94c-6827-4508-ae9c-42834c82171f None admin] RBAC: Authorization granted {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 15:48:06.833417 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 435/885] 10.222.0.15 () {56 vars in 1114 bytes} [Mon Aug 22 15:48:06 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 15:48:06.839083 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdcaba4c-7ede-4490-9c0d-959e48177575 None admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 15:48:06.955455 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdcaba4c-7ede-4490-9c0d-959e48177575 None admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 15:48:06.973780 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=100 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 15:48:06.992386 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51ed0826-ce32-442e-8f40-186ca32ae37d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 15:48:06.992994 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51ed0826-ce32-442e-8f40-186ca32ae37d None admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:48:06.993272 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51ed0826-ce32-442e-8f40-186ca32ae37d None admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:48:06.993546 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51ed0826-ce32-442e-8f40-186ca32ae37d None admin] PATH_INFO: `/v3/services` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:48:06.994201 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51ed0826-ce32-442e-8f40-186ca32ae37d None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=81768) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 15:48:06.994501 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51ed0826-ce32-442e-8f40-186ca32ae37d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 15:48:06.995899 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51ed0826-ce32-442e-8f40-186ca32ae37d None admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 15:48:07.001142 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 451/886] 10.222.0.15 () {56 vars in 1114 bytes} [Mon Aug 22 15:48:06 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 15:48:07.006896 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ae4d94c-6827-4508-ae9c-42834c82171f None admin] Authenticating user token {{(pid=81767) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 15:48:07.046806 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG dbcounter [-] [81767] Writing DB stats keystone:SELECT=100 {{(pid=81767) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 15:48:07.139615 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ae4d94c-6827-4508-ae9c-42834c82171f None admin] Validating token access rules against request {{(pid=81767) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 15:48:07.178803 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d89e9bd-5a14-4f5d-a920-f8afed843a00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81767) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 15:48:07.179865 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d89e9bd-5a14-4f5d-a920-f8afed843a00 None admin] REQUEST_METHOD: `GET` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:48:07.180366 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d89e9bd-5a14-4f5d-a920-f8afed843a00 None admin] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:48:07.180861 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d89e9bd-5a14-4f5d-a920-f8afed843a00 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:48:07.181810 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d89e9bd-5a14-4f5d-a920-f8afed843a00 None admin] RBAC: Adding query filter params (interface=public, service_id=c15c609cb3bf434bbda7301a3f0590a9, region_id=RegionOne) {{(pid=81767) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 15:48:07.182577 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d89e9bd-5a14-4f5d-a920-f8afed843a00 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 15:48:07.184225 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d89e9bd-5a14-4f5d-a920-f8afed843a00 None admin] RBAC: Authorization granted {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 15:48:07.193476 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 436/887] 10.222.0.15 () {56 vars in 1251 bytes} [Mon Aug 22 15:48:07 2022] GET /identity/v3/endpoints?service_id=c15c609cb3bf434bbda7301a3f0590a9&interface=public®ion_id=RegionOne => generated 194 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 15:48:07.199465 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51ed0826-ce32-442e-8f40-186ca32ae37d None admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 15:48:07.298258 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc1903f0-59a4-42a7-85e2-5c57702aa8a3 None None] REQUEST_METHOD: `GET` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:48:07.298635 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc1903f0-59a4-42a7-85e2-5c57702aa8a3 None None] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:48:07.298961 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc1903f0-59a4-42a7-85e2-5c57702aa8a3 None None] PATH_INFO: `/` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:48:07.300901 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 437/888] 10.222.0.15 () {54 vars in 927 bytes} [Mon Aug 22 15:48:07 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 22 15:48:07.307143 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-207dd8b0-edfb-413b-a15d-eb7639a6624d None None] REQUEST_METHOD: `POST` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:48:07.307773 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-207dd8b0-edfb-413b-a15d-eb7639a6624d None None] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:48:07.308047 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-207dd8b0-edfb-413b-a15d-eb7639a6624d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:48:07.319518 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51ed0826-ce32-442e-8f40-186ca32ae37d None admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 15:48:07.327802 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.auth.core [None req-207dd8b0-edfb-413b-a15d-eb7639a6624d None None] MFA Rules not processed for user `6b2199ae0fb0480aa917ada391f5ec54`. Rule list: `[]` (Enabled: `True`). {{(pid=81767) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 15:48:07.329242 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.fernet_utils [None req-207dd8b0-edfb-413b-a15d-eb7639a6624d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81767) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 15:48:07.359520 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-812de93f-4199-42e8-835e-859bbbd3d748 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 15:48:07.360132 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-812de93f-4199-42e8-835e-859bbbd3d748 None admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:48:07.360407 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-812de93f-4199-42e8-835e-859bbbd3d748 None admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:48:07.360680 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-812de93f-4199-42e8-835e-859bbbd3d748 None admin] PATH_INFO: `/v3/services` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:48:07.361355 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-812de93f-4199-42e8-835e-859bbbd3d748 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 15:48:07.362731 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-812de93f-4199-42e8-835e-859bbbd3d748 None admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 15:48:07.369827 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 452/889] 10.222.0.15 () {56 vars in 1087 bytes} [Mon Aug 22 15:48:07 2022] GET /identity/v3/services => generated 1268 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 22 15:48:07.413759 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 438/890] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Aug 22 15:48:07 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 22 15:48:07.589686 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-726498e0-3ecd-484f-bca4-0173bf9ef6da None None] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:48:07.590213 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-726498e0-3ecd-484f-bca4-0173bf9ef6da None None] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:48:07.590213 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-726498e0-3ecd-484f-bca4-0173bf9ef6da None None] PATH_INFO: `/` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:48:07.592129 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 453/891] 10.222.0.15 () {54 vars in 927 bytes} [Mon Aug 22 15:48:07 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 15:48:07.598178 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30bc57da-0d64-4c5e-a9d5-4f1d38d9d6ed None None] REQUEST_METHOD: `POST` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:48:07.598457 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30bc57da-0d64-4c5e-a9d5-4f1d38d9d6ed None None] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:48:07.598730 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30bc57da-0d64-4c5e-a9d5-4f1d38d9d6ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:48:07.619948 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.auth.core [None req-30bc57da-0d64-4c5e-a9d5-4f1d38d9d6ed None None] MFA Rules not processed for user `6b2199ae0fb0480aa917ada391f5ec54`. Rule list: `[]` (Enabled: `True`). {{(pid=81767) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 15:48:07.621169 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.fernet_utils [None req-30bc57da-0d64-4c5e-a9d5-4f1d38d9d6ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81767) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 15:48:07.646748 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG dbcounter [-] [81767] Writing DB stats keystone:SELECT=100 {{(pid=81767) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 15:48:07.697306 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 439/892] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Aug 22 15:48:07 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 22 15:48:07.708658 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-726498e0-3ecd-484f-bca4-0173bf9ef6da None None] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 15:48:07.764938 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=100 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 15:48:07.839255 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-726498e0-3ecd-484f-bca4-0173bf9ef6da None None] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 15:48:07.880480 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5ab1af0-aa4a-4abc-b96d-966dc4eff0ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 15:48:07.881017 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5ab1af0-aa4a-4abc-b96d-966dc4eff0ad None admin] REQUEST_METHOD: `POST` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:48:07.881318 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5ab1af0-aa4a-4abc-b96d-966dc4eff0ad None admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:48:07.881593 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5ab1af0-aa4a-4abc-b96d-966dc4eff0ad None admin] PATH_INFO: `/v3/services` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:48:07.882245 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5ab1af0-aa4a-4abc-b96d-966dc4eff0ad None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 15:48:07.883927 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5ab1af0-aa4a-4abc-b96d-966dc4eff0ad None admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 15:48:08.002055 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30bc57da-0d64-4c5e-a9d5-4f1d38d9d6ed None None] Authenticating user token {{(pid=81767) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 15:48:08.014102 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 454/893] 10.222.0.15 () {60 vars in 1141 bytes} [Mon Aug 22 15:48:07 2022] POST /identity/v3/services => generated 241 bytes in 306 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 22 15:48:08.078387 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f1e8de8-4bee-4c57-93c1-c1dda37ba583 None None] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:48:08.078800 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f1e8de8-4bee-4c57-93c1-c1dda37ba583 None None] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:48:08.078968 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f1e8de8-4bee-4c57-93c1-c1dda37ba583 None None] PATH_INFO: `/` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:48:08.080738 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 455/894] 10.222.0.15 () {54 vars in 927 bytes} [Mon Aug 22 15:48:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 22 15:48:08.088886 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12e1391f-fe16-423d-8751-bfd3d19956af None None] REQUEST_METHOD: `POST` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:48:08.089777 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12e1391f-fe16-423d-8751-bfd3d19956af None None] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:48:08.090167 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12e1391f-fe16-423d-8751-bfd3d19956af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:48:08.118557 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.auth.core [None req-12e1391f-fe16-423d-8751-bfd3d19956af None None] MFA Rules not processed for user `6b2199ae0fb0480aa917ada391f5ec54`. Rule list: `[]` (Enabled: `True`). {{(pid=81768) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 15:48:08.119876 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-12e1391f-fe16-423d-8751-bfd3d19956af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 15:48:08.154325 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30bc57da-0d64-4c5e-a9d5-4f1d38d9d6ed None None] Validating token access rules against request {{(pid=81767) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 15:48:08.214356 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9ff7d5f-35dd-4ecf-bc3f-e1b5089446c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81767) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 15:48:08.215010 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9ff7d5f-35dd-4ecf-bc3f-e1b5089446c8 None admin] REQUEST_METHOD: `GET` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:48:08.215313 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9ff7d5f-35dd-4ecf-bc3f-e1b5089446c8 None admin] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:48:08.215573 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9ff7d5f-35dd-4ecf-bc3f-e1b5089446c8 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:48:08.231506 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 456/895] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Aug 22 15:48:08 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 22 15:48:08.237668 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9ff7d5f-35dd-4ecf-bc3f-e1b5089446c8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 22 15:48:08.242997 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application [None req-e9ff7d5f-35dd-4ecf-bc3f-e1b5089446c8 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 22 15:48:08.242997 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 15:48:08.242997 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 15:48:08.242997 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 15:48:08.242997 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 15:48:08.242997 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 15:48:08.242997 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 15:48:08.242997 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 15:48:08.242997 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 15:48:08.242997 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 15:48:08.242997 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 15:48:08.242997 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 15:48:08.242997 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 22 15:48:08.242997 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 22 15:48:08.242997 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 22 15:48:08.242997 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 22 15:48:08.244078 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 22 15:48:08.244078 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 22 15:48:08.244078 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 22 15:48:08.244078 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 22 15:48:08.244078 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 15:48:08.244078 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 15:48:08.244078 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 15:48:08.244078 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 15:48:08.244078 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 15:48:08.244078 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 15:48:08.244078 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 15:48:08.244078 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application with Lock( Aug 22 15:48:08.244078 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 15:48:08.244078 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return self._enter() Aug 22 15:48:08.244078 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 15:48:08.244078 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 15:48:08.244889 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 15:48:08.244889 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return self.creator() Aug 22 15:48:08.244889 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 15:48:08.244889 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application created_value = creator( Aug 22 15:48:08.244889 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 22 15:48:08.244889 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 22 15:48:08.244889 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 22 15:48:08.244889 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 22 15:48:08.244889 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 22 15:48:08.244889 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 22 15:48:08.244889 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 22 15:48:08.244889 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application Aug 22 15:48:08.245962 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 440/896] 10.222.0.15 () {56 vars in 1120 bytes} [Mon Aug 22 15:48:08 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 244 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 22 15:48:08.252443 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12e1391f-fe16-423d-8751-bfd3d19956af None None] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 15:48:08.297574 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 15:48:08.383253 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12e1391f-fe16-423d-8751-bfd3d19956af None None] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 15:48:08.425127 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4df71c35-f0c5-49d0-a2ba-7401a3978e20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 15:48:08.425696 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4df71c35-f0c5-49d0-a2ba-7401a3978e20 None admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:48:08.425980 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4df71c35-f0c5-49d0-a2ba-7401a3978e20 None admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:48:08.426249 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4df71c35-f0c5-49d0-a2ba-7401a3978e20 None admin] PATH_INFO: `/v3/roles` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:48:08.426984 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4df71c35-f0c5-49d0-a2ba-7401a3978e20 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81768) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 15:48:08.427420 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4df71c35-f0c5-49d0-a2ba-7401a3978e20 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 15:48:08.429152 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4df71c35-f0c5-49d0-a2ba-7401a3978e20 None admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 15:48:08.440749 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 457/897] 10.222.0.15 () {56 vars in 1115 bytes} [Mon Aug 22 15:48:08 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 15:48:08.446149 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9ff7d5f-35dd-4ecf-bc3f-e1b5089446c8 None admin] Authenticating user token {{(pid=81767) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 15:48:08.464621 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG dbcounter [-] [81767] Writing DB stats keystone:SELECT=100 {{(pid=81767) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 15:48:08.548113 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4df71c35-f0c5-49d0-a2ba-7401a3978e20 None admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 15:48:08.571410 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9ff7d5f-35dd-4ecf-bc3f-e1b5089446c8 None admin] Validating token access rules against request {{(pid=81767) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 15:48:08.622420 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a79f5be4-4524-47e0-b14f-bc0442e7490b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81767) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 15:48:08.623138 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a79f5be4-4524-47e0-b14f-bc0442e7490b None admin] REQUEST_METHOD: `GET` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:48:08.623495 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a79f5be4-4524-47e0-b14f-bc0442e7490b None admin] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:48:08.623674 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a79f5be4-4524-47e0-b14f-bc0442e7490b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:48:08.638506 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a79f5be4-4524-47e0-b14f-bc0442e7490b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 15:48:08.640048 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a79f5be4-4524-47e0-b14f-bc0442e7490b None admin] RBAC: Authorization granted {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 15:48:08.659093 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application [None req-a79f5be4-4524-47e0-b14f-bc0442e7490b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 22 15:48:08.659093 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 15:48:08.659093 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 15:48:08.659093 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 15:48:08.659093 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 15:48:08.659093 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 15:48:08.659093 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 15:48:08.659093 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 15:48:08.659093 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 15:48:08.659093 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 15:48:08.659093 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 15:48:08.659093 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 15:48:08.659093 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 22 15:48:08.659093 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 22 15:48:08.659093 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 22 15:48:08.659093 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 22 15:48:08.660747 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 15:48:08.660747 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 15:48:08.660747 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 15:48:08.660747 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 15:48:08.660747 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 15:48:08.660747 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 15:48:08.660747 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 15:48:08.660747 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application with Lock( Aug 22 15:48:08.660747 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 15:48:08.660747 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return self._enter() Aug 22 15:48:08.660747 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 15:48:08.660747 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 15:48:08.660747 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 15:48:08.660747 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return self.creator() Aug 22 15:48:08.660747 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 15:48:08.660747 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application created_value = creator( Aug 22 15:48:08.660747 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 22 15:48:08.661992 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 22 15:48:08.661992 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 22 15:48:08.661992 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application Aug 22 15:48:08.666109 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 441/898] 10.222.0.15 () {56 vars in 1108 bytes} [Mon Aug 22 15:48:08 2022] GET /identity/v3/domains/Default => generated 87 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 15:48:08.667083 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a79f5be4-4524-47e0-b14f-bc0442e7490b None admin] Authenticating user token {{(pid=81767) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 15:48:08.688332 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=100 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 15:48:08.697945 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4df71c35-f0c5-49d0-a2ba-7401a3978e20 None admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 15:48:08.738472 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e894e0bd-9b3a-40a7-8937-b1919778366d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 15:48:08.739543 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e894e0bd-9b3a-40a7-8937-b1919778366d None admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:48:08.739655 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e894e0bd-9b3a-40a7-8937-b1919778366d None admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:48:08.739973 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e894e0bd-9b3a-40a7-8937-b1919778366d None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:48:08.740632 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e894e0bd-9b3a-40a7-8937-b1919778366d None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 15:48:08.742083 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e894e0bd-9b3a-40a7-8937-b1919778366d None admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 15:48:08.752640 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application [None req-e894e0bd-9b3a-40a7-8937-b1919778366d None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 22 15:48:08.752640 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 15:48:08.752640 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 22 15:48:08.752640 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 22 15:48:08.752640 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 22 15:48:08.752640 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 22 15:48:08.752640 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 22 15:48:08.752640 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 22 15:48:08.752640 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 22 15:48:08.752640 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application Aug 22 15:48:08.752640 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 22 15:48:08.752640 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application Aug 22 15:48:08.752640 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 15:48:08.752640 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 15:48:08.752640 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 15:48:08.752640 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 15:48:08.753604 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 15:48:08.753604 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 15:48:08.753604 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 15:48:08.753604 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 15:48:08.753604 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 15:48:08.753604 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 15:48:08.753604 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 15:48:08.753604 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 22 15:48:08.753604 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 22 15:48:08.753604 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 22 15:48:08.753604 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 22 15:48:08.753604 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 15:48:08.753604 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 15:48:08.753604 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 15:48:08.753604 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 15:48:08.753604 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 15:48:08.753604 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 15:48:08.759828 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 15:48:08.759828 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application with Lock( Aug 22 15:48:08.759828 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 15:48:08.759828 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self._enter() Aug 22 15:48:08.759828 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 15:48:08.759828 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 15:48:08.759828 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 15:48:08.759828 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self.creator() Aug 22 15:48:08.759828 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 15:48:08.759828 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application created_value = creator( Aug 22 15:48:08.759828 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 22 15:48:08.759828 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 22 15:48:08.759828 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 22 15:48:08.759828 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application Aug 22 15:48:08.759828 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 458/899] 10.222.0.15 () {56 vars in 1132 bytes} [Mon Aug 22 15:48:08 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 15:48:08.764697 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e894e0bd-9b3a-40a7-8937-b1919778366d None admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 15:48:08.767381 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG dbcounter [-] [81767] Writing DB stats keystone:SELECT=100 {{(pid=81767) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 15:48:08.817874 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a79f5be4-4524-47e0-b14f-bc0442e7490b None admin] Validating token access rules against request {{(pid=81767) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 15:48:08.864772 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51e71698-b1bd-4642-b82f-26e1623e8da9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81767) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 15:48:08.865704 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51e71698-b1bd-4642-b82f-26e1623e8da9 None admin] REQUEST_METHOD: `GET` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:48:08.866169 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51e71698-b1bd-4642-b82f-26e1623e8da9 None admin] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:48:08.866786 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51e71698-b1bd-4642-b82f-26e1623e8da9 None admin] PATH_INFO: `/v3/domains` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:48:08.867680 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51e71698-b1bd-4642-b82f-26e1623e8da9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81767) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 15:48:08.868151 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51e71698-b1bd-4642-b82f-26e1623e8da9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 15:48:08.869974 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51e71698-b1bd-4642-b82f-26e1623e8da9 None admin] RBAC: Authorization granted {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 15:48:08.887537 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 442/900] 10.222.0.15 () {56 vars in 1109 bytes} [Mon Aug 22 15:48:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 15:48:08.890314 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51e71698-b1bd-4642-b82f-26e1623e8da9 None admin] Authenticating user token {{(pid=81767) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 15:48:08.909357 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e894e0bd-9b3a-40a7-8937-b1919778366d None admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 15:48:08.949898 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4ae9c52-b1c9-4d52-b534-b30ed4dd28b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 15:48:08.950513 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4ae9c52-b1c9-4d52-b534-b30ed4dd28b2 None admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:48:08.950875 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4ae9c52-b1c9-4d52-b534-b30ed4dd28b2 None admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:48:08.951256 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4ae9c52-b1c9-4d52-b534-b30ed4dd28b2 None admin] PATH_INFO: `/v3/services` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:48:08.952025 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4ae9c52-b1c9-4d52-b534-b30ed4dd28b2 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=81768) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 15:48:08.952386 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4ae9c52-b1c9-4d52-b534-b30ed4dd28b2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 15:48:08.953979 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4ae9c52-b1c9-4d52-b534-b30ed4dd28b2 None admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 15:48:08.966041 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 459/901] 10.222.0.15 () {56 vars in 1126 bytes} [Mon Aug 22 15:48:08 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 15:48:08.967635 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4ae9c52-b1c9-4d52-b534-b30ed4dd28b2 None admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 15:48:08.998094 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=100 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 15:48:09.006267 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51e71698-b1bd-4642-b82f-26e1623e8da9 None admin] Validating token access rules against request {{(pid=81767) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 15:48:09.034728 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG dbcounter [-] [81767] Writing DB stats keystone:SELECT=100 {{(pid=81767) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 15:48:09.045416 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fea93308-eb66-45c8-90eb-aa6a08dcf44c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81767) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 15:48:09.046296 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fea93308-eb66-45c8-90eb-aa6a08dcf44c None admin] REQUEST_METHOD: `GET` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:48:09.046762 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fea93308-eb66-45c8-90eb-aa6a08dcf44c None admin] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:48:09.047180 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fea93308-eb66-45c8-90eb-aa6a08dcf44c None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:48:09.055814 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fea93308-eb66-45c8-90eb-aa6a08dcf44c None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 15:48:09.058425 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fea93308-eb66-45c8-90eb-aa6a08dcf44c None admin] RBAC: Authorization granted {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 15:48:09.071976 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application [None req-fea93308-eb66-45c8-90eb-aa6a08dcf44c None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 22 15:48:09.071976 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 15:48:09.071976 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 15:48:09.071976 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 15:48:09.071976 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 15:48:09.071976 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 15:48:09.071976 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 15:48:09.071976 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 15:48:09.071976 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 15:48:09.071976 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 15:48:09.071976 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 15:48:09.071976 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 15:48:09.071976 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 22 15:48:09.071976 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 22 15:48:09.071976 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 22 15:48:09.071976 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 22 15:48:09.072989 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 15:48:09.072989 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 15:48:09.072989 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 22 15:48:09.072989 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 22 15:48:09.072989 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 22 15:48:09.072989 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 22 15:48:09.072989 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 15:48:09.072989 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 15:48:09.072989 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 15:48:09.072989 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 15:48:09.072989 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 15:48:09.072989 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application with Lock( Aug 22 15:48:09.072989 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 15:48:09.072989 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return self._enter() Aug 22 15:48:09.072989 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 15:48:09.072989 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 15:48:09.072989 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 15:48:09.072989 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return self.creator() Aug 22 15:48:09.073895 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 15:48:09.073895 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application created_value = creator( Aug 22 15:48:09.073895 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 22 15:48:09.073895 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 22 15:48:09.073895 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 22 15:48:09.073895 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 22 15:48:09.073895 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 22 15:48:09.073895 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 22 15:48:09.073895 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 22 15:48:09.073895 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application Aug 22 15:48:09.081119 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 443/902] 10.222.0.15 () {56 vars in 1117 bytes} [Mon Aug 22 15:48:08 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 22 15:48:09.082697 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fea93308-eb66-45c8-90eb-aa6a08dcf44c None admin] Authenticating user token {{(pid=81767) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 15:48:09.084993 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4ae9c52-b1c9-4d52-b534-b30ed4dd28b2 None admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 15:48:09.124850 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dd7655c-4c1d-4043-98b6-e0d0e9d64205 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 15:48:09.125496 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd7655c-4c1d-4043-98b6-e0d0e9d64205 None admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:48:09.125788 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd7655c-4c1d-4043-98b6-e0d0e9d64205 None admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:48:09.126037 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd7655c-4c1d-4043-98b6-e0d0e9d64205 None admin] PATH_INFO: `/v3/services` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:48:09.126672 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dd7655c-4c1d-4043-98b6-e0d0e9d64205 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=81768) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 15:48:09.126966 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dd7655c-4c1d-4043-98b6-e0d0e9d64205 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 15:48:09.128444 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dd7655c-4c1d-4043-98b6-e0d0e9d64205 None admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 15:48:09.139189 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 460/903] 10.222.0.15 () {56 vars in 1126 bytes} [Mon Aug 22 15:48:08 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 22 15:48:09.140751 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dd7655c-4c1d-4043-98b6-e0d0e9d64205 None admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 15:48:09.197045 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fea93308-eb66-45c8-90eb-aa6a08dcf44c None admin] Validating token access rules against request {{(pid=81767) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 15:48:09.235008 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ef59201-412c-4a2d-9bde-223433915938 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81767) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 15:48:09.235982 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ef59201-412c-4a2d-9bde-223433915938 None admin] REQUEST_METHOD: `GET` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:48:09.236444 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ef59201-412c-4a2d-9bde-223433915938 None admin] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:48:09.236917 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ef59201-412c-4a2d-9bde-223433915938 None admin] PATH_INFO: `/v3/users` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:48:09.237971 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ef59201-412c-4a2d-9bde-223433915938 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81767) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 15:48:09.238464 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ef59201-412c-4a2d-9bde-223433915938 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 15:48:09.240346 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ef59201-412c-4a2d-9bde-223433915938 None admin] RBAC: Authorization granted {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 15:48:09.251367 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=100 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 15:48:09.266856 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 444/904] 10.222.0.15 () {56 vars in 1149 bytes} [Mon Aug 22 15:48:09 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 15:48:09.268719 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ef59201-412c-4a2d-9bde-223433915938 None admin] Authenticating user token {{(pid=81767) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 15:48:09.271245 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dd7655c-4c1d-4043-98b6-e0d0e9d64205 None admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 15:48:09.309993 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b7abec7-df87-45b8-a7a2-8e69acd45c46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 15:48:09.310996 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b7abec7-df87-45b8-a7a2-8e69acd45c46 None admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:48:09.312549 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b7abec7-df87-45b8-a7a2-8e69acd45c46 None admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:48:09.312850 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b7abec7-df87-45b8-a7a2-8e69acd45c46 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:48:09.313582 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b7abec7-df87-45b8-a7a2-8e69acd45c46 None admin] RBAC: Adding query filter params (interface=public, service_id=9bae78809d9a4c178392eda38e7b15f3, region_id=RegionOne) {{(pid=81768) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 15:48:09.313886 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b7abec7-df87-45b8-a7a2-8e69acd45c46 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 15:48:09.315288 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b7abec7-df87-45b8-a7a2-8e69acd45c46 None admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 15:48:09.326692 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 461/905] 10.222.0.15 () {56 vars in 1251 bytes} [Mon Aug 22 15:48:09 2022] GET /identity/v3/endpoints?service_id=9bae78809d9a4c178392eda38e7b15f3&interface=public®ion_id=RegionOne => generated 194 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 22 15:48:09.327772 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG dbcounter [-] [81767] Writing DB stats keystone:SELECT=100 {{(pid=81767) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 15:48:09.328939 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b7abec7-df87-45b8-a7a2-8e69acd45c46 None admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 15:48:09.392280 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ef59201-412c-4a2d-9bde-223433915938 None admin] Validating token access rules against request {{(pid=81767) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 15:48:09.449723 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cd8eed3-346e-461a-a5ee-1f4ef2c85764 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81767) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 15:48:09.450680 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cd8eed3-346e-461a-a5ee-1f4ef2c85764 None admin] REQUEST_METHOD: `GET` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:48:09.451136 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cd8eed3-346e-461a-a5ee-1f4ef2c85764 None admin] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:48:09.451616 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cd8eed3-346e-461a-a5ee-1f4ef2c85764 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:48:09.501156 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b7abec7-df87-45b8-a7a2-8e69acd45c46 None admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 15:48:09.537076 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cd8eed3-346e-461a-a5ee-1f4ef2c85764 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 15:48:09.538560 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cd8eed3-346e-461a-a5ee-1f4ef2c85764 None admin] RBAC: Authorization granted {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 15:48:09.539627 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea408001-8be4-4719-8cf2-29c802c42fec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 15:48:09.540369 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea408001-8be4-4719-8cf2-29c802c42fec None admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:48:09.541127 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea408001-8be4-4719-8cf2-29c802c42fec None admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:48:09.541400 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea408001-8be4-4719-8cf2-29c802c42fec None admin] PATH_INFO: `/v3/services` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:48:09.542610 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea408001-8be4-4719-8cf2-29c802c42fec None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 15:48:09.543447 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea408001-8be4-4719-8cf2-29c802c42fec None admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 15:48:09.557656 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 462/906] 10.222.0.15 () {56 vars in 1087 bytes} [Mon Aug 22 15:48:09 2022] GET /identity/v3/services => generated 1497 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 22 15:48:09.557656 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d6d8db9-fef1-4600-89d4-3880c32a360a None None] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:48:09.557656 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d6d8db9-fef1-4600-89d4-3880c32a360a None None] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:48:09.557656 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d6d8db9-fef1-4600-89d4-3880c32a360a None None] PATH_INFO: `/` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:48:09.557656 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 463/907] 10.222.0.15 () {54 vars in 927 bytes} [Mon Aug 22 15:48:09 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 15:48:09.560185 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c143453-0b9d-48bf-95a6-280427cb317f None None] REQUEST_METHOD: `POST` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:48:09.561044 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c143453-0b9d-48bf-95a6-280427cb317f None None] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:48:09.561044 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c143453-0b9d-48bf-95a6-280427cb317f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:48:09.584482 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.auth.core [None req-0c143453-0b9d-48bf-95a6-280427cb317f None None] MFA Rules not processed for user `6b2199ae0fb0480aa917ada391f5ec54`. Rule list: `[]` (Enabled: `True`). {{(pid=81768) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 15:48:09.585833 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-0c143453-0b9d-48bf-95a6-280427cb317f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 15:48:09.601055 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=100 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 15:48:09.616508 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG dbcounter [-] [81767] Writing DB stats keystone:SELECT=100 {{(pid=81767) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 15:48:09.687309 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 464/908] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Aug 22 15:48:09 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 22 15:48:09.714550 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 445/909] 10.222.0.15 () {58 vars in 1282 bytes} [Mon Aug 22 15:48:09 2022] GET /identity/v3/auth/tokens => generated 1474 bytes in 447 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 22 15:48:09.723212 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c143453-0b9d-48bf-95a6-280427cb317f None None] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 15:48:09.833191 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c143453-0b9d-48bf-95a6-280427cb317f None None] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 15:48:09.871251 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee24a43b-b531-45f7-b40a-25b570697952 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 15:48:09.876208 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee24a43b-b531-45f7-b40a-25b570697952 None admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:48:09.876208 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee24a43b-b531-45f7-b40a-25b570697952 None admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:48:09.876208 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee24a43b-b531-45f7-b40a-25b570697952 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:48:09.883742 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee24a43b-b531-45f7-b40a-25b570697952 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 15:48:09.885241 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee24a43b-b531-45f7-b40a-25b570697952 None admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 15:48:09.892721 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=100 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 15:48:09.902387 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application [None req-ee24a43b-b531-45f7-b40a-25b570697952 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 22 15:48:09.902387 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 15:48:09.902387 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 15:48:09.902387 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 15:48:09.902387 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 15:48:09.902387 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 15:48:09.902387 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 15:48:09.902387 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 15:48:09.902387 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 15:48:09.902387 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 15:48:09.902387 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 15:48:09.902387 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 15:48:09.902387 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 22 15:48:09.902387 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 22 15:48:09.902387 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 22 15:48:09.902387 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 22 15:48:09.904231 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 15:48:09.904231 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 15:48:09.904231 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 15:48:09.904231 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 15:48:09.904231 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 15:48:09.904231 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 15:48:09.904231 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 15:48:09.904231 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application with Lock( Aug 22 15:48:09.904231 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 15:48:09.904231 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self._enter() Aug 22 15:48:09.904231 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 15:48:09.904231 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 15:48:09.904231 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 15:48:09.904231 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self.creator() Aug 22 15:48:09.904231 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 15:48:09.904231 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application created_value = creator( Aug 22 15:48:09.904231 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 22 15:48:09.906739 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 22 15:48:09.906739 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 22 15:48:09.906739 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application Aug 22 15:48:09.906739 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 465/910] 10.222.0.15 () {56 vars in 1108 bytes} [Mon Aug 22 15:48:09 2022] GET /identity/v3/domains/Default => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 15:48:09.909962 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cd8eed3-346e-461a-a5ee-1f4ef2c85764 None admin] Authenticating user token {{(pid=81767) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 15:48:10.005383 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee24a43b-b531-45f7-b40a-25b570697952 None admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 15:48:10.020195 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cd8eed3-346e-461a-a5ee-1f4ef2c85764 None admin] Validating token access rules against request {{(pid=81767) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 15:48:10.065018 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c714c90-b013-4b85-804a-c0714cf53ca4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81767) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 15:48:10.065898 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c714c90-b013-4b85-804a-c0714cf53ca4 None admin] REQUEST_METHOD: `GET` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:48:10.066173 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c714c90-b013-4b85-804a-c0714cf53ca4 None admin] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:48:10.066430 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c714c90-b013-4b85-804a-c0714cf53ca4 None admin] PATH_INFO: `/v3/domains` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:48:10.067234 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c714c90-b013-4b85-804a-c0714cf53ca4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81767) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 15:48:10.067936 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c714c90-b013-4b85-804a-c0714cf53ca4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 15:48:10.069744 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c714c90-b013-4b85-804a-c0714cf53ca4 None admin] RBAC: Authorization granted {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 15:48:10.085207 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 446/911] 10.222.0.15 () {56 vars in 1109 bytes} [Mon Aug 22 15:48:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 15:48:10.093773 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c714c90-b013-4b85-804a-c0714cf53ca4 None admin] Authenticating user token {{(pid=81767) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 15:48:10.097756 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG dbcounter [-] [81767] Writing DB stats keystone:SELECT=100 {{(pid=81767) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 15:48:10.120462 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee24a43b-b531-45f7-b40a-25b570697952 None admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 15:48:10.157583 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad2d8b12-4e89-46b7-9159-3536ef66d05b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 15:48:10.158398 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad2d8b12-4e89-46b7-9159-3536ef66d05b None admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:48:10.158678 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad2d8b12-4e89-46b7-9159-3536ef66d05b None admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:48:10.158947 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad2d8b12-4e89-46b7-9159-3536ef66d05b None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:48:10.159571 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad2d8b12-4e89-46b7-9159-3536ef66d05b None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 15:48:10.160968 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad2d8b12-4e89-46b7-9159-3536ef66d05b None admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 15:48:10.171137 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application [None req-ad2d8b12-4e89-46b7-9159-3536ef66d05b None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 22 15:48:10.171137 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 15:48:10.171137 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 22 15:48:10.171137 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 22 15:48:10.171137 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 22 15:48:10.171137 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 22 15:48:10.171137 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 22 15:48:10.171137 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 22 15:48:10.171137 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 22 15:48:10.171137 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application Aug 22 15:48:10.171137 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 22 15:48:10.171137 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application Aug 22 15:48:10.171137 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 15:48:10.171137 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 15:48:10.171137 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 15:48:10.171137 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 15:48:10.171137 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 15:48:10.174844 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 15:48:10.174844 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 15:48:10.174844 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 15:48:10.174844 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 15:48:10.174844 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 15:48:10.174844 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 15:48:10.174844 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 22 15:48:10.174844 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 22 15:48:10.174844 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 22 15:48:10.174844 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 22 15:48:10.174844 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 15:48:10.174844 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 15:48:10.174844 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 15:48:10.174844 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 15:48:10.174844 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 15:48:10.174844 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 15:48:10.184828 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 15:48:10.184828 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application with Lock( Aug 22 15:48:10.184828 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 15:48:10.184828 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self._enter() Aug 22 15:48:10.184828 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 15:48:10.184828 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 15:48:10.184828 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 15:48:10.184828 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self.creator() Aug 22 15:48:10.184828 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 15:48:10.184828 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application created_value = creator( Aug 22 15:48:10.184828 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 22 15:48:10.184828 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 22 15:48:10.184828 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 22 15:48:10.184828 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application Aug 22 15:48:10.184828 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 466/912] 10.222.0.15 () {56 vars in 1114 bytes} [Mon Aug 22 15:48:10 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 22 15:48:10.187990 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad2d8b12-4e89-46b7-9159-3536ef66d05b None admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 15:48:10.204078 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c714c90-b013-4b85-804a-c0714cf53ca4 None admin] Validating token access rules against request {{(pid=81767) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 15:48:10.241540 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc1e262e-b9d8-4b38-a7b0-8d9eddf83091 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81767) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 15:48:10.242180 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc1e262e-b9d8-4b38-a7b0-8d9eddf83091 None admin] REQUEST_METHOD: `GET` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:48:10.242449 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc1e262e-b9d8-4b38-a7b0-8d9eddf83091 None admin] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:48:10.242714 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc1e262e-b9d8-4b38-a7b0-8d9eddf83091 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:48:10.250127 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc1e262e-b9d8-4b38-a7b0-8d9eddf83091 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 15:48:10.250499 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=100 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 15:48:10.251695 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc1e262e-b9d8-4b38-a7b0-8d9eddf83091 None admin] RBAC: Authorization granted {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 15:48:10.264476 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application [None req-cc1e262e-b9d8-4b38-a7b0-8d9eddf83091 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 22 15:48:10.264476 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 15:48:10.264476 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 15:48:10.264476 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 15:48:10.264476 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 15:48:10.264476 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 15:48:10.264476 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 15:48:10.264476 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 15:48:10.264476 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 15:48:10.264476 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 15:48:10.264476 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 15:48:10.264476 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 15:48:10.264476 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 22 15:48:10.264476 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 22 15:48:10.264476 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 22 15:48:10.265629 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 22 15:48:10.265629 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 15:48:10.265629 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 15:48:10.265629 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 15:48:10.265629 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 15:48:10.265629 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 15:48:10.265629 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 15:48:10.265629 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 15:48:10.265629 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application with Lock( Aug 22 15:48:10.265629 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 15:48:10.265629 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return self._enter() Aug 22 15:48:10.265629 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 15:48:10.265629 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 15:48:10.265629 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 15:48:10.265629 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return self.creator() Aug 22 15:48:10.265629 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 15:48:10.265629 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application created_value = creator( Aug 22 15:48:10.266570 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 22 15:48:10.266570 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 22 15:48:10.266570 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 22 15:48:10.266570 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 22 15:48:10.266570 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 22 15:48:10.266570 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 22 15:48:10.266570 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 22 15:48:10.266570 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application Aug 22 15:48:10.270864 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 447/913] 10.222.0.15 () {56 vars in 1111 bytes} [Mon Aug 22 15:48:10 2022] GET /identity/v3/projects/service => generated 88 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 22 15:48:10.273586 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc1e262e-b9d8-4b38-a7b0-8d9eddf83091 None admin] Authenticating user token {{(pid=81767) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 15:48:10.300192 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad2d8b12-4e89-46b7-9159-3536ef66d05b None admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 15:48:10.344091 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3af5b28d-50e2-4a2a-9462-6716632b9d79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 15:48:10.344945 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3af5b28d-50e2-4a2a-9462-6716632b9d79 None admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:48:10.345400 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3af5b28d-50e2-4a2a-9462-6716632b9d79 None admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:48:10.345843 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3af5b28d-50e2-4a2a-9462-6716632b9d79 None admin] PATH_INFO: `/v3/services` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:48:10.346737 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3af5b28d-50e2-4a2a-9462-6716632b9d79 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=81768) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 15:48:10.347212 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3af5b28d-50e2-4a2a-9462-6716632b9d79 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 15:48:10.348813 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3af5b28d-50e2-4a2a-9462-6716632b9d79 None admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 15:48:10.359781 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 467/914] 10.222.0.15 () {56 vars in 1114 bytes} [Mon Aug 22 15:48:10 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 15:48:10.361041 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3af5b28d-50e2-4a2a-9462-6716632b9d79 None admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 15:48:10.363506 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG dbcounter [-] [81767] Writing DB stats keystone:SELECT=100 {{(pid=81767) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 15:48:10.415591 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc1e262e-b9d8-4b38-a7b0-8d9eddf83091 None admin] Validating token access rules against request {{(pid=81767) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 15:48:10.452100 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bfabce8-c812-4caa-951f-de584a849f3b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81767) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 15:48:10.452953 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bfabce8-c812-4caa-951f-de584a849f3b None admin] REQUEST_METHOD: `GET` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:48:10.453086 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bfabce8-c812-4caa-951f-de584a849f3b None admin] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:48:10.453299 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bfabce8-c812-4caa-951f-de584a849f3b None admin] PATH_INFO: `/v3/projects` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:48:10.454005 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bfabce8-c812-4caa-951f-de584a849f3b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81767) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 15:48:10.454302 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bfabce8-c812-4caa-951f-de584a849f3b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 15:48:10.455785 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bfabce8-c812-4caa-951f-de584a849f3b None admin] RBAC: Authorization granted {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 15:48:10.469441 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 448/915] 10.222.0.15 () {56 vars in 1148 bytes} [Mon Aug 22 15:48:10 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 15:48:10.475894 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8754b1df-35ff-46a3-b55d-627bc923b3cc None None] REQUEST_METHOD: `GET` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:48:10.476200 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8754b1df-35ff-46a3-b55d-627bc923b3cc None None] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:48:10.476474 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8754b1df-35ff-46a3-b55d-627bc923b3cc None None] PATH_INFO: `/` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:48:10.478121 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 449/916] 10.222.0.15 () {54 vars in 927 bytes} [Mon Aug 22 15:48:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 15:48:10.478864 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8754b1df-35ff-46a3-b55d-627bc923b3cc None None] Authenticating user token {{(pid=81767) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 15:48:10.486617 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3af5b28d-50e2-4a2a-9462-6716632b9d79 None admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 15:48:10.515782 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=100 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 15:48:10.526045 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e40e34ad-cad0-48d6-a2ac-4fcd878c158f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 15:48:10.528234 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e40e34ad-cad0-48d6-a2ac-4fcd878c158f None admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:48:10.528234 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e40e34ad-cad0-48d6-a2ac-4fcd878c158f None admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:48:10.528234 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e40e34ad-cad0-48d6-a2ac-4fcd878c158f None admin] PATH_INFO: `/v3/services` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:48:10.528234 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e40e34ad-cad0-48d6-a2ac-4fcd878c158f None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=81768) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 15:48:10.528234 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e40e34ad-cad0-48d6-a2ac-4fcd878c158f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 15:48:10.529491 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e40e34ad-cad0-48d6-a2ac-4fcd878c158f None admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 15:48:10.535080 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 468/917] 10.222.0.15 () {56 vars in 1114 bytes} [Mon Aug 22 15:48:10 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 15:48:10.536395 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6165af71-9f3f-40ec-a144-72a737dfb7c2 None None] REQUEST_METHOD: `POST` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:48:10.536678 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6165af71-9f3f-40ec-a144-72a737dfb7c2 None None] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:48:10.536941 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6165af71-9f3f-40ec-a144-72a737dfb7c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:48:10.555811 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.auth.core [None req-6165af71-9f3f-40ec-a144-72a737dfb7c2 None None] MFA Rules not processed for user `6b2199ae0fb0480aa917ada391f5ec54`. Rule list: `[]` (Enabled: `True`). {{(pid=81768) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 15:48:10.557011 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-6165af71-9f3f-40ec-a144-72a737dfb7c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 15:48:10.599549 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8754b1df-35ff-46a3-b55d-627bc923b3cc None None] Validating token access rules against request {{(pid=81767) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 15:48:10.629776 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG dbcounter [-] [81767] Writing DB stats keystone:SELECT=100 {{(pid=81767) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 15:48:10.638745 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 469/918] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Aug 22 15:48:10 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 22 15:48:10.639750 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6165af71-9f3f-40ec-a144-72a737dfb7c2 None None] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 15:48:10.647596 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0451b13-9806-4a3a-bd2c-3047c3fa8644 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81767) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 15:48:10.648452 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0451b13-9806-4a3a-bd2c-3047c3fa8644 None admin] REQUEST_METHOD: `GET` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:48:10.648911 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0451b13-9806-4a3a-bd2c-3047c3fa8644 None admin] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:48:10.649363 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0451b13-9806-4a3a-bd2c-3047c3fa8644 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:48:10.651485 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0451b13-9806-4a3a-bd2c-3047c3fa8644 None admin] RBAC: Adding query filter params (role.id=806eb8803ece4a1fa1bc2487b4cd7c02, scope.project.id=c4ee0413f9ef4d4fb1f95a660babe090, user.id=058e76354dce412cb78b2875c88f2791) {{(pid=81767) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 15:48:10.652014 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0451b13-9806-4a3a-bd2c-3047c3fa8644 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 15:48:10.654540 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0451b13-9806-4a3a-bd2c-3047c3fa8644 None admin] RBAC: Authorization granted {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 15:48:10.662665 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 450/919] 10.222.0.15 () {56 vars in 1374 bytes} [Mon Aug 22 15:48:10 2022] GET /identity/v3/role_assignments?user.id=058e76354dce412cb78b2875c88f2791&scope.project.id=c4ee0413f9ef4d4fb1f95a660babe090&role.id=806eb8803ece4a1fa1bc2487b4cd7c02 => generated 259 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 15:48:10.766948 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6165af71-9f3f-40ec-a144-72a737dfb7c2 None None] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 15:48:10.791624 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=100 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 15:48:10.808455 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea80d21d-3e1e-4512-afbd-69ebbaa6e938 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 15:48:10.809070 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea80d21d-3e1e-4512-afbd-69ebbaa6e938 None admin] REQUEST_METHOD: `POST` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:48:10.809338 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea80d21d-3e1e-4512-afbd-69ebbaa6e938 None admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:48:10.809602 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea80d21d-3e1e-4512-afbd-69ebbaa6e938 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:48:10.810264 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea80d21d-3e1e-4512-afbd-69ebbaa6e938 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 15:48:10.811669 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea80d21d-3e1e-4512-afbd-69ebbaa6e938 None admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 15:48:10.922393 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 470/920] 10.222.0.15 () {60 vars in 1144 bytes} [Mon Aug 22 15:48:10 2022] POST /identity/v3/endpoints => generated 346 bytes in 283 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 22 15:48:10.928764 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0451b13-9806-4a3a-bd2c-3047c3fa8644 None admin] Authenticating user token {{(pid=81767) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 15:48:11.052318 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0451b13-9806-4a3a-bd2c-3047c3fa8644 None admin] Validating token access rules against request {{(pid=81767) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 15:48:11.098513 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4f078e3-aa92-4fc6-bae9-4a966d4cc061 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81767) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 15:48:11.099494 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4f078e3-aa92-4fc6-bae9-4a966d4cc061 None admin] REQUEST_METHOD: `GET` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:48:11.099912 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4f078e3-aa92-4fc6-bae9-4a966d4cc061 None admin] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:48:11.100318 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4f078e3-aa92-4fc6-bae9-4a966d4cc061 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:48:11.101227 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4f078e3-aa92-4fc6-bae9-4a966d4cc061 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 15:48:11.103553 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4f078e3-aa92-4fc6-bae9-4a966d4cc061 None admin] RBAC: Authorization granted {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 15:48:11.120930 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application [None req-d4f078e3-aa92-4fc6-bae9-4a966d4cc061 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 22 15:48:11.120930 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 15:48:11.120930 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 22 15:48:11.120930 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 22 15:48:11.120930 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 22 15:48:11.120930 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 22 15:48:11.120930 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 22 15:48:11.120930 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 22 15:48:11.120930 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 22 15:48:11.120930 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application Aug 22 15:48:11.120930 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 22 15:48:11.120930 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application Aug 22 15:48:11.120930 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 15:48:11.120930 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 15:48:11.120930 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 15:48:11.120930 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 15:48:11.120930 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 15:48:11.121954 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 15:48:11.121954 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 15:48:11.121954 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 15:48:11.121954 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 15:48:11.121954 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 15:48:11.121954 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 15:48:11.121954 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 22 15:48:11.121954 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 22 15:48:11.121954 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 22 15:48:11.121954 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 22 15:48:11.121954 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 15:48:11.121954 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 15:48:11.121954 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 15:48:11.121954 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 15:48:11.121954 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 15:48:11.121954 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 15:48:11.122770 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 15:48:11.122770 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application with Lock( Aug 22 15:48:11.122770 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 15:48:11.122770 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return self._enter() Aug 22 15:48:11.122770 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 15:48:11.122770 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 15:48:11.122770 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 15:48:11.122770 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return self.creator() Aug 22 15:48:11.122770 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 15:48:11.122770 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application created_value = creator( Aug 22 15:48:11.122770 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 22 15:48:11.122770 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 22 15:48:11.122770 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 22 15:48:11.122770 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application Aug 22 15:48:11.122770 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 451/921] 10.222.0.15 () {56 vars in 1126 bytes} [Mon Aug 22 15:48:10 2022] GET /identity/v3/services/object-store => generated 93 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 15:48:11.128221 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea80d21d-3e1e-4512-afbd-69ebbaa6e938 None admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 15:48:11.253820 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea80d21d-3e1e-4512-afbd-69ebbaa6e938 None admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 15:48:11.295190 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2f728db-3d13-4131-86ec-649d959e506a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 15:48:11.295839 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2f728db-3d13-4131-86ec-649d959e506a None admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:48:11.296116 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2f728db-3d13-4131-86ec-649d959e506a None admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:48:11.296412 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2f728db-3d13-4131-86ec-649d959e506a None admin] PATH_INFO: `/v3/services` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:48:11.297086 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2f728db-3d13-4131-86ec-649d959e506a None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81768) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 15:48:11.297413 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2f728db-3d13-4131-86ec-649d959e506a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 15:48:11.298919 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2f728db-3d13-4131-86ec-649d959e506a None admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 15:48:11.305334 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 471/922] 10.222.0.15 () {56 vars in 1122 bytes} [Mon Aug 22 15:48:11 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 15:48:11.311816 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4f078e3-aa92-4fc6-bae9-4a966d4cc061 None admin] Authenticating user token {{(pid=81767) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 15:48:11.377622 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG dbcounter [-] [81767] Writing DB stats keystone:SELECT=100 {{(pid=81767) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 15:48:11.432206 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4f078e3-aa92-4fc6-bae9-4a966d4cc061 None admin] Validating token access rules against request {{(pid=81767) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 15:48:11.469745 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-559c5f3a-aace-4334-a5c9-81d57e775b2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81767) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 15:48:11.470594 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-559c5f3a-aace-4334-a5c9-81d57e775b2e None admin] REQUEST_METHOD: `GET` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:48:11.471041 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-559c5f3a-aace-4334-a5c9-81d57e775b2e None admin] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:48:11.471489 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-559c5f3a-aace-4334-a5c9-81d57e775b2e None admin] PATH_INFO: `/v3/services` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:48:11.472255 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-559c5f3a-aace-4334-a5c9-81d57e775b2e None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81767) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 15:48:11.472695 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-559c5f3a-aace-4334-a5c9-81d57e775b2e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 15:48:11.474258 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-559c5f3a-aace-4334-a5c9-81d57e775b2e None admin] RBAC: Authorization granted {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 15:48:11.480476 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 452/923] 10.222.0.15 () {56 vars in 1122 bytes} [Mon Aug 22 15:48:11 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 15:48:11.491323 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2f728db-3d13-4131-86ec-649d959e506a None admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 15:48:11.533443 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 15:48:11.589953 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7da91c50-309e-4de6-8ff4-16938c12e191 None None] REQUEST_METHOD: `GET` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:48:11.590272 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7da91c50-309e-4de6-8ff4-16938c12e191 None None] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:48:11.590505 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7da91c50-309e-4de6-8ff4-16938c12e191 None None] PATH_INFO: `/` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:48:11.592242 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 453/924] 10.222.0.15 () {54 vars in 927 bytes} [Mon Aug 22 15:48:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 22 15:48:11.597206 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc50f41b-3676-4fe0-aeee-c32a9bfae628 None None] REQUEST_METHOD: `POST` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:48:11.597487 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc50f41b-3676-4fe0-aeee-c32a9bfae628 None None] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:48:11.597763 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc50f41b-3676-4fe0-aeee-c32a9bfae628 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:48:11.602005 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2f728db-3d13-4131-86ec-649d959e506a None admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 15:48:11.617082 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.auth.core [None req-cc50f41b-3676-4fe0-aeee-c32a9bfae628 None None] MFA Rules not processed for user `6b2199ae0fb0480aa917ada391f5ec54`. Rule list: `[]` (Enabled: `True`). {{(pid=81767) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 15:48:11.618278 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.fernet_utils [None req-cc50f41b-3676-4fe0-aeee-c32a9bfae628 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81767) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 15:48:11.640049 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7ccd22d-7516-4311-9203-55e1140d595f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 15:48:11.640691 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ccd22d-7516-4311-9203-55e1140d595f None admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:48:11.641066 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ccd22d-7516-4311-9203-55e1140d595f None admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:48:11.641386 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ccd22d-7516-4311-9203-55e1140d595f None admin] PATH_INFO: `/v3/endpoints` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:48:11.642239 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7ccd22d-7516-4311-9203-55e1140d595f None admin] RBAC: Adding query filter params (interface=public, service_id=6febf343b03e4ebfb02e74970d12ab1d, region_id=RegionOne) {{(pid=81768) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 15:48:11.642547 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7ccd22d-7516-4311-9203-55e1140d595f None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 15:48:11.644007 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7ccd22d-7516-4311-9203-55e1140d595f None admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 15:48:11.650824 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 472/925] 10.222.0.15 () {56 vars in 1251 bytes} [Mon Aug 22 15:48:11 2022] GET /identity/v3/endpoints?service_id=6febf343b03e4ebfb02e74970d12ab1d&interface=public®ion_id=RegionOne => generated 194 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 15:48:11.657412 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7ccd22d-7516-4311-9203-55e1140d595f None admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 15:48:11.702466 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 454/926] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Aug 22 15:48:11 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 22 15:48:11.778983 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7ccd22d-7516-4311-9203-55e1140d595f None admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 15:48:11.782034 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=100 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 15:48:11.816760 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24079948-c350-4201-8488-4aeea4205302 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 15:48:11.817723 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24079948-c350-4201-8488-4aeea4205302 None admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:48:11.818158 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24079948-c350-4201-8488-4aeea4205302 None admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:48:11.818579 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24079948-c350-4201-8488-4aeea4205302 None admin] PATH_INFO: `/v3/services` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:48:11.819410 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24079948-c350-4201-8488-4aeea4205302 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 15:48:11.820998 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24079948-c350-4201-8488-4aeea4205302 None admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 15:48:11.835474 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 473/927] 10.222.0.15 () {56 vars in 1087 bytes} [Mon Aug 22 15:48:11 2022] GET /identity/v3/services => generated 1497 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 22 15:48:11.978285 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc50f41b-3676-4fe0-aeee-c32a9bfae628 None None] Authenticating user token {{(pid=81767) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 15:48:12.022748 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG dbcounter [-] [81767] Writing DB stats keystone:SELECT=100 {{(pid=81767) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 15:48:12.082625 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc50f41b-3676-4fe0-aeee-c32a9bfae628 None None] Validating token access rules against request {{(pid=81767) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 15:48:12.121243 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ad8fccb-eb28-400b-b7a2-c6d61190cd9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81767) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 15:48:12.121887 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ad8fccb-eb28-400b-b7a2-c6d61190cd9d None admin] REQUEST_METHOD: `GET` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:48:12.122161 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ad8fccb-eb28-400b-b7a2-c6d61190cd9d None admin] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:48:12.122428 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ad8fccb-eb28-400b-b7a2-c6d61190cd9d None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:48:12.123020 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ad8fccb-eb28-400b-b7a2-c6d61190cd9d None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 15:48:12.124409 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ad8fccb-eb28-400b-b7a2-c6d61190cd9d None admin] RBAC: Authorization granted {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 15:48:12.133661 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application [None req-7ad8fccb-eb28-400b-b7a2-c6d61190cd9d None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 22 15:48:12.133661 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 15:48:12.133661 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 22 15:48:12.133661 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 22 15:48:12.133661 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 22 15:48:12.133661 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 22 15:48:12.133661 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 22 15:48:12.133661 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 22 15:48:12.133661 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 22 15:48:12.133661 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application Aug 22 15:48:12.133661 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 22 15:48:12.133661 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application Aug 22 15:48:12.133661 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 15:48:12.133661 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 15:48:12.133661 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 15:48:12.133661 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 15:48:12.134642 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 15:48:12.134642 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 15:48:12.134642 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 15:48:12.134642 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 15:48:12.134642 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 15:48:12.134642 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 15:48:12.134642 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 15:48:12.134642 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 22 15:48:12.134642 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 22 15:48:12.134642 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 22 15:48:12.134642 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 22 15:48:12.134642 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 15:48:12.134642 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 15:48:12.134642 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 15:48:12.134642 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 15:48:12.134642 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 15:48:12.134642 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 15:48:12.135953 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 15:48:12.135953 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application with Lock( Aug 22 15:48:12.135953 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 15:48:12.135953 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return self._enter() Aug 22 15:48:12.135953 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 15:48:12.135953 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 15:48:12.135953 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 15:48:12.135953 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return self.creator() Aug 22 15:48:12.135953 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 15:48:12.135953 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application created_value = creator( Aug 22 15:48:12.135953 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 22 15:48:12.135953 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 22 15:48:12.135953 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 22 15:48:12.135953 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application Aug 22 15:48:12.135953 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 455/928] 10.222.0.15 () {56 vars in 1132 bytes} [Mon Aug 22 15:48:11 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 15:48:12.141149 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24079948-c350-4201-8488-4aeea4205302 None admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 15:48:12.253773 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24079948-c350-4201-8488-4aeea4205302 None admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 15:48:12.290910 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-769d6d47-3908-416d-a758-691314347506 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 15:48:12.291544 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-769d6d47-3908-416d-a758-691314347506 None admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:48:12.291817 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-769d6d47-3908-416d-a758-691314347506 None admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:48:12.292134 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-769d6d47-3908-416d-a758-691314347506 None admin] PATH_INFO: `/v3/services` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:48:12.292764 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-769d6d47-3908-416d-a758-691314347506 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=81768) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 15:48:12.293060 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-769d6d47-3908-416d-a758-691314347506 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 15:48:12.294426 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-769d6d47-3908-416d-a758-691314347506 None admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 15:48:12.300010 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 474/929] 10.222.0.15 () {56 vars in 1126 bytes} [Mon Aug 22 15:48:12 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 15:48:12.305558 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ad8fccb-eb28-400b-b7a2-c6d61190cd9d None admin] Authenticating user token {{(pid=81767) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 15:48:12.425462 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ad8fccb-eb28-400b-b7a2-c6d61190cd9d None admin] Validating token access rules against request {{(pid=81767) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 15:48:12.433632 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG dbcounter [-] [81767] Writing DB stats keystone:SELECT=100 {{(pid=81767) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 15:48:12.463222 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03ec63b0-0435-4980-987e-c4f168142d9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81767) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 15:48:12.463867 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03ec63b0-0435-4980-987e-c4f168142d9b None admin] REQUEST_METHOD: `GET` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:48:12.464143 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03ec63b0-0435-4980-987e-c4f168142d9b None admin] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:48:12.464409 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03ec63b0-0435-4980-987e-c4f168142d9b None admin] PATH_INFO: `/v3/services` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:48:12.465049 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03ec63b0-0435-4980-987e-c4f168142d9b None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=81767) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 15:48:12.465349 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03ec63b0-0435-4980-987e-c4f168142d9b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 15:48:12.466770 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03ec63b0-0435-4980-987e-c4f168142d9b None admin] RBAC: Authorization granted {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 15:48:12.472714 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 456/930] 10.222.0.15 () {56 vars in 1126 bytes} [Mon Aug 22 15:48:12 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 15:48:12.478739 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-769d6d47-3908-416d-a758-691314347506 None admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 15:48:12.518156 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=100 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 15:48:12.591592 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-769d6d47-3908-416d-a758-691314347506 None admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 15:48:12.631088 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e0614ae-0523-4555-9f01-0e1febe9dee5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 15:48:12.631856 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e0614ae-0523-4555-9f01-0e1febe9dee5 None admin] REQUEST_METHOD: `POST` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:48:12.632184 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e0614ae-0523-4555-9f01-0e1febe9dee5 None admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:48:12.632486 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e0614ae-0523-4555-9f01-0e1febe9dee5 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:48:12.633282 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e0614ae-0523-4555-9f01-0e1febe9dee5 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 15:48:12.634922 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e0614ae-0523-4555-9f01-0e1febe9dee5 None admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 15:48:12.747136 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a6913c9-1c6a-4c53-ba28-66236da3a406 None None] REQUEST_METHOD: `GET` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:48:12.747842 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a6913c9-1c6a-4c53-ba28-66236da3a406 None None] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:48:12.748311 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a6913c9-1c6a-4c53-ba28-66236da3a406 None None] PATH_INFO: `/` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:48:12.750162 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 457/931] 10.222.0.15 () {54 vars in 927 bytes} [Mon Aug 22 15:48:12 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 15:48:12.755941 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52c420c2-47bf-477f-8dac-3af5bc823c3e None None] REQUEST_METHOD: `POST` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:48:12.756406 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52c420c2-47bf-477f-8dac-3af5bc823c3e None None] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:48:12.757090 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52c420c2-47bf-477f-8dac-3af5bc823c3e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:48:12.779142 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.auth.core [None req-52c420c2-47bf-477f-8dac-3af5bc823c3e None None] MFA Rules not processed for user `6b2199ae0fb0480aa917ada391f5ec54`. Rule list: `[]` (Enabled: `True`). {{(pid=81767) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 15:48:12.784334 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.fernet_utils [None req-52c420c2-47bf-477f-8dac-3af5bc823c3e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81767) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 15:48:12.787937 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 475/932] 10.222.0.15 () {60 vars in 1144 bytes} [Mon Aug 22 15:48:12 2022] POST /identity/v3/endpoints => generated 347 bytes in 309 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 22 15:48:12.884017 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 458/933] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Aug 22 15:48:12 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 22 15:48:13.106500 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee9b7026-368b-4aae-b4a3-4ed89524f7fb None None] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:48:13.107514 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee9b7026-368b-4aae-b4a3-4ed89524f7fb None None] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:48:13.108235 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee9b7026-368b-4aae-b4a3-4ed89524f7fb None None] PATH_INFO: `/` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:48:13.111049 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 476/934] 10.222.0.15 () {54 vars in 927 bytes} [Mon Aug 22 15:48:13 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 15:48:13.117458 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-374d37df-6b36-49ba-8362-1304f028acdb None None] REQUEST_METHOD: `POST` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:48:13.117991 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-374d37df-6b36-49ba-8362-1304f028acdb None None] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:48:13.117991 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-374d37df-6b36-49ba-8362-1304f028acdb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:48:13.137820 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.auth.core [None req-374d37df-6b36-49ba-8362-1304f028acdb None None] MFA Rules not processed for user `6b2199ae0fb0480aa917ada391f5ec54`. Rule list: `[]` (Enabled: `True`). {{(pid=81767) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 15:48:13.138954 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.fernet_utils [None req-374d37df-6b36-49ba-8362-1304f028acdb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81767) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 15:48:13.163837 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee9b7026-368b-4aae-b4a3-4ed89524f7fb None None] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 15:48:13.238976 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 459/935] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Aug 22 15:48:13 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 22 15:48:13.281403 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 15:48:13.298182 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee9b7026-368b-4aae-b4a3-4ed89524f7fb None None] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 15:48:13.338329 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89ee43aa-6ca5-48d8-8b5a-c8704f782d8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 15:48:13.339261 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89ee43aa-6ca5-48d8-8b5a-c8704f782d8b None admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:48:13.339763 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89ee43aa-6ca5-48d8-8b5a-c8704f782d8b None admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:48:13.340181 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89ee43aa-6ca5-48d8-8b5a-c8704f782d8b None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:48:13.351799 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89ee43aa-6ca5-48d8-8b5a-c8704f782d8b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 22 15:48:13.357361 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application [None req-89ee43aa-6ca5-48d8-8b5a-c8704f782d8b None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 22 15:48:13.357361 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 15:48:13.357361 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 15:48:13.357361 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 15:48:13.357361 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 15:48:13.357361 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 15:48:13.357361 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 15:48:13.357361 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 15:48:13.357361 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 15:48:13.357361 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 15:48:13.357361 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 15:48:13.357361 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 15:48:13.357361 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 22 15:48:13.357361 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 22 15:48:13.357361 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 22 15:48:13.357361 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 22 15:48:13.358485 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 22 15:48:13.358485 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 22 15:48:13.358485 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 22 15:48:13.358485 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 22 15:48:13.358485 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 15:48:13.358485 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 15:48:13.358485 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 15:48:13.358485 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 15:48:13.358485 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 15:48:13.358485 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 15:48:13.358485 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 15:48:13.358485 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application with Lock( Aug 22 15:48:13.358485 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 15:48:13.358485 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self._enter() Aug 22 15:48:13.358485 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 15:48:13.358485 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 15:48:13.363593 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 15:48:13.363593 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self.creator() Aug 22 15:48:13.363593 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 15:48:13.363593 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application created_value = creator( Aug 22 15:48:13.363593 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 22 15:48:13.363593 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 22 15:48:13.363593 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 22 15:48:13.363593 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 22 15:48:13.363593 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 22 15:48:13.363593 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 22 15:48:13.363593 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 22 15:48:13.363593 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application Aug 22 15:48:13.363593 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 477/936] 10.222.0.15 () {56 vars in 1120 bytes} [Mon Aug 22 15:48:13 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 15:48:13.364490 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-374d37df-6b36-49ba-8362-1304f028acdb None None] Authenticating user token {{(pid=81767) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 15:48:13.395204 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG dbcounter [-] [81767] Writing DB stats keystone:SELECT=100 {{(pid=81767) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 15:48:13.478674 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-374d37df-6b36-49ba-8362-1304f028acdb None None] Validating token access rules against request {{(pid=81767) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 15:48:13.517550 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0e45589-1a28-4988-b186-4205735ebc1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81767) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 15:48:13.518222 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0e45589-1a28-4988-b186-4205735ebc1d None admin] REQUEST_METHOD: `GET` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:48:13.518496 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0e45589-1a28-4988-b186-4205735ebc1d None admin] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:48:13.518760 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0e45589-1a28-4988-b186-4205735ebc1d None admin] PATH_INFO: `/v3/roles` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:48:13.519453 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0e45589-1a28-4988-b186-4205735ebc1d None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81767) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 15:48:13.519749 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0e45589-1a28-4988-b186-4205735ebc1d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 15:48:13.521113 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0e45589-1a28-4988-b186-4205735ebc1d None admin] RBAC: Authorization granted {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 15:48:13.529911 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 460/937] 10.222.0.15 () {56 vars in 1115 bytes} [Mon Aug 22 15:48:13 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 15:48:13.535542 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89ee43aa-6ca5-48d8-8b5a-c8704f782d8b None admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 15:48:13.565121 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0e45589-1a28-4988-b186-4205735ebc1d None admin] Authenticating user token {{(pid=81767) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 15:48:13.653186 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89ee43aa-6ca5-48d8-8b5a-c8704f782d8b None admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 15:48:13.676405 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG dbcounter [-] [81767] Writing DB stats keystone:SELECT=100 {{(pid=81767) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 15:48:13.692945 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10b7cf10-3e56-439a-99e7-e6136fcc8696 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 15:48:13.693149 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0e45589-1a28-4988-b186-4205735ebc1d None admin] Validating token access rules against request {{(pid=81767) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 15:48:13.696136 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b7cf10-3e56-439a-99e7-e6136fcc8696 None admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:48:13.696136 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b7cf10-3e56-439a-99e7-e6136fcc8696 None admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:48:13.696136 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b7cf10-3e56-439a-99e7-e6136fcc8696 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:48:13.708306 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10b7cf10-3e56-439a-99e7-e6136fcc8696 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 15:48:13.709939 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10b7cf10-3e56-439a-99e7-e6136fcc8696 None admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 15:48:13.720008 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=100 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 15:48:13.730457 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application [None req-10b7cf10-3e56-439a-99e7-e6136fcc8696 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 22 15:48:13.730457 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 15:48:13.730457 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 15:48:13.730457 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 15:48:13.730457 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 15:48:13.730457 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 15:48:13.730457 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 15:48:13.730457 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 15:48:13.730457 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 15:48:13.730457 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 15:48:13.730457 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 15:48:13.730457 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 15:48:13.730457 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 22 15:48:13.730457 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 22 15:48:13.730457 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 22 15:48:13.730457 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 22 15:48:13.738146 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 15:48:13.738146 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 15:48:13.738146 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 15:48:13.738146 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 15:48:13.738146 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 15:48:13.738146 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 15:48:13.738146 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 15:48:13.738146 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application with Lock( Aug 22 15:48:13.738146 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 15:48:13.738146 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self._enter() Aug 22 15:48:13.738146 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 15:48:13.738146 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 15:48:13.738146 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 15:48:13.738146 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self.creator() Aug 22 15:48:13.738146 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 15:48:13.738146 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application created_value = creator( Aug 22 15:48:13.738146 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 22 15:48:13.739253 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 22 15:48:13.739253 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 22 15:48:13.739253 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application Aug 22 15:48:13.739253 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 478/938] 10.222.0.15 () {56 vars in 1108 bytes} [Mon Aug 22 15:48:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 15:48:13.744840 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10b7cf10-3e56-439a-99e7-e6136fcc8696 None admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 15:48:13.749316 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c18b7ae-6fd3-4094-b940-e26f126d6aec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81767) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 15:48:13.750852 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c18b7ae-6fd3-4094-b940-e26f126d6aec None admin] REQUEST_METHOD: `POST` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:48:13.751160 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c18b7ae-6fd3-4094-b940-e26f126d6aec None admin] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:48:13.751516 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c18b7ae-6fd3-4094-b940-e26f126d6aec None admin] PATH_INFO: `/v3/projects` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:48:13.752162 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c18b7ae-6fd3-4094-b940-e26f126d6aec None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 15:48:13.753599 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c18b7ae-6fd3-4094-b940-e26f126d6aec None admin] RBAC: Authorization granted {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 15:48:13.756253 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: WARNING keystone.server.flask.common [None req-0c18b7ae-6fd3-4094-b940-e26f126d6aec None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. Aug 22 15:48:13.823644 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.sql.core [None req-0c18b7ae-6fd3-4094-b940-e26f126d6aec None admin] Conflict project: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Aug 22 15:48:13.823644 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Aug 22 15:48:13.823644 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [parameters: {'id': '045d3548659e4ff59a1c8876228707fc', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Aug 22 15:48:13.823644 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=81767) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Aug 22 15:48:13.833886 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application [None req-0c18b7ae-6fd3-4094-b940-e26f126d6aec None admin] Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 045d3548659e4ff59a1c8876228707fc.: keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 045d3548659e4ff59a1c8876228707fc. Aug 22 15:48:13.833886 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 15:48:13.833886 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Aug 22 15:48:13.833886 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 22 15:48:13.833886 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 22 15:48:13.833886 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 22 15:48:13.833886 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 22 15:48:13.833886 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application result = self._query(query) Aug 22 15:48:13.833886 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 22 15:48:13.833886 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application conn.query(q) Aug 22 15:48:13.833886 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 22 15:48:13.833886 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 22 15:48:13.833886 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 22 15:48:13.834755 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application result.read() Aug 22 15:48:13.834755 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 22 15:48:13.834755 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 22 15:48:13.834755 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 22 15:48:13.834755 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 22 15:48:13.834755 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 22 15:48:13.834755 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 22 15:48:13.834755 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 22 15:48:13.834755 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 22 15:48:13.834755 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Aug 22 15:48:13.834755 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application Aug 22 15:48:13.834755 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Aug 22 15:48:13.834755 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application Aug 22 15:48:13.834755 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 15:48:13.834755 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Aug 22 15:48:13.834755 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return method(*args, **kwargs) Aug 22 15:48:13.834755 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 241, in create_project Aug 22 15:48:13.834755 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return project_ref.to_dict() Aug 22 15:48:13.835760 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 22 15:48:13.835760 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application next(self.gen) Aug 22 15:48:13.835760 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Aug 22 15:48:13.835760 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application yield resource Aug 22 15:48:13.835760 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 22 15:48:13.835760 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application next(self.gen) Aug 22 15:48:13.835760 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Aug 22 15:48:13.835760 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application self.session.rollback() Aug 22 15:48:13.835760 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Aug 22 15:48:13.835760 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application self.force_reraise() Aug 22 15:48:13.835760 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Aug 22 15:48:13.835760 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application raise self.value Aug 22 15:48:13.835760 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Aug 22 15:48:13.835760 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Aug 22 15:48:13.835760 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Aug 22 15:48:13.835760 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application session.commit() Aug 22 15:48:13.835760 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Aug 22 15:48:13.836644 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Aug 22 15:48:13.836644 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Aug 22 15:48:13.836644 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application self._prepare_impl() Aug 22 15:48:13.836644 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Aug 22 15:48:13.836644 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application self.session.flush() Aug 22 15:48:13.836644 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Aug 22 15:48:13.836644 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application self._flush(objects) Aug 22 15:48:13.836644 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Aug 22 15:48:13.836644 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Aug 22 15:48:13.836644 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Aug 22 15:48:13.836644 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application compat.raise_( Aug 22 15:48:13.836644 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 22 15:48:13.836644 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application raise exception Aug 22 15:48:13.836644 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Aug 22 15:48:13.836644 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application flush_context.execute() Aug 22 15:48:13.836644 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Aug 22 15:48:13.836644 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application rec.execute(self) Aug 22 15:48:13.837622 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Aug 22 15:48:13.837622 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Aug 22 15:48:13.837622 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Aug 22 15:48:13.837622 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application _emit_insert_statements( Aug 22 15:48:13.837622 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Aug 22 15:48:13.837622 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application c = connection._execute_20( Aug 22 15:48:13.837622 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Aug 22 15:48:13.837622 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Aug 22 15:48:13.837622 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Aug 22 15:48:13.837622 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Aug 22 15:48:13.837622 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Aug 22 15:48:13.837622 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application ret = self._execute_context( Aug 22 15:48:13.837622 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Aug 22 15:48:13.837622 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Aug 22 15:48:13.837622 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Aug 22 15:48:13.838522 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Aug 22 15:48:13.838522 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 22 15:48:13.838522 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application raise exception Aug 22 15:48:13.838522 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Aug 22 15:48:13.838522 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 22 15:48:13.838522 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 22 15:48:13.838522 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 22 15:48:13.838522 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 22 15:48:13.838522 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application result = self._query(query) Aug 22 15:48:13.838522 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 22 15:48:13.838522 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application conn.query(q) Aug 22 15:48:13.838522 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 22 15:48:13.838522 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 22 15:48:13.838522 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 22 15:48:13.838522 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application result.read() Aug 22 15:48:13.838522 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 22 15:48:13.838522 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 22 15:48:13.841518 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 22 15:48:13.841518 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 22 15:48:13.841518 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 22 15:48:13.841518 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 22 15:48:13.841518 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 22 15:48:13.841518 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 22 15:48:13.841518 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Aug 22 15:48:13.841518 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Aug 22 15:48:13.841518 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application [parameters: {'id': '045d3548659e4ff59a1c8876228707fc', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Aug 22 15:48:13.841518 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Aug 22 15:48:13.841518 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application Aug 22 15:48:13.841518 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 22 15:48:13.841518 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application Aug 22 15:48:13.841518 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 15:48:13.841518 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 216, in create_project Aug 22 15:48:13.843622 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application ret = self.driver.create_project(project_id, project) Aug 22 15:48:13.843622 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Aug 22 15:48:13.843622 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Aug 22 15:48:13.843622 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - Duplicate entry found with name service at domain ID default. Aug 22 15:48:13.843622 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application Aug 22 15:48:13.843622 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 22 15:48:13.843622 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application Aug 22 15:48:13.843622 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 15:48:13.843622 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 15:48:13.843622 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 15:48:13.843622 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 15:48:13.843622 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 15:48:13.843622 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 15:48:13.843622 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 15:48:13.843622 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 15:48:13.843622 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 15:48:13.843622 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 15:48:13.847029 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 15:48:13.847029 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 176, in post Aug 22 15:48:13.847029 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application ref = PROVIDERS.resource_api.create_project( Aug 22 15:48:13.847029 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 15:48:13.847029 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 15:48:13.847029 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 218, in create_project Aug 22 15:48:13.847029 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application raise exception.Conflict( Aug 22 15:48:13.847029 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 045d3548659e4ff59a1c8876228707fc. Aug 22 15:48:13.847029 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application Aug 22 15:48:13.847029 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 461/939] 10.222.0.15 () {60 vars in 1140 bytes} [Mon Aug 22 15:48:13 2022] POST /identity/v3/projects => generated 262 bytes in 275 msecs (HTTP/1.1 409) 5 headers in 183 bytes (2 switches on core 0) Aug 22 15:48:13.847029 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c18b7ae-6fd3-4094-b940-e26f126d6aec None admin] Authenticating user token {{(pid=81767) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 15:48:13.889065 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10b7cf10-3e56-439a-99e7-e6136fcc8696 None admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 15:48:13.933625 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f885aab-1652-4aec-8520-31d9dcc3b300 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 15:48:13.934166 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f885aab-1652-4aec-8520-31d9dcc3b300 None admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:48:13.934452 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f885aab-1652-4aec-8520-31d9dcc3b300 None admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:48:13.934727 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f885aab-1652-4aec-8520-31d9dcc3b300 None admin] PATH_INFO: `/v3/domains` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:48:13.935421 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f885aab-1652-4aec-8520-31d9dcc3b300 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81768) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 15:48:13.935725 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f885aab-1652-4aec-8520-31d9dcc3b300 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 15:48:13.937126 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f885aab-1652-4aec-8520-31d9dcc3b300 None admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 15:48:13.950153 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 479/940] 10.222.0.15 () {56 vars in 1109 bytes} [Mon Aug 22 15:48:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 15:48:13.951525 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a81ab28a-f603-4f61-91e7-dfe36e6f5bda None None] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:48:13.951810 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a81ab28a-f603-4f61-91e7-dfe36e6f5bda None None] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:48:13.952083 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a81ab28a-f603-4f61-91e7-dfe36e6f5bda None None] PATH_INFO: `/` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:48:13.953448 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 480/941] 10.222.0.15 () {54 vars in 927 bytes} [Mon Aug 22 15:48:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 15:48:13.955828 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a81ab28a-f603-4f61-91e7-dfe36e6f5bda None None] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 15:48:13.982726 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c18b7ae-6fd3-4094-b940-e26f126d6aec None admin] Validating token access rules against request {{(pid=81767) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 15:48:14.022405 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d90a149c-f0cb-4573-a5b2-529df962d3b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81767) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 15:48:14.023176 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d90a149c-f0cb-4573-a5b2-529df962d3b6 None admin] REQUEST_METHOD: `GET` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:48:14.023478 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d90a149c-f0cb-4573-a5b2-529df962d3b6 None admin] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:48:14.023745 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d90a149c-f0cb-4573-a5b2-529df962d3b6 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:48:14.027290 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=100 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 15:48:14.031731 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d90a149c-f0cb-4573-a5b2-529df962d3b6 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 15:48:14.033370 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d90a149c-f0cb-4573-a5b2-529df962d3b6 None admin] RBAC: Authorization granted {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 15:48:14.045953 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application [None req-d90a149c-f0cb-4573-a5b2-529df962d3b6 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 22 15:48:14.045953 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 15:48:14.045953 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 15:48:14.045953 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 15:48:14.045953 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 15:48:14.045953 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 15:48:14.045953 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 15:48:14.045953 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 15:48:14.045953 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 15:48:14.045953 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 15:48:14.045953 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 15:48:14.045953 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 15:48:14.045953 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 22 15:48:14.045953 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 22 15:48:14.045953 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 22 15:48:14.048196 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 22 15:48:14.048196 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 15:48:14.048196 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 15:48:14.048196 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 15:48:14.048196 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 15:48:14.048196 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 15:48:14.048196 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 15:48:14.048196 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 15:48:14.048196 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application with Lock( Aug 22 15:48:14.048196 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 15:48:14.048196 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return self._enter() Aug 22 15:48:14.048196 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 15:48:14.048196 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 15:48:14.048196 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 15:48:14.048196 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return self.creator() Aug 22 15:48:14.048196 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 15:48:14.048196 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application created_value = creator( Aug 22 15:48:14.049109 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 22 15:48:14.049109 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 22 15:48:14.049109 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 22 15:48:14.049109 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 22 15:48:14.049109 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 22 15:48:14.049109 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 22 15:48:14.049109 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 22 15:48:14.049109 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application Aug 22 15:48:14.049109 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 462/942] 10.222.0.15 () {56 vars in 1111 bytes} [Mon Aug 22 15:48:13 2022] GET /identity/v3/projects/service => generated 88 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 22 15:48:14.049109 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b90ba862-9e6c-4891-8516-637f0af5ff85 None None] REQUEST_METHOD: `POST` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:48:14.049109 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b90ba862-9e6c-4891-8516-637f0af5ff85 None None] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:48:14.049694 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b90ba862-9e6c-4891-8516-637f0af5ff85 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:48:14.059021 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG dbcounter [-] [81767] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81767) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 15:48:14.070562 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.auth.core [None req-b90ba862-9e6c-4891-8516-637f0af5ff85 None None] MFA Rules not processed for user `6b2199ae0fb0480aa917ada391f5ec54`. Rule list: `[]` (Enabled: `True`). {{(pid=81767) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 15:48:14.072612 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.fernet_utils [None req-b90ba862-9e6c-4891-8516-637f0af5ff85 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81767) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 15:48:14.084734 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a81ab28a-f603-4f61-91e7-dfe36e6f5bda None None] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 15:48:14.126606 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07e88e37-d73c-4cec-868e-aeff1448ab95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 15:48:14.127234 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07e88e37-d73c-4cec-868e-aeff1448ab95 None admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:48:14.131309 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07e88e37-d73c-4cec-868e-aeff1448ab95 None admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:48:14.131309 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07e88e37-d73c-4cec-868e-aeff1448ab95 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:48:14.147540 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07e88e37-d73c-4cec-868e-aeff1448ab95 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 15:48:14.149437 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07e88e37-d73c-4cec-868e-aeff1448ab95 None admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 15:48:14.163399 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application [None req-07e88e37-d73c-4cec-868e-aeff1448ab95 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 22 15:48:14.163399 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 15:48:14.163399 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 15:48:14.163399 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 15:48:14.163399 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 15:48:14.163399 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 15:48:14.163399 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 15:48:14.163399 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 15:48:14.163399 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 15:48:14.163399 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 15:48:14.163399 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 15:48:14.163399 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 15:48:14.163399 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 22 15:48:14.163399 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 22 15:48:14.163399 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 22 15:48:14.163399 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 22 15:48:14.164424 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 15:48:14.164424 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 15:48:14.164424 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 22 15:48:14.164424 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 22 15:48:14.164424 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 22 15:48:14.164424 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 22 15:48:14.164424 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 15:48:14.164424 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 15:48:14.164424 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 15:48:14.164424 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 15:48:14.164424 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 15:48:14.164424 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application with Lock( Aug 22 15:48:14.164424 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 15:48:14.164424 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self._enter() Aug 22 15:48:14.164424 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 15:48:14.164424 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 15:48:14.164424 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 15:48:14.164424 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self.creator() Aug 22 15:48:14.165902 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 15:48:14.165902 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application created_value = creator( Aug 22 15:48:14.165902 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 22 15:48:14.165902 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 22 15:48:14.165902 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 22 15:48:14.165902 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 22 15:48:14.165902 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 22 15:48:14.165902 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 22 15:48:14.165902 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 22 15:48:14.165902 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application Aug 22 15:48:14.166950 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 463/943] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Aug 22 15:48:14 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 22 15:48:14.168419 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b90ba862-9e6c-4891-8516-637f0af5ff85 None None] Authenticating user token {{(pid=81767) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 15:48:14.169692 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 481/944] 10.222.0.15 () {56 vars in 1117 bytes} [Mon Aug 22 15:48:13 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 22 15:48:14.170520 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07e88e37-d73c-4cec-868e-aeff1448ab95 None admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 15:48:14.293276 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b90ba862-9e6c-4891-8516-637f0af5ff85 None None] Validating token access rules against request {{(pid=81767) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 15:48:14.314733 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07e88e37-d73c-4cec-868e-aeff1448ab95 None admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 15:48:14.331113 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7da1d2ef-2f88-49e6-9441-96609d8a6ae5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81767) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 15:48:14.331701 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7da1d2ef-2f88-49e6-9441-96609d8a6ae5 None admin] REQUEST_METHOD: `GET` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:48:14.331974 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7da1d2ef-2f88-49e6-9441-96609d8a6ae5 None admin] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:48:14.332247 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7da1d2ef-2f88-49e6-9441-96609d8a6ae5 None admin] PATH_INFO: `/v3/projects` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:48:14.332905 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7da1d2ef-2f88-49e6-9441-96609d8a6ae5 None admin] RBAC: Adding query filter params (name=service) {{(pid=81767) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 15:48:14.333198 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7da1d2ef-2f88-49e6-9441-96609d8a6ae5 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 15:48:14.334857 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7da1d2ef-2f88-49e6-9441-96609d8a6ae5 None admin] RBAC: Authorization granted {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 15:48:14.336586 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG dbcounter [-] [81767] Writing DB stats keystone:SELECT=100 {{(pid=81767) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 15:48:14.338277 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=100 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 15:48:14.354479 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f759d939-837e-4d4f-a491-2d43a8c4ac06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 15:48:14.355327 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f759d939-837e-4d4f-a491-2d43a8c4ac06 None admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:48:14.355646 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f759d939-837e-4d4f-a491-2d43a8c4ac06 None admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:48:14.355977 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f759d939-837e-4d4f-a491-2d43a8c4ac06 None admin] PATH_INFO: `/v3/users` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:48:14.356826 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f759d939-837e-4d4f-a491-2d43a8c4ac06 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81768) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 15:48:14.362361 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f759d939-837e-4d4f-a491-2d43a8c4ac06 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 15:48:14.362361 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f759d939-837e-4d4f-a491-2d43a8c4ac06 None admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 15:48:14.362536 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 464/945] 10.222.0.15 () {56 vars in 1112 bytes} [Mon Aug 22 15:48:14 2022] GET /identity/v3/projects?name=service => generated 411 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 15:48:14.388395 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 482/946] 10.222.0.15 () {56 vars in 1149 bytes} [Mon Aug 22 15:48:14 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 15:48:14.395363 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7da1d2ef-2f88-49e6-9441-96609d8a6ae5 None admin] Authenticating user token {{(pid=81767) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 15:48:14.460894 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f759d939-837e-4d4f-a491-2d43a8c4ac06 None admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 15:48:14.523449 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7da1d2ef-2f88-49e6-9441-96609d8a6ae5 None admin] Validating token access rules against request {{(pid=81767) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 15:48:14.567247 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ec729ea-497c-4b7c-b629-17f3d219dc31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81767) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 15:48:14.568509 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ec729ea-497c-4b7c-b629-17f3d219dc31 None admin] REQUEST_METHOD: `GET` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:48:14.568787 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ec729ea-497c-4b7c-b629-17f3d219dc31 None admin] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:48:14.569056 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ec729ea-497c-4b7c-b629-17f3d219dc31 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:48:14.581186 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ec729ea-497c-4b7c-b629-17f3d219dc31 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 15:48:14.582924 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ec729ea-497c-4b7c-b629-17f3d219dc31 None admin] RBAC: Authorization granted {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 15:48:14.601791 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application [None req-4ec729ea-497c-4b7c-b629-17f3d219dc31 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 22 15:48:14.601791 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 15:48:14.601791 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 15:48:14.601791 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 15:48:14.601791 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 15:48:14.601791 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 15:48:14.601791 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 15:48:14.601791 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 15:48:14.601791 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 15:48:14.601791 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 15:48:14.601791 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 15:48:14.601791 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 15:48:14.601791 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 22 15:48:14.601791 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 22 15:48:14.601791 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 22 15:48:14.601791 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 22 15:48:14.602849 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 15:48:14.602849 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 15:48:14.602849 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 15:48:14.602849 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 15:48:14.602849 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 15:48:14.602849 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 15:48:14.602849 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 15:48:14.602849 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application with Lock( Aug 22 15:48:14.602849 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 15:48:14.602849 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return self._enter() Aug 22 15:48:14.602849 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 15:48:14.602849 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 15:48:14.602849 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 15:48:14.602849 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return self.creator() Aug 22 15:48:14.602849 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 15:48:14.602849 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application created_value = creator( Aug 22 15:48:14.602849 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 22 15:48:14.604267 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 22 15:48:14.604267 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 22 15:48:14.604267 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application Aug 22 15:48:14.606420 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 465/947] 10.222.0.15 () {56 vars in 1108 bytes} [Mon Aug 22 15:48:14 2022] GET /identity/v3/domains/Default => generated 87 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 15:48:14.606881 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f759d939-837e-4d4f-a491-2d43a8c4ac06 None admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 15:48:14.613232 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ec729ea-497c-4b7c-b629-17f3d219dc31 None admin] Authenticating user token {{(pid=81767) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 15:48:14.654306 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb2f8ee6-1cd2-4e1a-ad6a-159ea2fc0c7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 15:48:14.655370 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb2f8ee6-1cd2-4e1a-ad6a-159ea2fc0c7b None admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:48:14.655968 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb2f8ee6-1cd2-4e1a-ad6a-159ea2fc0c7b None admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:48:14.656430 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb2f8ee6-1cd2-4e1a-ad6a-159ea2fc0c7b None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:48:14.657026 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb2f8ee6-1cd2-4e1a-ad6a-159ea2fc0c7b None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 15:48:14.658412 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb2f8ee6-1cd2-4e1a-ad6a-159ea2fc0c7b None admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 15:48:14.668421 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application [None req-fb2f8ee6-1cd2-4e1a-ad6a-159ea2fc0c7b None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 22 15:48:14.668421 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 15:48:14.668421 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 22 15:48:14.668421 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 22 15:48:14.668421 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 22 15:48:14.668421 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 22 15:48:14.668421 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 22 15:48:14.668421 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 22 15:48:14.668421 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 22 15:48:14.668421 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application Aug 22 15:48:14.668421 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 22 15:48:14.668421 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application Aug 22 15:48:14.668421 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 15:48:14.668421 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 15:48:14.668421 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 15:48:14.668421 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 15:48:14.668421 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 15:48:14.669475 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 15:48:14.669475 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 15:48:14.669475 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 15:48:14.669475 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 15:48:14.669475 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 15:48:14.669475 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 15:48:14.669475 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 22 15:48:14.669475 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 22 15:48:14.669475 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 22 15:48:14.669475 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 22 15:48:14.669475 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 15:48:14.669475 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 15:48:14.669475 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 15:48:14.669475 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 15:48:14.669475 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 15:48:14.669475 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 15:48:14.670854 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 15:48:14.670854 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application with Lock( Aug 22 15:48:14.670854 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 15:48:14.670854 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self._enter() Aug 22 15:48:14.670854 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 15:48:14.670854 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 15:48:14.670854 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 15:48:14.670854 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self.creator() Aug 22 15:48:14.670854 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 15:48:14.670854 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application created_value = creator( Aug 22 15:48:14.670854 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 22 15:48:14.670854 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 22 15:48:14.670854 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 22 15:48:14.670854 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application Aug 22 15:48:14.670854 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 483/948] 10.222.0.15 () {56 vars in 1126 bytes} [Mon Aug 22 15:48:14 2022] GET /identity/v3/services/object-store => generated 93 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 15:48:14.675843 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb2f8ee6-1cd2-4e1a-ad6a-159ea2fc0c7b None admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 15:48:14.679327 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG dbcounter [-] [81767] Writing DB stats keystone:SELECT=100 {{(pid=81767) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 15:48:14.725781 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=100 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 15:48:14.753382 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ec729ea-497c-4b7c-b629-17f3d219dc31 None admin] Validating token access rules against request {{(pid=81767) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 15:48:14.792675 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79c41ff8-3428-4fb7-8fb6-3f10652e91c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81767) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 15:48:14.793353 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c41ff8-3428-4fb7-8fb6-3f10652e91c0 None admin] REQUEST_METHOD: `GET` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:48:14.793614 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c41ff8-3428-4fb7-8fb6-3f10652e91c0 None admin] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:48:14.793913 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c41ff8-3428-4fb7-8fb6-3f10652e91c0 None admin] PATH_INFO: `/v3/domains` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:48:14.794547 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79c41ff8-3428-4fb7-8fb6-3f10652e91c0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81767) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 15:48:14.794850 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79c41ff8-3428-4fb7-8fb6-3f10652e91c0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 15:48:14.796265 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79c41ff8-3428-4fb7-8fb6-3f10652e91c0 None admin] RBAC: Authorization granted {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 15:48:14.797986 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb2f8ee6-1cd2-4e1a-ad6a-159ea2fc0c7b None admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 15:48:14.814130 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 466/949] 10.222.0.15 () {56 vars in 1109 bytes} [Mon Aug 22 15:48:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 22 15:48:14.815484 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79c41ff8-3428-4fb7-8fb6-3f10652e91c0 None admin] Authenticating user token {{(pid=81767) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 15:48:14.841847 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41157e8b-cf33-4342-aecf-70cb539f3bbf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 15:48:14.842809 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41157e8b-cf33-4342-aecf-70cb539f3bbf None admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:48:14.843269 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41157e8b-cf33-4342-aecf-70cb539f3bbf None admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:48:14.843795 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41157e8b-cf33-4342-aecf-70cb539f3bbf None admin] PATH_INFO: `/v3/services` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:48:14.844604 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41157e8b-cf33-4342-aecf-70cb539f3bbf None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81768) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 15:48:14.845049 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41157e8b-cf33-4342-aecf-70cb539f3bbf None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 15:48:14.846684 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41157e8b-cf33-4342-aecf-70cb539f3bbf None admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 15:48:14.853029 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 484/950] 10.222.0.15 () {56 vars in 1122 bytes} [Mon Aug 22 15:48:14 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 15:48:14.858108 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41157e8b-cf33-4342-aecf-70cb539f3bbf None admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 15:48:14.931771 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79c41ff8-3428-4fb7-8fb6-3f10652e91c0 None admin] Validating token access rules against request {{(pid=81767) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 15:48:14.940303 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG dbcounter [-] [81767] Writing DB stats keystone:SELECT=100 {{(pid=81767) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 15:48:14.974512 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a4f63bc-540b-40cb-9fba-e4ce0806f683 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81767) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 15:48:14.975159 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a4f63bc-540b-40cb-9fba-e4ce0806f683 None admin] REQUEST_METHOD: `GET` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:48:14.975468 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a4f63bc-540b-40cb-9fba-e4ce0806f683 None admin] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:48:14.975743 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a4f63bc-540b-40cb-9fba-e4ce0806f683 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:48:14.983168 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a4f63bc-540b-40cb-9fba-e4ce0806f683 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 15:48:14.983738 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41157e8b-cf33-4342-aecf-70cb539f3bbf None admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 15:48:14.984386 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a4f63bc-540b-40cb-9fba-e4ce0806f683 None admin] RBAC: Authorization granted {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 15:48:14.993922 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=100 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 15:48:14.996112 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application [None req-5a4f63bc-540b-40cb-9fba-e4ce0806f683 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 22 15:48:14.996112 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 15:48:14.996112 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 15:48:14.996112 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 15:48:14.996112 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 15:48:14.996112 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 15:48:14.996112 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 15:48:14.996112 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 15:48:14.996112 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 15:48:14.996112 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 15:48:14.996112 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 15:48:14.996112 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 15:48:14.996112 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 22 15:48:14.996112 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 22 15:48:14.996112 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 22 15:48:14.997036 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 22 15:48:14.997036 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 15:48:14.997036 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 15:48:14.997036 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 15:48:14.997036 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 15:48:14.997036 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 15:48:14.997036 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 15:48:14.997036 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 15:48:14.997036 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application with Lock( Aug 22 15:48:14.997036 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 15:48:14.997036 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return self._enter() Aug 22 15:48:14.997036 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 15:48:14.997036 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 15:48:14.997036 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 15:48:14.997036 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return self.creator() Aug 22 15:48:14.997036 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 15:48:14.997036 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application created_value = creator( Aug 22 15:48:14.997973 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 22 15:48:14.997973 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 22 15:48:14.997973 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 22 15:48:14.997973 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 22 15:48:14.997973 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 22 15:48:14.997973 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 22 15:48:14.997973 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 22 15:48:14.997973 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application Aug 22 15:48:14.997973 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 467/951] 10.222.0.15 () {56 vars in 1111 bytes} [Mon Aug 22 15:48:14 2022] GET /identity/v3/projects/service => generated 88 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 22 15:48:14.998662 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fa1731b-3efe-43fd-a5e9-a2c4976c2c89 None None] REQUEST_METHOD: `GET` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:48:14.998949 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fa1731b-3efe-43fd-a5e9-a2c4976c2c89 None None] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:48:14.999227 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fa1731b-3efe-43fd-a5e9-a2c4976c2c89 None None] PATH_INFO: `/` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:48:15.000628 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 468/952] 10.222.0.15 () {54 vars in 927 bytes} [Mon Aug 22 15:48:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 15:48:15.003051 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fa1731b-3efe-43fd-a5e9-a2c4976c2c89 None None] Authenticating user token {{(pid=81767) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 15:48:15.033000 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4e5ce9e-da4d-452f-a7b1-d76e71707651 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 15:48:15.033610 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e5ce9e-da4d-452f-a7b1-d76e71707651 None admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:48:15.033888 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e5ce9e-da4d-452f-a7b1-d76e71707651 None admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:48:15.034159 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e5ce9e-da4d-452f-a7b1-d76e71707651 None admin] PATH_INFO: `/v3/services` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:48:15.034783 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4e5ce9e-da4d-452f-a7b1-d76e71707651 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81768) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 15:48:15.035084 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4e5ce9e-da4d-452f-a7b1-d76e71707651 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 15:48:15.036507 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4e5ce9e-da4d-452f-a7b1-d76e71707651 None admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 15:48:15.042582 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 485/953] 10.222.0.15 () {56 vars in 1122 bytes} [Mon Aug 22 15:48:14 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 15:48:15.043866 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9b53a64-3e55-42ea-b31c-227b8dfbb10a None None] REQUEST_METHOD: `POST` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:48:15.044312 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9b53a64-3e55-42ea-b31c-227b8dfbb10a None None] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:48:15.044585 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9b53a64-3e55-42ea-b31c-227b8dfbb10a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:48:15.065313 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.auth.core [None req-b9b53a64-3e55-42ea-b31c-227b8dfbb10a None None] MFA Rules not processed for user `6b2199ae0fb0480aa917ada391f5ec54`. Rule list: `[]` (Enabled: `True`). {{(pid=81768) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 15:48:15.066792 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-b9b53a64-3e55-42ea-b31c-227b8dfbb10a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 15:48:15.132417 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fa1731b-3efe-43fd-a5e9-a2c4976c2c89 None None] Validating token access rules against request {{(pid=81767) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 15:48:15.140705 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 486/954] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Aug 22 15:48:15 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 22 15:48:15.141340 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9b53a64-3e55-42ea-b31c-227b8dfbb10a None None] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 15:48:15.187943 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71334f0d-44fc-4d43-95c6-6e3fe46e76bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81767) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 15:48:15.189743 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71334f0d-44fc-4d43-95c6-6e3fe46e76bc None admin] REQUEST_METHOD: `GET` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:48:15.190327 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71334f0d-44fc-4d43-95c6-6e3fe46e76bc None admin] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:48:15.190778 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71334f0d-44fc-4d43-95c6-6e3fe46e76bc None admin] PATH_INFO: `/v3/projects` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:48:15.191925 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71334f0d-44fc-4d43-95c6-6e3fe46e76bc None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81767) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 15:48:15.192403 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71334f0d-44fc-4d43-95c6-6e3fe46e76bc None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 15:48:15.194737 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71334f0d-44fc-4d43-95c6-6e3fe46e76bc None admin] RBAC: Authorization granted {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 15:48:15.215573 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 469/955] 10.222.0.15 () {56 vars in 1148 bytes} [Mon Aug 22 15:48:15 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 22 15:48:15.217819 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71334f0d-44fc-4d43-95c6-6e3fe46e76bc None admin] Authenticating user token {{(pid=81767) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 15:48:15.255587 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=100 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 15:48:15.258620 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9b53a64-3e55-42ea-b31c-227b8dfbb10a None None] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 15:48:15.259024 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG dbcounter [-] [81767] Writing DB stats keystone:SELECT=100 {{(pid=81767) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 15:48:15.297024 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe843f70-dd57-4eac-8207-5de43fd5a3dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 15:48:15.297773 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe843f70-dd57-4eac-8207-5de43fd5a3dd None admin] REQUEST_METHOD: `POST` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:48:15.298495 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe843f70-dd57-4eac-8207-5de43fd5a3dd None admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:48:15.299435 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe843f70-dd57-4eac-8207-5de43fd5a3dd None admin] PATH_INFO: `/v3/endpoints` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:48:15.300046 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe843f70-dd57-4eac-8207-5de43fd5a3dd None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 15:48:15.302048 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe843f70-dd57-4eac-8207-5de43fd5a3dd None admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 15:48:15.351029 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71334f0d-44fc-4d43-95c6-6e3fe46e76bc None admin] Validating token access rules against request {{(pid=81767) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 15:48:15.392038 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5363700f-a0e9-414f-9296-8d8877700a64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81767) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 15:48:15.392709 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5363700f-a0e9-414f-9296-8d8877700a64 None admin] REQUEST_METHOD: `PUT` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:48:15.392988 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5363700f-a0e9-414f-9296-8d8877700a64 None admin] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:48:15.393261 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5363700f-a0e9-414f-9296-8d8877700a64 None admin] PATH_INFO: `/v3/projects/c4ee0413f9ef4d4fb1f95a660babe090/users/058e76354dce412cb78b2875c88f2791/roles/806eb8803ece4a1fa1bc2487b4cd7c02` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:48:15.395466 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5363700f-a0e9-414f-9296-8d8877700a64 None admin] RBAC: Authorizing `identity:create_grant(project_id=c4ee0413f9ef4d4fb1f95a660babe090, user_id=058e76354dce412cb78b2875c88f2791, role_id=806eb8803ece4a1fa1bc2487b4cd7c02)` {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 15:48:15.399298 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5363700f-a0e9-414f-9296-8d8877700a64 None admin] RBAC: Authorization granted {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 15:48:15.407502 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 487/956] 10.222.0.15 () {60 vars in 1144 bytes} [Mon Aug 22 15:48:15 2022] POST /identity/v3/endpoints => generated 349 bytes in 266 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 22 15:48:15.431266 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe843f70-dd57-4eac-8207-5de43fd5a3dd None admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 15:48:15.484573 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 470/957] 10.222.0.15 () {58 vars in 1439 bytes} [Mon Aug 22 15:48:15 2022] PUT /identity/v3/projects/c4ee0413f9ef4d4fb1f95a660babe090/users/058e76354dce412cb78b2875c88f2791/roles/806eb8803ece4a1fa1bc2487b4cd7c02 => generated 0 bytes in 267 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 15:48:15.574231 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe843f70-dd57-4eac-8207-5de43fd5a3dd None admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 15:48:15.615061 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ed075f5-f8d9-4620-836d-d864e44a12d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 15:48:15.615738 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ed075f5-f8d9-4620-836d-d864e44a12d4 None admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:48:15.616023 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ed075f5-f8d9-4620-836d-d864e44a12d4 None admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:48:15.616285 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ed075f5-f8d9-4620-836d-d864e44a12d4 None admin] PATH_INFO: `/v3/services/compute` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:48:15.616917 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ed075f5-f8d9-4620-836d-d864e44a12d4 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 15:48:15.618444 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ed075f5-f8d9-4620-836d-d864e44a12d4 None admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 15:48:15.628466 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application [None req-9ed075f5-f8d9-4620-836d-d864e44a12d4 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Aug 22 15:48:15.628466 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 15:48:15.628466 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 22 15:48:15.628466 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 22 15:48:15.628466 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 22 15:48:15.628466 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 22 15:48:15.628466 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 22 15:48:15.628466 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 22 15:48:15.628466 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 22 15:48:15.628466 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application Aug 22 15:48:15.628466 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 22 15:48:15.628466 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application Aug 22 15:48:15.628466 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 15:48:15.628466 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 15:48:15.628466 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 15:48:15.628466 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 15:48:15.628466 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 15:48:15.629611 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 15:48:15.629611 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 15:48:15.629611 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 15:48:15.629611 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 15:48:15.629611 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 15:48:15.629611 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 15:48:15.629611 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 22 15:48:15.629611 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 22 15:48:15.629611 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 22 15:48:15.629611 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 22 15:48:15.629611 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 15:48:15.629611 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 15:48:15.629611 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 15:48:15.629611 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 15:48:15.629611 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 15:48:15.629611 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 15:48:15.630679 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 15:48:15.630679 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application with Lock( Aug 22 15:48:15.630679 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 15:48:15.630679 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self._enter() Aug 22 15:48:15.630679 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 15:48:15.630679 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 15:48:15.630679 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 15:48:15.630679 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self.creator() Aug 22 15:48:15.630679 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 15:48:15.630679 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application created_value = creator( Aug 22 15:48:15.630679 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 22 15:48:15.630679 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 22 15:48:15.630679 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 22 15:48:15.630679 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application Aug 22 15:48:15.630679 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 488/958] 10.222.0.15 () {56 vars in 1111 bytes} [Mon Aug 22 15:48:15 2022] GET /identity/v3/services/compute => generated 88 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 22 15:48:15.636394 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5363700f-a0e9-414f-9296-8d8877700a64 None admin] Authenticating user token {{(pid=81767) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 15:48:15.744323 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG dbcounter [-] [81767] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81767) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 15:48:15.764833 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5363700f-a0e9-414f-9296-8d8877700a64 None admin] Validating token access rules against request {{(pid=81767) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 15:48:15.804311 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aad7385-f71a-429b-89ac-6459e108f8f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81767) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 15:48:15.805401 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aad7385-f71a-429b-89ac-6459e108f8f5 None admin] REQUEST_METHOD: `GET` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:48:15.805876 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aad7385-f71a-429b-89ac-6459e108f8f5 None admin] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:48:15.806362 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aad7385-f71a-429b-89ac-6459e108f8f5 None admin] PATH_INFO: `/v3/services` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:48:15.807390 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aad7385-f71a-429b-89ac-6459e108f8f5 None admin] RBAC: Adding query filter params (name=compute) {{(pid=81767) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 15:48:15.808174 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aad7385-f71a-429b-89ac-6459e108f8f5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 15:48:15.810043 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aad7385-f71a-429b-89ac-6459e108f8f5 None admin] RBAC: Authorization granted {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 15:48:15.819232 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 471/959] 10.222.0.15 () {56 vars in 1112 bytes} [Mon Aug 22 15:48:15 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 15:48:15.831313 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ed075f5-f8d9-4620-836d-d864e44a12d4 None admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 15:48:15.845569 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 15:48:15.944236 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ed075f5-f8d9-4620-836d-d864e44a12d4 None admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 15:48:15.983131 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72974f8d-b960-4c64-8eed-b6cbc083066f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 15:48:15.983775 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72974f8d-b960-4c64-8eed-b6cbc083066f None admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:48:15.984316 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72974f8d-b960-4c64-8eed-b6cbc083066f None admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:48:15.984316 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72974f8d-b960-4c64-8eed-b6cbc083066f None admin] PATH_INFO: `/v3/services` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:48:15.984938 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72974f8d-b960-4c64-8eed-b6cbc083066f None admin] RBAC: Adding query filter params (type=compute) {{(pid=81768) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 15:48:15.985292 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72974f8d-b960-4c64-8eed-b6cbc083066f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 15:48:15.986770 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72974f8d-b960-4c64-8eed-b6cbc083066f None admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 15:48:15.992321 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 489/960] 10.222.0.15 () {56 vars in 1112 bytes} [Mon Aug 22 15:48:15 2022] GET /identity/v3/services?type=compute => generated 124 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 15:48:16.596116 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e4bbce-ad92-45db-90e3-fd26c0f577ac None None] REQUEST_METHOD: `GET` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:48:16.597101 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e4bbce-ad92-45db-90e3-fd26c0f577ac None None] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:48:16.597576 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e4bbce-ad92-45db-90e3-fd26c0f577ac None None] PATH_INFO: `/` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:48:16.599518 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 472/961] 10.222.0.15 () {54 vars in 927 bytes} [Mon Aug 22 15:48:16 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 15:48:16.606810 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ad8bba7-64d4-4db0-9dd5-668349e73310 None None] REQUEST_METHOD: `POST` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:48:16.607142 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ad8bba7-64d4-4db0-9dd5-668349e73310 None None] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:48:16.607625 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ad8bba7-64d4-4db0-9dd5-668349e73310 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:48:16.628135 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.auth.core [None req-4ad8bba7-64d4-4db0-9dd5-668349e73310 None None] MFA Rules not processed for user `6b2199ae0fb0480aa917ada391f5ec54`. Rule list: `[]` (Enabled: `True`). {{(pid=81768) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 15:48:16.629362 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-4ad8bba7-64d4-4db0-9dd5-668349e73310 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 15:48:16.709257 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 490/962] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Aug 22 15:48:16 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 22 15:48:16.987043 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89e4bbce-ad92-45db-90e3-fd26c0f577ac None None] Authenticating user token {{(pid=81767) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 15:48:17.099345 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89e4bbce-ad92-45db-90e3-fd26c0f577ac None None] Validating token access rules against request {{(pid=81767) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 15:48:17.137451 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d116680d-c793-4a47-a797-14c4947cec42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81767) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 15:48:17.138078 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d116680d-c793-4a47-a797-14c4947cec42 None admin] REQUEST_METHOD: `POST` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:48:17.138358 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d116680d-c793-4a47-a797-14c4947cec42 None admin] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:48:17.138632 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d116680d-c793-4a47-a797-14c4947cec42 None admin] PATH_INFO: `/v3/users` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:48:17.139309 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d116680d-c793-4a47-a797-14c4947cec42 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 15:48:17.140759 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d116680d-c793-4a47-a797-14c4947cec42 None admin] RBAC: Authorization granted {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 15:48:17.188147 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.sql.core [None req-d116680d-c793-4a47-a797-14c4947cec42 None admin] Conflict user: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Aug 22 15:48:17.188147 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Aug 22 15:48:17.188147 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [parameters: {'user_id': 'ca3d49dce1544099abe937814e51508d', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Aug 22 15:48:17.188147 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=81767) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Aug 22 15:48:17.195000 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application [None req-d116680d-c793-4a47-a797-14c4947cec42 None admin] Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default.: keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Aug 22 15:48:17.195000 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 15:48:17.195000 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Aug 22 15:48:17.195000 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 22 15:48:17.195000 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 22 15:48:17.195000 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 22 15:48:17.195000 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 22 15:48:17.195000 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application result = self._query(query) Aug 22 15:48:17.195000 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 22 15:48:17.195000 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application conn.query(q) Aug 22 15:48:17.195000 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 22 15:48:17.195000 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 22 15:48:17.195000 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 22 15:48:17.195000 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application result.read() Aug 22 15:48:17.195000 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 22 15:48:17.196134 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 22 15:48:17.196134 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 22 15:48:17.196134 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 22 15:48:17.196134 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 22 15:48:17.196134 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 22 15:48:17.196134 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 22 15:48:17.196134 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 22 15:48:17.196134 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Aug 22 15:48:17.196134 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application Aug 22 15:48:17.196134 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Aug 22 15:48:17.196134 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application Aug 22 15:48:17.196134 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 15:48:17.196134 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Aug 22 15:48:17.196134 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return method(*args, **kwargs) Aug 22 15:48:17.196134 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 141, in create_user Aug 22 15:48:17.196134 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return base.filter_user(user_ref.to_dict()) Aug 22 15:48:17.196134 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 22 15:48:17.196134 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application next(self.gen) Aug 22 15:48:17.197960 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Aug 22 15:48:17.197960 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application yield resource Aug 22 15:48:17.197960 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 22 15:48:17.197960 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application next(self.gen) Aug 22 15:48:17.197960 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Aug 22 15:48:17.197960 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application self.session.rollback() Aug 22 15:48:17.197960 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Aug 22 15:48:17.197960 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application self.force_reraise() Aug 22 15:48:17.197960 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Aug 22 15:48:17.197960 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application raise self.value Aug 22 15:48:17.197960 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Aug 22 15:48:17.197960 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Aug 22 15:48:17.197960 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Aug 22 15:48:17.197960 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application session.commit() Aug 22 15:48:17.197960 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Aug 22 15:48:17.197960 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Aug 22 15:48:17.197960 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Aug 22 15:48:17.199101 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application self._prepare_impl() Aug 22 15:48:17.199101 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Aug 22 15:48:17.199101 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application self.session.flush() Aug 22 15:48:17.199101 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Aug 22 15:48:17.199101 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application self._flush(objects) Aug 22 15:48:17.199101 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Aug 22 15:48:17.199101 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Aug 22 15:48:17.199101 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Aug 22 15:48:17.199101 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application compat.raise_( Aug 22 15:48:17.199101 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 22 15:48:17.199101 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application raise exception Aug 22 15:48:17.199101 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Aug 22 15:48:17.199101 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application flush_context.execute() Aug 22 15:48:17.199101 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Aug 22 15:48:17.199101 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application rec.execute(self) Aug 22 15:48:17.199101 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Aug 22 15:48:17.199101 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Aug 22 15:48:17.200171 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Aug 22 15:48:17.200171 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application _emit_insert_statements( Aug 22 15:48:17.200171 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1238, in _emit_insert_statements Aug 22 15:48:17.200171 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application result = connection._execute_20( Aug 22 15:48:17.200171 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Aug 22 15:48:17.200171 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Aug 22 15:48:17.200171 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Aug 22 15:48:17.200171 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Aug 22 15:48:17.200171 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Aug 22 15:48:17.200171 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application ret = self._execute_context( Aug 22 15:48:17.200171 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Aug 22 15:48:17.200171 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Aug 22 15:48:17.200171 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Aug 22 15:48:17.200171 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Aug 22 15:48:17.200171 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 22 15:48:17.200171 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application raise exception Aug 22 15:48:17.201044 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Aug 22 15:48:17.201044 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 22 15:48:17.201044 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 22 15:48:17.201044 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 22 15:48:17.201044 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 22 15:48:17.201044 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application result = self._query(query) Aug 22 15:48:17.201044 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 22 15:48:17.201044 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application conn.query(q) Aug 22 15:48:17.201044 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 22 15:48:17.201044 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 22 15:48:17.201044 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 22 15:48:17.201044 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application result.read() Aug 22 15:48:17.201044 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 22 15:48:17.201044 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 22 15:48:17.201044 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 22 15:48:17.201044 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 22 15:48:17.202777 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 22 15:48:17.202777 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 22 15:48:17.202777 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 22 15:48:17.202777 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 22 15:48:17.202777 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Aug 22 15:48:17.202777 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Aug 22 15:48:17.202777 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application [parameters: {'user_id': 'ca3d49dce1544099abe937814e51508d', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Aug 22 15:48:17.202777 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Aug 22 15:48:17.202777 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application Aug 22 15:48:17.202777 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 22 15:48:17.202777 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application Aug 22 15:48:17.202777 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 15:48:17.202777 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 15:48:17.202777 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 15:48:17.202777 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 15:48:17.203653 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 15:48:17.203653 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 15:48:17.203653 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 15:48:17.203653 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 15:48:17.203653 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 15:48:17.203653 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 15:48:17.203653 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 15:48:17.203653 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 244, in post Aug 22 15:48:17.203653 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.create_user( Aug 22 15:48:17.203653 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 15:48:17.203653 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 15:48:17.203653 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 22 15:48:17.203653 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 22 15:48:17.203653 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 22 15:48:17.203653 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 22 15:48:17.203653 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1000, in create_user Aug 22 15:48:17.203653 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application ref = self._create_user_with_federated_objects(user, driver) Aug 22 15:48:17.204700 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 966, in _create_user_with_federated_objects Aug 22 15:48:17.204700 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application user = driver.create_user(user['id'], user) Aug 22 15:48:17.204700 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Aug 22 15:48:17.204700 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Aug 22 15:48:17.204700 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Aug 22 15:48:17.204700 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application Aug 22 15:48:17.204700 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 473/963] 10.222.0.15 () {60 vars in 1131 bytes} [Mon Aug 22 15:48:16 2022] POST /identity/v3/users => generated 160 bytes in 210 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Aug 22 15:48:17.205084 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ad8bba7-64d4-4db0-9dd5-668349e73310 None None] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 15:48:17.216907 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=100 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 15:48:17.316912 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ad8bba7-64d4-4db0-9dd5-668349e73310 None None] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 15:48:17.354914 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dff415b-01d8-40b7-962f-d5c8aa1e3f2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 15:48:17.355586 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dff415b-01d8-40b7-962f-d5c8aa1e3f2e None admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:48:17.356539 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dff415b-01d8-40b7-962f-d5c8aa1e3f2e None admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:48:17.356539 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dff415b-01d8-40b7-962f-d5c8aa1e3f2e None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:48:17.364515 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dff415b-01d8-40b7-962f-d5c8aa1e3f2e None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 15:48:17.366054 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dff415b-01d8-40b7-962f-d5c8aa1e3f2e None admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 15:48:17.378707 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application [None req-1dff415b-01d8-40b7-962f-d5c8aa1e3f2e None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Aug 22 15:48:17.378707 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 15:48:17.378707 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 15:48:17.378707 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 15:48:17.378707 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 15:48:17.378707 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 15:48:17.378707 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 15:48:17.378707 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 15:48:17.378707 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 15:48:17.378707 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 15:48:17.378707 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 15:48:17.378707 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 15:48:17.378707 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 22 15:48:17.378707 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 22 15:48:17.378707 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 22 15:48:17.378707 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 22 15:48:17.379874 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 15:48:17.379874 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 15:48:17.379874 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 22 15:48:17.379874 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 22 15:48:17.379874 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 22 15:48:17.379874 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 22 15:48:17.379874 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 15:48:17.379874 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 15:48:17.379874 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 15:48:17.379874 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 15:48:17.379874 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 15:48:17.379874 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application with Lock( Aug 22 15:48:17.379874 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 15:48:17.379874 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self._enter() Aug 22 15:48:17.379874 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 15:48:17.379874 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 15:48:17.379874 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 15:48:17.379874 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self.creator() Aug 22 15:48:17.380810 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 15:48:17.380810 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application created_value = creator( Aug 22 15:48:17.380810 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 22 15:48:17.380810 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 22 15:48:17.380810 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 22 15:48:17.380810 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 22 15:48:17.380810 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 22 15:48:17.380810 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 22 15:48:17.380810 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Aug 22 15:48:17.380810 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application Aug 22 15:48:17.380810 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 491/964] 10.222.0.15 () {56 vars in 1099 bytes} [Mon Aug 22 15:48:17 2022] GET /identity/v3/users/cinder => generated 84 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 15:48:17.386990 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d116680d-c793-4a47-a797-14c4947cec42 None admin] Authenticating user token {{(pid=81767) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 15:48:17.404140 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG dbcounter [-] [81767] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81767) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 15:48:17.509424 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d116680d-c793-4a47-a797-14c4947cec42 None admin] Validating token access rules against request {{(pid=81767) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 15:48:17.555738 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14318087-5ef5-4705-89ba-bb6ab816bd05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81767) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 15:48:17.557304 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14318087-5ef5-4705-89ba-bb6ab816bd05 None admin] REQUEST_METHOD: `GET` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:48:17.558027 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14318087-5ef5-4705-89ba-bb6ab816bd05 None admin] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:48:17.558469 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14318087-5ef5-4705-89ba-bb6ab816bd05 None admin] PATH_INFO: `/v3/users` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:48:17.559865 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14318087-5ef5-4705-89ba-bb6ab816bd05 None admin] RBAC: Adding query filter params (name=cinder) {{(pid=81767) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 15:48:17.560357 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14318087-5ef5-4705-89ba-bb6ab816bd05 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 15:48:17.563418 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14318087-5ef5-4705-89ba-bb6ab816bd05 None admin] RBAC: Authorization granted {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 15:48:17.608301 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a643fb-f7dd-44eb-9732-4b19b5407b88 None None] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:48:17.608629 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a643fb-f7dd-44eb-9732-4b19b5407b88 None None] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:48:17.608912 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a643fb-f7dd-44eb-9732-4b19b5407b88 None None] PATH_INFO: `/` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:48:17.610381 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 492/965] 10.222.0.15 () {54 vars in 927 bytes} [Mon Aug 22 15:48:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 15:48:17.613358 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 474/966] 10.222.0.15 () {56 vars in 1101 bytes} [Mon Aug 22 15:48:17 2022] GET /identity/v3/users?name=cinder => generated 354 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 15:48:17.619463 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aebde95-5096-41ce-a185-a0a151b6fc1a None None] REQUEST_METHOD: `POST` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:48:17.619753 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aebde95-5096-41ce-a185-a0a151b6fc1a None None] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:48:17.620011 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aebde95-5096-41ce-a185-a0a151b6fc1a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:48:17.644073 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.auth.core [None req-6aebde95-5096-41ce-a185-a0a151b6fc1a None None] MFA Rules not processed for user `6b2199ae0fb0480aa917ada391f5ec54`. Rule list: `[]` (Enabled: `True`). {{(pid=81768) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 15:48:17.645301 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-6aebde95-5096-41ce-a185-a0a151b6fc1a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 15:48:17.729119 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 493/967] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Aug 22 15:48:17 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 22 15:48:17.729933 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=100 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 15:48:17.767902 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f7f5012-80e2-416b-bf24-bd5cb1ed8a4c None None] REQUEST_METHOD: `GET` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:48:17.768247 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f7f5012-80e2-416b-bf24-bd5cb1ed8a4c None None] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:48:17.768460 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f7f5012-80e2-416b-bf24-bd5cb1ed8a4c None None] PATH_INFO: `/` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:48:17.769945 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 475/968] 10.222.0.15 () {54 vars in 927 bytes} [Mon Aug 22 15:48:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 15:48:17.775953 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2af33234-0b5d-4acf-bb4e-10d3f4ac6ccc None None] REQUEST_METHOD: `POST` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:48:17.776233 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2af33234-0b5d-4acf-bb4e-10d3f4ac6ccc None None] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:48:17.776528 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2af33234-0b5d-4acf-bb4e-10d3f4ac6ccc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:48:17.795053 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.auth.core [None req-2af33234-0b5d-4acf-bb4e-10d3f4ac6ccc None None] MFA Rules not processed for user `6b2199ae0fb0480aa917ada391f5ec54`. Rule list: `[]` (Enabled: `True`). {{(pid=81768) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 15:48:17.796378 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-2af33234-0b5d-4acf-bb4e-10d3f4ac6ccc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 15:48:17.875168 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 494/969] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Aug 22 15:48:17 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 22 15:48:18.026205 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f7f5012-80e2-416b-bf24-bd5cb1ed8a4c None None] Authenticating user token {{(pid=81767) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 15:48:18.052042 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-397a9d45-c77c-4b24-a038-82266718ce22 None None] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:48:18.052403 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-397a9d45-c77c-4b24-a038-82266718ce22 None None] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:48:18.052494 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-397a9d45-c77c-4b24-a038-82266718ce22 None None] PATH_INFO: `/` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:48:18.054128 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 495/970] 10.222.0.15 () {54 vars in 927 bytes} [Mon Aug 22 15:48:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 22 15:48:18.059421 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d5ba826-fa9d-4984-887a-d72a0e1c22e6 None None] REQUEST_METHOD: `POST` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:48:18.059738 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d5ba826-fa9d-4984-887a-d72a0e1c22e6 None None] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:48:18.059962 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d5ba826-fa9d-4984-887a-d72a0e1c22e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:48:18.078557 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.auth.core [None req-6d5ba826-fa9d-4984-887a-d72a0e1c22e6 None None] MFA Rules not processed for user `6b2199ae0fb0480aa917ada391f5ec54`. Rule list: `[]` (Enabled: `True`). {{(pid=81768) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 15:48:18.079774 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-6d5ba826-fa9d-4984-887a-d72a0e1c22e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 15:48:18.108276 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG dbcounter [-] [81767] Writing DB stats keystone:SELECT=100 {{(pid=81767) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 15:48:18.157747 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f7f5012-80e2-416b-bf24-bd5cb1ed8a4c None None] Validating token access rules against request {{(pid=81767) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 15:48:18.158116 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 496/971] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Aug 22 15:48:18 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 22 15:48:18.158116 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d5ba826-fa9d-4984-887a-d72a0e1c22e6 None None] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 15:48:18.178221 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62500500-9a6d-4af1-9887-39292b543a5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81767) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 15:48:18.178854 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62500500-9a6d-4af1-9887-39292b543a5b None admin] REQUEST_METHOD: `GET` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:48:18.179126 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62500500-9a6d-4af1-9887-39292b543a5b None admin] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:48:18.179428 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62500500-9a6d-4af1-9887-39292b543a5b None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:48:18.180006 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62500500-9a6d-4af1-9887-39292b543a5b None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 15:48:18.181891 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62500500-9a6d-4af1-9887-39292b543a5b None admin] RBAC: Authorization granted {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 15:48:18.192352 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application [None req-62500500-9a6d-4af1-9887-39292b543a5b None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 22 15:48:18.192352 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 15:48:18.192352 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 22 15:48:18.192352 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 22 15:48:18.192352 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 22 15:48:18.192352 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 22 15:48:18.192352 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 22 15:48:18.192352 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 22 15:48:18.192352 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 22 15:48:18.192352 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application Aug 22 15:48:18.192352 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 22 15:48:18.192352 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application Aug 22 15:48:18.192352 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 15:48:18.192352 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 15:48:18.192352 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 15:48:18.192352 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 15:48:18.192352 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 15:48:18.193407 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 15:48:18.193407 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 15:48:18.193407 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 15:48:18.193407 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 15:48:18.193407 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 15:48:18.193407 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 15:48:18.193407 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 22 15:48:18.193407 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 22 15:48:18.193407 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 22 15:48:18.193407 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 22 15:48:18.193407 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 15:48:18.193407 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 15:48:18.193407 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 15:48:18.193407 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 15:48:18.193407 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 15:48:18.193407 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 15:48:18.194244 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 15:48:18.194244 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application with Lock( Aug 22 15:48:18.194244 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 15:48:18.194244 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return self._enter() Aug 22 15:48:18.194244 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 15:48:18.194244 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 15:48:18.194244 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 15:48:18.194244 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return self.creator() Aug 22 15:48:18.194244 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 15:48:18.194244 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application created_value = creator( Aug 22 15:48:18.194244 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 22 15:48:18.194244 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 22 15:48:18.194244 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 22 15:48:18.194244 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application Aug 22 15:48:18.194244 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 476/972] 10.222.0.15 () {56 vars in 1126 bytes} [Mon Aug 22 15:48:18 2022] GET /identity/v3/services/object-store => generated 93 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 15:48:18.200014 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62500500-9a6d-4af1-9887-39292b543a5b None admin] Authenticating user token {{(pid=81767) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 15:48:18.218522 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=100 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 15:48:18.265909 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d5ba826-fa9d-4984-887a-d72a0e1c22e6 None None] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 15:48:18.308308 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62500500-9a6d-4af1-9887-39292b543a5b None admin] Validating token access rules against request {{(pid=81767) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 15:48:18.312838 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e79f67e-1f79-4052-b996-97ff180cf497 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 15:48:18.313772 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e79f67e-1f79-4052-b996-97ff180cf497 None admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:48:18.314274 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e79f67e-1f79-4052-b996-97ff180cf497 None admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:48:18.314753 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e79f67e-1f79-4052-b996-97ff180cf497 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:48:18.327782 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e79f67e-1f79-4052-b996-97ff180cf497 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 22 15:48:18.332561 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application [None req-5e79f67e-1f79-4052-b996-97ff180cf497 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 22 15:48:18.332561 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 15:48:18.332561 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 15:48:18.332561 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 15:48:18.332561 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 15:48:18.332561 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 15:48:18.332561 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 15:48:18.332561 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 15:48:18.332561 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 15:48:18.332561 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 15:48:18.332561 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 15:48:18.332561 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 15:48:18.332561 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 22 15:48:18.332561 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 22 15:48:18.332561 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 22 15:48:18.332561 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 22 15:48:18.333645 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 22 15:48:18.333645 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 22 15:48:18.333645 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 22 15:48:18.333645 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 22 15:48:18.333645 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 15:48:18.333645 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 15:48:18.333645 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 15:48:18.333645 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 15:48:18.333645 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 15:48:18.333645 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 15:48:18.333645 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 15:48:18.333645 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application with Lock( Aug 22 15:48:18.333645 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 15:48:18.333645 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self._enter() Aug 22 15:48:18.333645 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 15:48:18.333645 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 15:48:18.334463 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 15:48:18.334463 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self.creator() Aug 22 15:48:18.334463 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 15:48:18.334463 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application created_value = creator( Aug 22 15:48:18.334463 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 22 15:48:18.334463 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 22 15:48:18.334463 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 22 15:48:18.334463 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 22 15:48:18.334463 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 22 15:48:18.334463 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 22 15:48:18.334463 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 22 15:48:18.334463 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application Aug 22 15:48:18.334463 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 497/973] 10.222.0.15 () {56 vars in 1120 bytes} [Mon Aug 22 15:48:18 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 15:48:18.341895 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e79f67e-1f79-4052-b996-97ff180cf497 None admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 15:48:18.358673 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42bfdc80-6a50-4fca-b07f-716d3d52eba0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81767) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 15:48:18.359498 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42bfdc80-6a50-4fca-b07f-716d3d52eba0 None admin] REQUEST_METHOD: `GET` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:48:18.359776 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42bfdc80-6a50-4fca-b07f-716d3d52eba0 None admin] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:48:18.360050 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42bfdc80-6a50-4fca-b07f-716d3d52eba0 None admin] PATH_INFO: `/v3/services` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:48:18.360689 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42bfdc80-6a50-4fca-b07f-716d3d52eba0 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81767) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 15:48:18.360994 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42bfdc80-6a50-4fca-b07f-716d3d52eba0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 15:48:18.362374 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42bfdc80-6a50-4fca-b07f-716d3d52eba0 None admin] RBAC: Authorization granted {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 15:48:18.363890 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG dbcounter [-] [81767] Writing DB stats keystone:SELECT=100 {{(pid=81767) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 15:48:18.370468 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 477/974] 10.222.0.15 () {56 vars in 1122 bytes} [Mon Aug 22 15:48:18 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 22 15:48:18.375932 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42bfdc80-6a50-4fca-b07f-716d3d52eba0 None admin] Authenticating user token {{(pid=81767) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 15:48:18.491115 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e79f67e-1f79-4052-b996-97ff180cf497 None admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 15:48:18.516261 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42bfdc80-6a50-4fca-b07f-716d3d52eba0 None admin] Validating token access rules against request {{(pid=81767) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 15:48:18.521568 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=100 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 15:48:18.539362 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-503d23b0-3688-46db-946a-2282bd026eb6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 15:48:18.539966 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-503d23b0-3688-46db-946a-2282bd026eb6 None admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:48:18.540274 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-503d23b0-3688-46db-946a-2282bd026eb6 None admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:48:18.540541 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-503d23b0-3688-46db-946a-2282bd026eb6 None admin] PATH_INFO: `/v3/roles` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:48:18.541205 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-503d23b0-3688-46db-946a-2282bd026eb6 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81768) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 15:48:18.541505 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-503d23b0-3688-46db-946a-2282bd026eb6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 15:48:18.542887 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-503d23b0-3688-46db-946a-2282bd026eb6 None admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 15:48:18.551183 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 498/975] 10.222.0.15 () {56 vars in 1115 bytes} [Mon Aug 22 15:48:18 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 22 15:48:18.551985 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-503d23b0-3688-46db-946a-2282bd026eb6 None admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 15:48:18.562507 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2dc6361-f42d-4683-a433-7f1a20bb1974 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81767) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 15:48:18.563117 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2dc6361-f42d-4683-a433-7f1a20bb1974 None admin] REQUEST_METHOD: `GET` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:48:18.563425 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2dc6361-f42d-4683-a433-7f1a20bb1974 None admin] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:48:18.563715 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2dc6361-f42d-4683-a433-7f1a20bb1974 None admin] PATH_INFO: `/v3/services` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:48:18.564339 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2dc6361-f42d-4683-a433-7f1a20bb1974 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81767) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 15:48:18.564630 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2dc6361-f42d-4683-a433-7f1a20bb1974 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 15:48:18.565979 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2dc6361-f42d-4683-a433-7f1a20bb1974 None admin] RBAC: Authorization granted {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 15:48:18.571867 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 478/976] 10.222.0.15 () {56 vars in 1122 bytes} [Mon Aug 22 15:48:18 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 22 15:48:18.572606 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2dc6361-f42d-4683-a433-7f1a20bb1974 None admin] Authenticating user token {{(pid=81767) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 15:48:18.638494 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG dbcounter [-] [81767] Writing DB stats keystone:SELECT=100 {{(pid=81767) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 15:48:18.669400 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-503d23b0-3688-46db-946a-2282bd026eb6 None admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 15:48:18.680331 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2dc6361-f42d-4683-a433-7f1a20bb1974 None admin] Validating token access rules against request {{(pid=81767) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 15:48:18.706953 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4d6f96d-70eb-4055-b2f5-1f20b7c829ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 15:48:18.707593 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4d6f96d-70eb-4055-b2f5-1f20b7c829ce None admin] REQUEST_METHOD: `POST` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:48:18.707873 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4d6f96d-70eb-4055-b2f5-1f20b7c829ce None admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:48:18.708141 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4d6f96d-70eb-4055-b2f5-1f20b7c829ce None admin] PATH_INFO: `/v3/services` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:48:18.708776 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4d6f96d-70eb-4055-b2f5-1f20b7c829ce None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 15:48:18.710285 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4d6f96d-70eb-4055-b2f5-1f20b7c829ce None admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 15:48:18.717879 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8047a8f2-ee26-4782-833b-a6ff7e051308 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81767) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 15:48:18.718507 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8047a8f2-ee26-4782-833b-a6ff7e051308 None admin] REQUEST_METHOD: `GET` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:48:18.718783 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8047a8f2-ee26-4782-833b-a6ff7e051308 None admin] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:48:18.719055 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8047a8f2-ee26-4782-833b-a6ff7e051308 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:48:18.729995 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8047a8f2-ee26-4782-833b-a6ff7e051308 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 15:48:18.736034 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8047a8f2-ee26-4782-833b-a6ff7e051308 None admin] RBAC: Authorization granted {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 15:48:18.745819 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application [None req-8047a8f2-ee26-4782-833b-a6ff7e051308 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 22 15:48:18.745819 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 15:48:18.745819 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 15:48:18.745819 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 15:48:18.745819 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 15:48:18.745819 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 15:48:18.745819 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 15:48:18.745819 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 15:48:18.745819 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 15:48:18.745819 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 15:48:18.745819 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 15:48:18.745819 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 15:48:18.745819 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 22 15:48:18.745819 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 22 15:48:18.745819 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 22 15:48:18.745819 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 22 15:48:18.746921 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 15:48:18.746921 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 15:48:18.746921 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 15:48:18.746921 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 15:48:18.746921 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 15:48:18.746921 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 15:48:18.746921 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 15:48:18.746921 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application with Lock( Aug 22 15:48:18.746921 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 15:48:18.746921 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return self._enter() Aug 22 15:48:18.746921 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 15:48:18.746921 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 15:48:18.746921 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 15:48:18.746921 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return self.creator() Aug 22 15:48:18.746921 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 15:48:18.746921 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application created_value = creator( Aug 22 15:48:18.746921 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 22 15:48:18.748144 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 22 15:48:18.748144 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 22 15:48:18.748144 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application Aug 22 15:48:18.748144 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 479/977] 10.222.0.15 () {56 vars in 1108 bytes} [Mon Aug 22 15:48:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 15:48:18.748144 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8047a8f2-ee26-4782-833b-a6ff7e051308 None admin] Authenticating user token {{(pid=81767) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 15:48:18.799507 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 499/978] 10.222.0.15 () {60 vars in 1141 bytes} [Mon Aug 22 15:48:18 2022] POST /identity/v3/services => generated 242 bytes in 243 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 22 15:48:18.799507 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4d6f96d-70eb-4055-b2f5-1f20b7c829ce None admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 15:48:18.859202 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 15:48:18.870493 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8047a8f2-ee26-4782-833b-a6ff7e051308 None admin] Validating token access rules against request {{(pid=81767) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 15:48:18.889187 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG dbcounter [-] [81767] Writing DB stats keystone:SELECT=100 {{(pid=81767) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 15:48:18.925947 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-972f9cac-35ce-4cb4-b8e7-61ada5e0a57b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81767) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 15:48:18.926871 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-972f9cac-35ce-4cb4-b8e7-61ada5e0a57b None admin] REQUEST_METHOD: `GET` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:48:18.927139 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-972f9cac-35ce-4cb4-b8e7-61ada5e0a57b None admin] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:48:18.927443 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-972f9cac-35ce-4cb4-b8e7-61ada5e0a57b None admin] PATH_INFO: `/v3/endpoints` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:48:18.928195 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-972f9cac-35ce-4cb4-b8e7-61ada5e0a57b None admin] RBAC: Adding query filter params (interface=admin, service_id=6febf343b03e4ebfb02e74970d12ab1d, region_id=RegionOne) {{(pid=81767) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 15:48:18.928506 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-972f9cac-35ce-4cb4-b8e7-61ada5e0a57b None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 15:48:18.929931 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-972f9cac-35ce-4cb4-b8e7-61ada5e0a57b None admin] RBAC: Authorization granted {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 15:48:18.930168 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4d6f96d-70eb-4055-b2f5-1f20b7c829ce None admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 15:48:18.936585 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 480/979] 10.222.0.15 () {56 vars in 1249 bytes} [Mon Aug 22 15:48:18 2022] GET /identity/v3/endpoints?service_id=6febf343b03e4ebfb02e74970d12ab1d&interface=admin®ion_id=RegionOne => generated 193 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 15:48:18.942804 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-972f9cac-35ce-4cb4-b8e7-61ada5e0a57b None admin] Authenticating user token {{(pid=81767) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 15:48:18.983525 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eba4555-a398-4c7f-b5e2-10a4f764e108 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 15:48:18.984165 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eba4555-a398-4c7f-b5e2-10a4f764e108 None admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:48:18.984488 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eba4555-a398-4c7f-b5e2-10a4f764e108 None admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:48:18.984727 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eba4555-a398-4c7f-b5e2-10a4f764e108 None admin] PATH_INFO: `/v3/domains` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:48:18.985387 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eba4555-a398-4c7f-b5e2-10a4f764e108 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81768) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 15:48:18.985690 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eba4555-a398-4c7f-b5e2-10a4f764e108 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 15:48:18.987237 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eba4555-a398-4c7f-b5e2-10a4f764e108 None admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 15:48:19.005571 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 500/980] 10.222.0.15 () {56 vars in 1109 bytes} [Mon Aug 22 15:48:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 15:48:19.011946 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eba4555-a398-4c7f-b5e2-10a4f764e108 None admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 15:48:19.065494 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-972f9cac-35ce-4cb4-b8e7-61ada5e0a57b None admin] Validating token access rules against request {{(pid=81767) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 15:48:19.106733 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91df0013-b534-4f50-9c2a-c78979884454 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81767) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 15:48:19.107379 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91df0013-b534-4f50-9c2a-c78979884454 None admin] REQUEST_METHOD: `GET` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:48:19.107657 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91df0013-b534-4f50-9c2a-c78979884454 None admin] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:48:19.107927 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91df0013-b534-4f50-9c2a-c78979884454 None admin] PATH_INFO: `/v3/services` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:48:19.108547 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91df0013-b534-4f50-9c2a-c78979884454 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 15:48:19.109983 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91df0013-b534-4f50-9c2a-c78979884454 None admin] RBAC: Authorization granted {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 15:48:19.127314 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 481/981] 10.222.0.15 () {56 vars in 1087 bytes} [Mon Aug 22 15:48:18 2022] GET /identity/v3/services => generated 1727 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 22 15:48:19.146589 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eba4555-a398-4c7f-b5e2-10a4f764e108 None admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 15:48:19.149016 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=100 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 15:48:19.188948 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccbf749e-e9f7-4153-bb08-befe9eb17634 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 15:48:19.189593 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccbf749e-e9f7-4153-bb08-befe9eb17634 None admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:48:19.189867 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccbf749e-e9f7-4153-bb08-befe9eb17634 None admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:48:19.190135 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccbf749e-e9f7-4153-bb08-befe9eb17634 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:48:19.198577 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccbf749e-e9f7-4153-bb08-befe9eb17634 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 15:48:19.200296 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccbf749e-e9f7-4153-bb08-befe9eb17634 None admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 15:48:19.214597 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application [None req-ccbf749e-e9f7-4153-bb08-befe9eb17634 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 22 15:48:19.214597 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 15:48:19.214597 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 15:48:19.214597 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 15:48:19.214597 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 15:48:19.214597 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 15:48:19.214597 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 15:48:19.214597 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 15:48:19.214597 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 15:48:19.214597 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 15:48:19.214597 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 15:48:19.214597 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 15:48:19.214597 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 22 15:48:19.214597 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 22 15:48:19.214597 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 22 15:48:19.214597 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 22 15:48:19.215756 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 15:48:19.215756 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 15:48:19.215756 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 22 15:48:19.215756 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 22 15:48:19.215756 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 22 15:48:19.215756 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 22 15:48:19.215756 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 15:48:19.215756 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 15:48:19.215756 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 15:48:19.215756 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 15:48:19.215756 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 15:48:19.215756 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application with Lock( Aug 22 15:48:19.215756 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 15:48:19.215756 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self._enter() Aug 22 15:48:19.215756 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 15:48:19.215756 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 15:48:19.215756 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 15:48:19.215756 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self.creator() Aug 22 15:48:19.216684 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 15:48:19.216684 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application created_value = creator( Aug 22 15:48:19.216684 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 22 15:48:19.216684 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 22 15:48:19.216684 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 22 15:48:19.216684 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 22 15:48:19.216684 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 22 15:48:19.216684 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 22 15:48:19.216684 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 22 15:48:19.216684 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application Aug 22 15:48:19.219020 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 501/982] 10.222.0.15 () {56 vars in 1117 bytes} [Mon Aug 22 15:48:19 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 15:48:19.224903 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91df0013-b534-4f50-9c2a-c78979884454 None admin] Authenticating user token {{(pid=81767) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 15:48:19.276919 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG dbcounter [-] [81767] Writing DB stats keystone:SELECT=100 {{(pid=81767) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 15:48:19.350582 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91df0013-b534-4f50-9c2a-c78979884454 None admin] Validating token access rules against request {{(pid=81767) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 15:48:19.389127 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40d43130-6ba6-4c4c-b029-2a4ef3264361 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81767) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 15:48:19.389737 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40d43130-6ba6-4c4c-b029-2a4ef3264361 None admin] REQUEST_METHOD: `GET` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:48:19.390016 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40d43130-6ba6-4c4c-b029-2a4ef3264361 None admin] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:48:19.390287 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40d43130-6ba6-4c4c-b029-2a4ef3264361 None admin] PATH_INFO: `/v3/users` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:48:19.391029 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40d43130-6ba6-4c4c-b029-2a4ef3264361 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81767) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 15:48:19.391352 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40d43130-6ba6-4c4c-b029-2a4ef3264361 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 15:48:19.392804 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40d43130-6ba6-4c4c-b029-2a4ef3264361 None admin] RBAC: Authorization granted {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 15:48:19.413219 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 482/983] 10.222.0.15 () {56 vars in 1149 bytes} [Mon Aug 22 15:48:19 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 15:48:19.419186 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccbf749e-e9f7-4153-bb08-befe9eb17634 None admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 15:48:19.532894 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccbf749e-e9f7-4153-bb08-befe9eb17634 None admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 15:48:19.569271 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bd970c1-504a-4f07-b64a-7f1422f182b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 15:48:19.569875 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd970c1-504a-4f07-b64a-7f1422f182b6 None admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:48:19.570220 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd970c1-504a-4f07-b64a-7f1422f182b6 None admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:48:19.570495 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd970c1-504a-4f07-b64a-7f1422f182b6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:48:19.604291 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=100 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 15:48:19.645120 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bd970c1-504a-4f07-b64a-7f1422f182b6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 15:48:19.646582 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bd970c1-504a-4f07-b64a-7f1422f182b6 None admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 15:48:19.754752 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 502/984] 10.222.0.15 () {58 vars in 1282 bytes} [Mon Aug 22 15:48:19 2022] GET /identity/v3/auth/tokens => generated 1570 bytes in 336 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 22 15:48:19.760911 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40d43130-6ba6-4c4c-b029-2a4ef3264361 None admin] Authenticating user token {{(pid=81767) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 15:48:19.863602 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG dbcounter [-] [81767] Writing DB stats keystone:SELECT=100 {{(pid=81767) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 15:48:19.870701 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40d43130-6ba6-4c4c-b029-2a4ef3264361 None admin] Validating token access rules against request {{(pid=81767) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 15:48:19.909302 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1827f0b7-9c5e-4a8d-941f-1d98093a5bff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81767) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 15:48:19.909928 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1827f0b7-9c5e-4a8d-941f-1d98093a5bff None admin] REQUEST_METHOD: `GET` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:48:19.910203 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1827f0b7-9c5e-4a8d-941f-1d98093a5bff None admin] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:48:19.910468 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1827f0b7-9c5e-4a8d-941f-1d98093a5bff None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:48:19.921269 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1827f0b7-9c5e-4a8d-941f-1d98093a5bff None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 15:48:19.922898 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1827f0b7-9c5e-4a8d-941f-1d98093a5bff None admin] RBAC: Authorization granted {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 15:48:19.939253 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application [None req-1827f0b7-9c5e-4a8d-941f-1d98093a5bff None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 22 15:48:19.939253 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 15:48:19.939253 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 15:48:19.939253 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 15:48:19.939253 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 15:48:19.939253 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 15:48:19.939253 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 15:48:19.939253 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 15:48:19.939253 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 15:48:19.939253 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 15:48:19.939253 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 15:48:19.939253 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 15:48:19.939253 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 22 15:48:19.939253 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 22 15:48:19.939253 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 22 15:48:19.939253 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 22 15:48:19.940336 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 15:48:19.940336 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 15:48:19.940336 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 15:48:19.940336 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 15:48:19.940336 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 15:48:19.940336 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 15:48:19.940336 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 15:48:19.940336 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application with Lock( Aug 22 15:48:19.940336 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 15:48:19.940336 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return self._enter() Aug 22 15:48:19.940336 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 15:48:19.940336 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 15:48:19.940336 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 15:48:19.940336 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return self.creator() Aug 22 15:48:19.940336 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 15:48:19.940336 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application created_value = creator( Aug 22 15:48:19.940336 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 22 15:48:19.941205 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 22 15:48:19.941205 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 22 15:48:19.941205 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application Aug 22 15:48:19.941205 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 483/985] 10.222.0.15 () {56 vars in 1108 bytes} [Mon Aug 22 15:48:19 2022] GET /identity/v3/domains/Default => generated 87 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 22 15:48:19.948145 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bd970c1-504a-4f07-b64a-7f1422f182b6 None admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 15:48:20.015566 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=100 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 15:48:20.056646 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bd970c1-504a-4f07-b64a-7f1422f182b6 None admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 15:48:20.091603 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76e31757-67ed-4109-8e0c-e77b352a83d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 15:48:20.092198 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e31757-67ed-4109-8e0c-e77b352a83d1 None admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:48:20.092494 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e31757-67ed-4109-8e0c-e77b352a83d1 None admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:48:20.092762 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e31757-67ed-4109-8e0c-e77b352a83d1 None admin] PATH_INFO: `/v3/domains` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:48:20.093363 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76e31757-67ed-4109-8e0c-e77b352a83d1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81768) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 15:48:20.093660 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76e31757-67ed-4109-8e0c-e77b352a83d1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 15:48:20.095025 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76e31757-67ed-4109-8e0c-e77b352a83d1 None admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 15:48:20.111454 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 503/986] 10.222.0.15 () {56 vars in 1109 bytes} [Mon Aug 22 15:48:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 15:48:20.113346 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1827f0b7-9c5e-4a8d-941f-1d98093a5bff None admin] Authenticating user token {{(pid=81767) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 15:48:20.228330 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1827f0b7-9c5e-4a8d-941f-1d98093a5bff None admin] Validating token access rules against request {{(pid=81767) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 15:48:20.267442 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-401ad525-862e-477d-a6aa-3d3b85b69a4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81767) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 15:48:20.268082 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-401ad525-862e-477d-a6aa-3d3b85b69a4c None admin] REQUEST_METHOD: `GET` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:48:20.268364 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-401ad525-862e-477d-a6aa-3d3b85b69a4c None admin] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:48:20.268709 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-401ad525-862e-477d-a6aa-3d3b85b69a4c None admin] PATH_INFO: `/v3/projects/service` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:48:20.276769 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-401ad525-862e-477d-a6aa-3d3b85b69a4c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 15:48:20.278268 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-401ad525-862e-477d-a6aa-3d3b85b69a4c None admin] RBAC: Authorization granted {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 15:48:20.295484 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application [None req-401ad525-862e-477d-a6aa-3d3b85b69a4c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 22 15:48:20.295484 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 15:48:20.295484 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 15:48:20.295484 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 15:48:20.295484 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 15:48:20.295484 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 15:48:20.295484 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 15:48:20.295484 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 15:48:20.295484 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 15:48:20.295484 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 15:48:20.295484 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 15:48:20.295484 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 15:48:20.295484 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 22 15:48:20.295484 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 22 15:48:20.295484 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 22 15:48:20.296844 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 22 15:48:20.296844 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 15:48:20.296844 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 15:48:20.296844 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 15:48:20.296844 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 15:48:20.296844 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 15:48:20.296844 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 15:48:20.296844 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 15:48:20.296844 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application with Lock( Aug 22 15:48:20.296844 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 15:48:20.296844 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return self._enter() Aug 22 15:48:20.296844 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 15:48:20.296844 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 15:48:20.296844 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 15:48:20.296844 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return self.creator() Aug 22 15:48:20.296844 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 15:48:20.296844 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application created_value = creator( Aug 22 15:48:20.298980 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 22 15:48:20.298980 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 22 15:48:20.298980 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 22 15:48:20.298980 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 22 15:48:20.298980 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 22 15:48:20.298980 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 22 15:48:20.298980 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 22 15:48:20.298980 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application Aug 22 15:48:20.298980 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 484/987] 10.222.0.15 () {56 vars in 1111 bytes} [Mon Aug 22 15:48:20 2022] GET /identity/v3/projects/service => generated 88 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 22 15:48:20.302554 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76e31757-67ed-4109-8e0c-e77b352a83d1 None admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 15:48:20.424411 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76e31757-67ed-4109-8e0c-e77b352a83d1 None admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 15:48:20.449387 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=100 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 15:48:20.464867 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f14e57c0-d06c-4ee2-9894-3b2a2f432cfe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 15:48:20.465468 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f14e57c0-d06c-4ee2-9894-3b2a2f432cfe None admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:48:20.465819 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f14e57c0-d06c-4ee2-9894-3b2a2f432cfe None admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:48:20.466058 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f14e57c0-d06c-4ee2-9894-3b2a2f432cfe None admin] PATH_INFO: `/v3/projects` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:48:20.467099 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f14e57c0-d06c-4ee2-9894-3b2a2f432cfe None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81768) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 15:48:20.467473 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f14e57c0-d06c-4ee2-9894-3b2a2f432cfe None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 15:48:20.468952 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f14e57c0-d06c-4ee2-9894-3b2a2f432cfe None admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 15:48:20.484916 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 504/988] 10.222.0.15 () {56 vars in 1148 bytes} [Mon Aug 22 15:48:20 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 15:48:20.490748 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-401ad525-862e-477d-a6aa-3d3b85b69a4c None admin] Authenticating user token {{(pid=81767) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 15:48:20.504063 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG dbcounter [-] [81767] Writing DB stats keystone:SELECT=100 {{(pid=81767) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 15:48:20.604548 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-401ad525-862e-477d-a6aa-3d3b85b69a4c None admin] Validating token access rules against request {{(pid=81767) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 15:48:20.646819 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-115d91c8-4086-44e8-aed5-236bb9fcba4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81767) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 15:48:20.648059 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-115d91c8-4086-44e8-aed5-236bb9fcba4a None admin] REQUEST_METHOD: `GET` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:48:20.648495 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-115d91c8-4086-44e8-aed5-236bb9fcba4a None admin] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:48:20.648924 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-115d91c8-4086-44e8-aed5-236bb9fcba4a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:48:20.649932 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-115d91c8-4086-44e8-aed5-236bb9fcba4a None admin] RBAC: Adding query filter params (role.id=806eb8803ece4a1fa1bc2487b4cd7c02, scope.project.id=c4ee0413f9ef4d4fb1f95a660babe090, user.id=058e76354dce412cb78b2875c88f2791) {{(pid=81767) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 15:48:20.650415 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-115d91c8-4086-44e8-aed5-236bb9fcba4a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 15:48:20.652280 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-115d91c8-4086-44e8-aed5-236bb9fcba4a None admin] RBAC: Authorization granted {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 15:48:20.660253 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 485/989] 10.222.0.15 () {56 vars in 1374 bytes} [Mon Aug 22 15:48:20 2022] GET /identity/v3/role_assignments?user.id=058e76354dce412cb78b2875c88f2791&scope.project.id=c4ee0413f9ef4d4fb1f95a660babe090&role.id=806eb8803ece4a1fa1bc2487b4cd7c02 => generated 608 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 15:48:21.173518 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-060eda5c-04b4-419e-95b8-74ba7979cc75 None None] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:48:21.173892 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-060eda5c-04b4-419e-95b8-74ba7979cc75 None None] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:48:21.174092 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-060eda5c-04b4-419e-95b8-74ba7979cc75 None None] PATH_INFO: `/` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:48:21.175779 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 505/990] 10.222.0.15 () {54 vars in 927 bytes} [Mon Aug 22 15:48:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 15:48:21.182106 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5ef8ac5-9326-428d-a037-1b8bef949ea7 None None] REQUEST_METHOD: `POST` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:48:21.182819 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5ef8ac5-9326-428d-a037-1b8bef949ea7 None None] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:48:21.182997 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5ef8ac5-9326-428d-a037-1b8bef949ea7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:48:21.199224 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb257701-0709-4283-95c5-e9a4352ef707 None None] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:48:21.199530 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb257701-0709-4283-95c5-e9a4352ef707 None None] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:48:21.199804 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb257701-0709-4283-95c5-e9a4352ef707 None None] PATH_INFO: `/` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:48:21.201130 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 506/991] 10.222.0.15 () {54 vars in 927 bytes} [Mon Aug 22 15:48:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 15:48:21.204582 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.auth.core [None req-a5ef8ac5-9326-428d-a037-1b8bef949ea7 None None] MFA Rules not processed for user `6b2199ae0fb0480aa917ada391f5ec54`. Rule list: `[]` (Enabled: `True`). {{(pid=81767) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 15:48:21.206244 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.fernet_utils [None req-a5ef8ac5-9326-428d-a037-1b8bef949ea7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81767) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 15:48:21.206707 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b037c3f4-5b2a-48e7-ae6a-6c17cd65775e None None] REQUEST_METHOD: `POST` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:48:21.206988 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b037c3f4-5b2a-48e7-ae6a-6c17cd65775e None None] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:48:21.207315 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b037c3f4-5b2a-48e7-ae6a-6c17cd65775e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:48:21.228647 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.auth.core [None req-b037c3f4-5b2a-48e7-ae6a-6c17cd65775e None None] MFA Rules not processed for user `6b2199ae0fb0480aa917ada391f5ec54`. Rule list: `[]` (Enabled: `True`). {{(pid=81768) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 15:48:21.229964 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-b037c3f4-5b2a-48e7-ae6a-6c17cd65775e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 15:48:21.286788 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 486/992] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Aug 22 15:48:21 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 22 15:48:21.310694 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 507/993] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Aug 22 15:48:21 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 22 15:48:21.557084 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5ef8ac5-9326-428d-a037-1b8bef949ea7 None None] Authenticating user token {{(pid=81767) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 15:48:21.559536 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG dbcounter [-] [81767] Writing DB stats keystone:SELECT=100 {{(pid=81767) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 15:48:21.576198 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b037c3f4-5b2a-48e7-ae6a-6c17cd65775e None None] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 15:48:21.683506 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5ef8ac5-9326-428d-a037-1b8bef949ea7 None None] Validating token access rules against request {{(pid=81767) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 15:48:21.699767 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b037c3f4-5b2a-48e7-ae6a-6c17cd65775e None None] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 15:48:21.719211 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=100 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 15:48:21.728912 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e94526c4-3baf-4c70-808e-3691eff72af8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81767) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 15:48:21.730227 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e94526c4-3baf-4c70-808e-3691eff72af8 None admin] REQUEST_METHOD: `GET` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:48:21.730856 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e94526c4-3baf-4c70-808e-3691eff72af8 None admin] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:48:21.731508 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e94526c4-3baf-4c70-808e-3691eff72af8 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:48:21.732492 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e94526c4-3baf-4c70-808e-3691eff72af8 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 15:48:21.734987 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e94526c4-3baf-4c70-808e-3691eff72af8 None admin] RBAC: Authorization granted {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 15:48:21.752980 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application [None req-e94526c4-3baf-4c70-808e-3691eff72af8 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 22 15:48:21.752980 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 15:48:21.752980 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 22 15:48:21.752980 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 22 15:48:21.752980 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 22 15:48:21.752980 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 22 15:48:21.752980 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 22 15:48:21.752980 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 22 15:48:21.752980 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 22 15:48:21.752980 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application Aug 22 15:48:21.752980 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 22 15:48:21.752980 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application Aug 22 15:48:21.752980 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 15:48:21.752980 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 15:48:21.752980 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 15:48:21.752980 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 15:48:21.752980 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 15:48:21.754956 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 15:48:21.754956 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 15:48:21.754956 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 15:48:21.754956 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 15:48:21.754956 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 15:48:21.754956 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 15:48:21.754956 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 22 15:48:21.754956 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 22 15:48:21.754956 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 22 15:48:21.754956 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 22 15:48:21.754956 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 15:48:21.754956 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 15:48:21.754956 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 15:48:21.754956 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 15:48:21.754956 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 15:48:21.754956 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 15:48:21.756638 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 15:48:21.756638 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application with Lock( Aug 22 15:48:21.756638 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 15:48:21.756638 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return self._enter() Aug 22 15:48:21.756638 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 15:48:21.756638 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 15:48:21.756638 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 15:48:21.756638 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return self.creator() Aug 22 15:48:21.756638 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 15:48:21.756638 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application created_value = creator( Aug 22 15:48:21.756638 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 22 15:48:21.756638 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 22 15:48:21.756638 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 22 15:48:21.756638 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application Aug 22 15:48:21.758033 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5f8d472-d8d4-4160-8997-8dec06374e51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 15:48:21.758033 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5f8d472-d8d4-4160-8997-8dec06374e51 None admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:48:21.758033 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5f8d472-d8d4-4160-8997-8dec06374e51 None admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:48:21.758033 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5f8d472-d8d4-4160-8997-8dec06374e51 None admin] PATH_INFO: `/v3/services/compute` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:48:21.758530 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5f8d472-d8d4-4160-8997-8dec06374e51 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 15:48:21.759523 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5f8d472-d8d4-4160-8997-8dec06374e51 None admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 15:48:21.761346 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 487/994] 10.222.0.15 () {56 vars in 1126 bytes} [Mon Aug 22 15:48:21 2022] GET /identity/v3/services/object-store => generated 93 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 15:48:21.762700 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e94526c4-3baf-4c70-808e-3691eff72af8 None admin] Authenticating user token {{(pid=81767) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 15:48:21.773526 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application [None req-a5f8d472-d8d4-4160-8997-8dec06374e51 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Aug 22 15:48:21.773526 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 15:48:21.773526 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 22 15:48:21.773526 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 22 15:48:21.773526 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 22 15:48:21.773526 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 22 15:48:21.773526 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 22 15:48:21.773526 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 22 15:48:21.773526 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 22 15:48:21.773526 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application Aug 22 15:48:21.773526 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 22 15:48:21.773526 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application Aug 22 15:48:21.773526 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 15:48:21.773526 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 15:48:21.773526 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 15:48:21.773526 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 15:48:21.773526 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 15:48:21.774664 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 15:48:21.774664 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 15:48:21.774664 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 15:48:21.774664 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 15:48:21.774664 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 15:48:21.774664 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 15:48:21.774664 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 22 15:48:21.774664 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 22 15:48:21.774664 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 22 15:48:21.774664 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 22 15:48:21.774664 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 15:48:21.774664 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 15:48:21.774664 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 15:48:21.774664 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 15:48:21.774664 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 15:48:21.774664 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 15:48:21.775597 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 15:48:21.775597 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application with Lock( Aug 22 15:48:21.775597 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 15:48:21.775597 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self._enter() Aug 22 15:48:21.775597 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 15:48:21.775597 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 15:48:21.775597 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 15:48:21.775597 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self.creator() Aug 22 15:48:21.775597 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 15:48:21.775597 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application created_value = creator( Aug 22 15:48:21.775597 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 22 15:48:21.775597 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 22 15:48:21.775597 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 22 15:48:21.775597 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application Aug 22 15:48:21.775597 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 508/995] 10.222.0.15 () {56 vars in 1111 bytes} [Mon Aug 22 15:48:21 2022] GET /identity/v3/services/compute => generated 88 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 22 15:48:21.780811 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5f8d472-d8d4-4160-8997-8dec06374e51 None admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 15:48:21.855159 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG dbcounter [-] [81767] Writing DB stats keystone:SELECT=100 {{(pid=81767) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 15:48:21.906484 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e94526c4-3baf-4c70-808e-3691eff72af8 None admin] Validating token access rules against request {{(pid=81767) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 15:48:21.914803 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5f8d472-d8d4-4160-8997-8dec06374e51 None admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 15:48:21.962814 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b8747b8-daad-4c45-878c-c12152dc16c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 15:48:21.963070 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41abf380-7413-47c5-99d1-cab2ff924436 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81767) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 15:48:21.963719 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b8747b8-daad-4c45-878c-c12152dc16c3 None admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:48:21.963824 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41abf380-7413-47c5-99d1-cab2ff924436 None admin] REQUEST_METHOD: `GET` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:48:21.964157 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b8747b8-daad-4c45-878c-c12152dc16c3 None admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:48:21.964270 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41abf380-7413-47c5-99d1-cab2ff924436 None admin] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:48:21.964588 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b8747b8-daad-4c45-878c-c12152dc16c3 None admin] PATH_INFO: `/v3/services` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:48:21.964700 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41abf380-7413-47c5-99d1-cab2ff924436 None admin] PATH_INFO: `/v3/services` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:48:21.965598 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b8747b8-daad-4c45-878c-c12152dc16c3 None admin] RBAC: Adding query filter params (name=compute) {{(pid=81768) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 15:48:21.965707 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41abf380-7413-47c5-99d1-cab2ff924436 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81767) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 15:48:21.966079 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41abf380-7413-47c5-99d1-cab2ff924436 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 15:48:21.966185 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b8747b8-daad-4c45-878c-c12152dc16c3 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 15:48:21.968390 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41abf380-7413-47c5-99d1-cab2ff924436 None admin] RBAC: Authorization granted {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 15:48:21.968505 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b8747b8-daad-4c45-878c-c12152dc16c3 None admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 15:48:21.978409 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 509/996] 10.222.0.15 () {56 vars in 1112 bytes} [Mon Aug 22 15:48:21 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 22 15:48:21.978590 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 488/997] 10.222.0.15 () {56 vars in 1122 bytes} [Mon Aug 22 15:48:21 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 15:48:21.986276 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b8747b8-daad-4c45-878c-c12152dc16c3 None admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 15:48:21.986530 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41abf380-7413-47c5-99d1-cab2ff924436 None admin] Authenticating user token {{(pid=81767) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 15:48:22.036287 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=100 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 15:48:22.119091 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b8747b8-daad-4c45-878c-c12152dc16c3 None admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 15:48:22.120601 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41abf380-7413-47c5-99d1-cab2ff924436 None admin] Validating token access rules against request {{(pid=81767) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 15:48:22.162064 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-051a19c9-de9f-4e49-b881-04c7851b5fa3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 15:48:22.162840 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG dbcounter [-] [81767] Writing DB stats keystone:SELECT=100 {{(pid=81767) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 15:48:22.162959 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-051a19c9-de9f-4e49-b881-04c7851b5fa3 None admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:48:22.163298 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-051a19c9-de9f-4e49-b881-04c7851b5fa3 None admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:48:22.163921 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-051a19c9-de9f-4e49-b881-04c7851b5fa3 None admin] PATH_INFO: `/v3/services` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:48:22.164304 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-051a19c9-de9f-4e49-b881-04c7851b5fa3 None admin] RBAC: Adding query filter params (type=compute) {{(pid=81768) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 15:48:22.164600 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-051a19c9-de9f-4e49-b881-04c7851b5fa3 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 15:48:22.166564 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-051a19c9-de9f-4e49-b881-04c7851b5fa3 None admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 15:48:22.171597 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88844467-598e-419b-b673-40754b077691 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81767) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 15:48:22.172176 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 510/998] 10.222.0.15 () {56 vars in 1112 bytes} [Mon Aug 22 15:48:21 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 15:48:22.172628 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88844467-598e-419b-b673-40754b077691 None admin] REQUEST_METHOD: `GET` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:48:22.186198 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88844467-598e-419b-b673-40754b077691 None admin] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:48:22.186198 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88844467-598e-419b-b673-40754b077691 None admin] PATH_INFO: `/v3/services` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:48:22.186198 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88844467-598e-419b-b673-40754b077691 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81767) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 15:48:22.186198 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88844467-598e-419b-b673-40754b077691 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 15:48:22.186198 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88844467-598e-419b-b673-40754b077691 None admin] RBAC: Authorization granted {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 15:48:22.187056 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-051a19c9-de9f-4e49-b881-04c7851b5fa3 None admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 15:48:22.187159 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 489/999] 10.222.0.15 () {56 vars in 1122 bytes} [Mon Aug 22 15:48:21 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 15:48:22.192012 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88844467-598e-419b-b673-40754b077691 None admin] Authenticating user token {{(pid=81767) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 15:48:22.312191 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-051a19c9-de9f-4e49-b881-04c7851b5fa3 None admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 15:48:22.315538 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=100 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 15:48:22.316947 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88844467-598e-419b-b673-40754b077691 None admin] Validating token access rules against request {{(pid=81767) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 15:48:22.360609 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a348b625-fcc0-498e-a63f-73ce1f805757 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 15:48:22.361282 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a348b625-fcc0-498e-a63f-73ce1f805757 None admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:48:22.361573 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a348b625-fcc0-498e-a63f-73ce1f805757 None admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:48:22.361894 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a348b625-fcc0-498e-a63f-73ce1f805757 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:48:22.362656 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a348b625-fcc0-498e-a63f-73ce1f805757 None admin] RBAC: Adding query filter params (interface=public, service_id=48b195dbe84f4b36939758730022a5cb, region_id=RegionOne) {{(pid=81768) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 15:48:22.362984 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a348b625-fcc0-498e-a63f-73ce1f805757 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 15:48:22.364592 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a348b625-fcc0-498e-a63f-73ce1f805757 None admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 15:48:22.368010 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9ab818b-b822-4c68-b319-9418d41451c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81767) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 15:48:22.369484 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9ab818b-b822-4c68-b319-9418d41451c1 None admin] REQUEST_METHOD: `POST` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:48:22.369591 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9ab818b-b822-4c68-b319-9418d41451c1 None admin] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:48:22.369852 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9ab818b-b822-4c68-b319-9418d41451c1 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:48:22.370494 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9ab818b-b822-4c68-b319-9418d41451c1 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 15:48:22.371879 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 511/1000] 10.222.0.15 () {56 vars in 1251 bytes} [Mon Aug 22 15:48:22 2022] GET /identity/v3/endpoints?service_id=48b195dbe84f4b36939758730022a5cb&interface=public®ion_id=RegionOne => generated 194 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 15:48:22.372002 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9ab818b-b822-4c68-b319-9418d41451c1 None admin] RBAC: Authorization granted {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 15:48:22.377123 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a348b625-fcc0-498e-a63f-73ce1f805757 None admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 15:48:22.497966 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a348b625-fcc0-498e-a63f-73ce1f805757 None admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 15:48:22.523824 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 490/1001] 10.222.0.15 () {60 vars in 1144 bytes} [Mon Aug 22 15:48:22 2022] POST /identity/v3/endpoints => generated 325 bytes in 332 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 22 15:48:22.544347 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ac16c4e-ea75-4fad-9663-23b95a44a534 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 15:48:22.545018 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac16c4e-ea75-4fad-9663-23b95a44a534 None admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:48:22.545388 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac16c4e-ea75-4fad-9663-23b95a44a534 None admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:48:22.545566 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac16c4e-ea75-4fad-9663-23b95a44a534 None admin] PATH_INFO: `/v3/services` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:48:22.546190 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ac16c4e-ea75-4fad-9663-23b95a44a534 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 15:48:22.547646 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ac16c4e-ea75-4fad-9663-23b95a44a534 None admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 15:48:22.561949 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 512/1002] 10.222.0.15 () {56 vars in 1087 bytes} [Mon Aug 22 15:48:22 2022] GET /identity/v3/services => generated 1727 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 22 15:48:22.706502 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19a35582-bd9e-49b6-9c10-39e564239ee5 None None] REQUEST_METHOD: `GET` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:48:22.706748 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19a35582-bd9e-49b6-9c10-39e564239ee5 None None] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:48:22.707024 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19a35582-bd9e-49b6-9c10-39e564239ee5 None None] PATH_INFO: `/` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:48:22.708536 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 491/1003] 10.222.0.15 () {54 vars in 927 bytes} [Mon Aug 22 15:48:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 15:48:22.715411 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80002db1-379d-4e04-95d7-c41896811bb3 None None] REQUEST_METHOD: `POST` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:48:22.715684 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80002db1-379d-4e04-95d7-c41896811bb3 None None] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:48:22.715956 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80002db1-379d-4e04-95d7-c41896811bb3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:48:22.740908 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.auth.core [None req-80002db1-379d-4e04-95d7-c41896811bb3 None None] MFA Rules not processed for user `6b2199ae0fb0480aa917ada391f5ec54`. Rule list: `[]` (Enabled: `True`). {{(pid=81768) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 15:48:22.742097 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-80002db1-379d-4e04-95d7-c41896811bb3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 15:48:22.775299 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=100 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 15:48:22.837790 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 513/1004] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Aug 22 15:48:22 2022] POST /identity/v3/auth/tokens => generated 1717 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 22 15:48:23.150756 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19a35582-bd9e-49b6-9c10-39e564239ee5 None None] Authenticating user token {{(pid=81767) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 15:48:23.217458 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG dbcounter [-] [81767] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81767) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 15:48:23.300748 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19a35582-bd9e-49b6-9c10-39e564239ee5 None None] Validating token access rules against request {{(pid=81767) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 15:48:23.347236 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5afce09d-829f-4fa6-a55b-160d2af9c81f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81767) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 15:48:23.347873 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5afce09d-829f-4fa6-a55b-160d2af9c81f None admin] REQUEST_METHOD: `GET` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:48:23.348149 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5afce09d-829f-4fa6-a55b-160d2af9c81f None admin] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:48:23.348422 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5afce09d-829f-4fa6-a55b-160d2af9c81f None admin] PATH_INFO: `/v3/services/image` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:48:23.349014 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5afce09d-829f-4fa6-a55b-160d2af9c81f None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 15:48:23.350382 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5afce09d-829f-4fa6-a55b-160d2af9c81f None admin] RBAC: Authorization granted {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 15:48:23.365740 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application [None req-5afce09d-829f-4fa6-a55b-160d2af9c81f None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Aug 22 15:48:23.365740 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 15:48:23.365740 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 22 15:48:23.365740 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 22 15:48:23.365740 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 22 15:48:23.365740 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 22 15:48:23.365740 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 22 15:48:23.365740 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 22 15:48:23.365740 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 22 15:48:23.365740 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application Aug 22 15:48:23.365740 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 22 15:48:23.365740 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application Aug 22 15:48:23.365740 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 15:48:23.365740 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 15:48:23.365740 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 15:48:23.365740 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 15:48:23.365740 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 15:48:23.367432 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 15:48:23.367432 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 15:48:23.367432 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 15:48:23.367432 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 15:48:23.367432 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 15:48:23.367432 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 15:48:23.367432 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 22 15:48:23.367432 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 22 15:48:23.367432 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 22 15:48:23.367432 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 22 15:48:23.367432 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 15:48:23.367432 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 15:48:23.367432 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 15:48:23.367432 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 15:48:23.367432 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 15:48:23.367432 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 15:48:23.368376 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 15:48:23.368376 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application with Lock( Aug 22 15:48:23.368376 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 15:48:23.368376 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return self._enter() Aug 22 15:48:23.368376 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 15:48:23.368376 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 15:48:23.368376 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 15:48:23.368376 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return self.creator() Aug 22 15:48:23.368376 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 15:48:23.368376 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application created_value = creator( Aug 22 15:48:23.368376 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 22 15:48:23.368376 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 22 15:48:23.368376 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 22 15:48:23.368376 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application Aug 22 15:48:23.368376 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 492/1005] 10.222.0.15 () {56 vars in 1105 bytes} [Mon Aug 22 15:48:23 2022] GET /identity/v3/services/image => generated 86 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 15:48:23.373495 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80002db1-379d-4e04-95d7-c41896811bb3 None None] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 15:48:23.505597 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80002db1-379d-4e04-95d7-c41896811bb3 None None] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 15:48:23.554724 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b44fbb2-47d2-4299-9228-cbc1d197959b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 15:48:23.554724 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b44fbb2-47d2-4299-9228-cbc1d197959b None admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:48:23.554724 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b44fbb2-47d2-4299-9228-cbc1d197959b None admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:48:23.555106 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b44fbb2-47d2-4299-9228-cbc1d197959b None admin] PATH_INFO: `/v3/services` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:48:23.556130 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b44fbb2-47d2-4299-9228-cbc1d197959b None admin] RBAC: Adding query filter params (name=image) {{(pid=81768) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 15:48:23.556513 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b44fbb2-47d2-4299-9228-cbc1d197959b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 15:48:23.558235 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b44fbb2-47d2-4299-9228-cbc1d197959b None admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 15:48:23.568920 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 514/1006] 10.222.0.15 () {56 vars in 1108 bytes} [Mon Aug 22 15:48:23 2022] GET /identity/v3/services?name=image => generated 122 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 15:48:23.570252 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5afce09d-829f-4fa6-a55b-160d2af9c81f None admin] Authenticating user token {{(pid=81767) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 15:48:23.715592 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5afce09d-829f-4fa6-a55b-160d2af9c81f None admin] Validating token access rules against request {{(pid=81767) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 15:48:23.738858 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG dbcounter [-] [81767] Writing DB stats keystone:SELECT=100 {{(pid=81767) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 15:48:23.761128 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09b91e2a-869c-4876-a68d-7cb193ed7ff2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81767) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 15:48:23.761642 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b91e2a-869c-4876-a68d-7cb193ed7ff2 None admin] REQUEST_METHOD: `GET` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:48:23.761920 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b91e2a-869c-4876-a68d-7cb193ed7ff2 None admin] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:48:23.762165 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b91e2a-869c-4876-a68d-7cb193ed7ff2 None admin] PATH_INFO: `/v3/services` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:48:23.762918 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09b91e2a-869c-4876-a68d-7cb193ed7ff2 None admin] RBAC: Adding query filter params (type=image) {{(pid=81767) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 15:48:23.763214 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09b91e2a-869c-4876-a68d-7cb193ed7ff2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 15:48:23.764606 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09b91e2a-869c-4876-a68d-7cb193ed7ff2 None admin] RBAC: Authorization granted {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 15:48:23.769663 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 493/1007] 10.222.0.15 () {56 vars in 1108 bytes} [Mon Aug 22 15:48:23 2022] GET /identity/v3/services?type=image => generated 122 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 15:48:24.590167 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7501e801-2714-4d2f-87f3-7f45ca2d0164 None None] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:48:24.590728 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7501e801-2714-4d2f-87f3-7f45ca2d0164 None None] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:48:24.590728 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7501e801-2714-4d2f-87f3-7f45ca2d0164 None None] PATH_INFO: `/` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:48:24.592154 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 515/1008] 10.222.0.15 () {54 vars in 927 bytes} [Mon Aug 22 15:48:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 22 15:48:24.597798 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee2e625f-d39e-4bd7-87c9-c718b5de552f None None] REQUEST_METHOD: `POST` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:48:24.598251 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee2e625f-d39e-4bd7-87c9-c718b5de552f None None] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:48:24.598326 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee2e625f-d39e-4bd7-87c9-c718b5de552f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:48:24.606505 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3726103c-597d-4a22-a1b8-6df45feb3fbf None None] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:48:24.606776 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3726103c-597d-4a22-a1b8-6df45feb3fbf None None] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:48:24.607230 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3726103c-597d-4a22-a1b8-6df45feb3fbf None None] PATH_INFO: `/` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:48:24.608981 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 516/1009] 10.222.0.15 () {54 vars in 927 bytes} [Mon Aug 22 15:48:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 15:48:24.614583 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-069f18eb-938e-4516-9f0c-51cd7b5098b1 None None] REQUEST_METHOD: `POST` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:48:24.614923 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-069f18eb-938e-4516-9f0c-51cd7b5098b1 None None] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:48:24.615114 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-069f18eb-938e-4516-9f0c-51cd7b5098b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:48:24.617191 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.auth.core [None req-ee2e625f-d39e-4bd7-87c9-c718b5de552f None None] MFA Rules not processed for user `6b2199ae0fb0480aa917ada391f5ec54`. Rule list: `[]` (Enabled: `True`). {{(pid=81767) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 15:48:24.618756 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.fernet_utils [None req-ee2e625f-d39e-4bd7-87c9-c718b5de552f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81767) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 15:48:24.635424 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.auth.core [None req-069f18eb-938e-4516-9f0c-51cd7b5098b1 None None] MFA Rules not processed for user `6b2199ae0fb0480aa917ada391f5ec54`. Rule list: `[]` (Enabled: `True`). {{(pid=81768) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 15:48:24.636614 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-069f18eb-938e-4516-9f0c-51cd7b5098b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 15:48:24.657497 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=100 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 15:48:24.715302 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 494/1010] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Aug 22 15:48:24 2022] POST /identity/v3/auth/tokens => generated 1717 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 22 15:48:24.717009 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 517/1011] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Aug 22 15:48:24 2022] POST /identity/v3/auth/tokens => generated 1717 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 22 15:48:24.986921 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee2e625f-d39e-4bd7-87c9-c718b5de552f None None] Authenticating user token {{(pid=81767) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 15:48:25.084990 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-069f18eb-938e-4516-9f0c-51cd7b5098b1 None None] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 15:48:25.142514 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee2e625f-d39e-4bd7-87c9-c718b5de552f None None] Validating token access rules against request {{(pid=81767) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 15:48:25.166542 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG dbcounter [-] [81767] Writing DB stats keystone:SELECT=100 {{(pid=81767) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 15:48:25.204274 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8526d11-9b80-4638-953b-ac2926b4c174 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81767) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 15:48:25.205533 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8526d11-9b80-4638-953b-ac2926b4c174 None admin] REQUEST_METHOD: `GET` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:48:25.206029 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8526d11-9b80-4638-953b-ac2926b4c174 None admin] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:48:25.206484 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8526d11-9b80-4638-953b-ac2926b4c174 None admin] PATH_INFO: `/v3/services/compute` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:48:25.207637 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8526d11-9b80-4638-953b-ac2926b4c174 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 15:48:25.210039 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8526d11-9b80-4638-953b-ac2926b4c174 None admin] RBAC: Authorization granted {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 15:48:25.221798 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application [None req-c8526d11-9b80-4638-953b-ac2926b4c174 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Aug 22 15:48:25.221798 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 15:48:25.221798 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 22 15:48:25.221798 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 22 15:48:25.221798 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 22 15:48:25.221798 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 22 15:48:25.221798 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 22 15:48:25.221798 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 22 15:48:25.221798 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 22 15:48:25.221798 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application Aug 22 15:48:25.221798 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 22 15:48:25.221798 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application Aug 22 15:48:25.221798 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 15:48:25.221798 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 15:48:25.221798 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 15:48:25.221798 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 15:48:25.221798 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 15:48:25.222888 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 15:48:25.222888 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 15:48:25.222888 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 15:48:25.222888 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 15:48:25.222888 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 15:48:25.222888 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 15:48:25.222888 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 22 15:48:25.222888 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 22 15:48:25.222888 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 22 15:48:25.222888 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 22 15:48:25.222888 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 15:48:25.222888 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 15:48:25.222888 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 15:48:25.222888 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 15:48:25.222888 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 15:48:25.222888 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 15:48:25.223960 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 15:48:25.223960 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application with Lock( Aug 22 15:48:25.223960 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 15:48:25.223960 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return self._enter() Aug 22 15:48:25.223960 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 15:48:25.223960 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 15:48:25.223960 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 15:48:25.223960 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return self.creator() Aug 22 15:48:25.223960 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 15:48:25.223960 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application created_value = creator( Aug 22 15:48:25.223960 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 22 15:48:25.223960 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 22 15:48:25.223960 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 22 15:48:25.223960 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application Aug 22 15:48:25.223960 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 495/1012] 10.222.0.15 () {56 vars in 1111 bytes} [Mon Aug 22 15:48:24 2022] GET /identity/v3/services/compute => generated 88 bytes in 237 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 15:48:25.228246 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-069f18eb-938e-4516-9f0c-51cd7b5098b1 None None] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 15:48:25.232325 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8526d11-9b80-4638-953b-ac2926b4c174 None admin] Authenticating user token {{(pid=81767) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 15:48:25.281199 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2879c07a-b003-437a-ad03-007cfa1fec6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 15:48:25.282082 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2879c07a-b003-437a-ad03-007cfa1fec6c None admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:48:25.282399 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2879c07a-b003-437a-ad03-007cfa1fec6c None admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:48:25.282803 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2879c07a-b003-437a-ad03-007cfa1fec6c None admin] PATH_INFO: `/v3/domains/default` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:48:25.284499 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2879c07a-b003-437a-ad03-007cfa1fec6c None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 15:48:25.286511 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2879c07a-b003-437a-ad03-007cfa1fec6c None admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 15:48:25.293804 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 518/1013] 10.222.0.15 () {56 vars in 1108 bytes} [Mon Aug 22 15:48:25 2022] GET /identity/v3/domains/default => generated 199 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 15:48:25.295320 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2879c07a-b003-437a-ad03-007cfa1fec6c None admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 15:48:25.327969 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=100 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 15:48:25.373249 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8526d11-9b80-4638-953b-ac2926b4c174 None admin] Validating token access rules against request {{(pid=81767) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 15:48:25.429342 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c66f6f8-9e82-48ad-806e-02361693c2d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81767) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 15:48:25.430308 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c66f6f8-9e82-48ad-806e-02361693c2d8 None admin] REQUEST_METHOD: `GET` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:48:25.431405 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c66f6f8-9e82-48ad-806e-02361693c2d8 None admin] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:48:25.431874 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c66f6f8-9e82-48ad-806e-02361693c2d8 None admin] PATH_INFO: `/v3/services` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:48:25.433182 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c66f6f8-9e82-48ad-806e-02361693c2d8 None admin] RBAC: Adding query filter params (name=compute) {{(pid=81767) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 15:48:25.433697 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c66f6f8-9e82-48ad-806e-02361693c2d8 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 15:48:25.436083 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c66f6f8-9e82-48ad-806e-02361693c2d8 None admin] RBAC: Authorization granted {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 15:48:25.445133 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 496/1014] 10.222.0.15 () {56 vars in 1112 bytes} [Mon Aug 22 15:48:25 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 22 15:48:25.453239 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c66f6f8-9e82-48ad-806e-02361693c2d8 None admin] Authenticating user token {{(pid=81767) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 15:48:25.471042 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2879c07a-b003-437a-ad03-007cfa1fec6c None admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 15:48:25.518616 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG dbcounter [-] [81767] Writing DB stats keystone:SELECT=100 {{(pid=81767) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 15:48:25.538773 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5bae706-b260-47c2-9a9a-297ad1083b73 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 15:48:25.539725 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5bae706-b260-47c2-9a9a-297ad1083b73 None admin] REQUEST_METHOD: `POST` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:48:25.540675 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5bae706-b260-47c2-9a9a-297ad1083b73 None admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:48:25.541083 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5bae706-b260-47c2-9a9a-297ad1083b73 None admin] PATH_INFO: `/v3/projects` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:48:25.542195 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5bae706-b260-47c2-9a9a-297ad1083b73 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 15:48:25.544676 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5bae706-b260-47c2-9a9a-297ad1083b73 None admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 15:48:25.610003 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c66f6f8-9e82-48ad-806e-02361693c2d8 None admin] Validating token access rules against request {{(pid=81767) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 15:48:25.661210 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f623770-0498-4beb-96f1-4f1f39cc0fcf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81767) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 15:48:25.662417 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f623770-0498-4beb-96f1-4f1f39cc0fcf None admin] REQUEST_METHOD: `GET` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:48:25.662951 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f623770-0498-4beb-96f1-4f1f39cc0fcf None admin] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:48:25.663538 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f623770-0498-4beb-96f1-4f1f39cc0fcf None admin] PATH_INFO: `/v3/services` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:48:25.664663 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f623770-0498-4beb-96f1-4f1f39cc0fcf None admin] RBAC: Adding query filter params (type=compute) {{(pid=81767) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 15:48:25.665677 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f623770-0498-4beb-96f1-4f1f39cc0fcf None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 15:48:25.668308 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f623770-0498-4beb-96f1-4f1f39cc0fcf None admin] RBAC: Authorization granted {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 15:48:25.674705 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 497/1015] 10.222.0.15 () {56 vars in 1112 bytes} [Mon Aug 22 15:48:25 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 15:48:25.682085 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f623770-0498-4beb-96f1-4f1f39cc0fcf None admin] Authenticating user token {{(pid=81767) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 15:48:25.708425 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 519/1016] 10.222.0.15 () {60 vars in 1141 bytes} [Mon Aug 22 15:48:25 2022] POST /identity/v3/projects => generated 311 bytes in 413 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 22 15:48:25.785881 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8414c1e1-2d1f-45f6-b261-4b681ffd2bc5 None None] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:48:25.786263 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8414c1e1-2d1f-45f6-b261-4b681ffd2bc5 None None] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:48:25.786594 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8414c1e1-2d1f-45f6-b261-4b681ffd2bc5 None None] PATH_INFO: `/` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:48:25.788460 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 520/1017] 10.222.0.15 () {54 vars in 927 bytes} [Mon Aug 22 15:48:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 22 15:48:25.795367 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-484edba3-c995-4734-8d91-1f3e2e317f5f None None] REQUEST_METHOD: `POST` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:48:25.795569 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-484edba3-c995-4734-8d91-1f3e2e317f5f None None] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:48:25.795828 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-484edba3-c995-4734-8d91-1f3e2e317f5f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:48:25.820938 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.auth.core [None req-484edba3-c995-4734-8d91-1f3e2e317f5f None None] MFA Rules not processed for user `6b2199ae0fb0480aa917ada391f5ec54`. Rule list: `[]` (Enabled: `True`). {{(pid=81768) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 15:48:25.822740 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-484edba3-c995-4734-8d91-1f3e2e317f5f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 15:48:25.870277 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f623770-0498-4beb-96f1-4f1f39cc0fcf None admin] Validating token access rules against request {{(pid=81767) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 15:48:25.875104 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG dbcounter [-] [81767] Writing DB stats keystone:SELECT=100 {{(pid=81767) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 15:48:25.924998 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c46ef35-1847-42d2-bed6-5a9e4fc75653 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81767) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 15:48:25.925814 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c46ef35-1847-42d2-bed6-5a9e4fc75653 None admin] REQUEST_METHOD: `POST` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:48:25.926248 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c46ef35-1847-42d2-bed6-5a9e4fc75653 None admin] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:48:25.926766 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c46ef35-1847-42d2-bed6-5a9e4fc75653 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:48:25.928625 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c46ef35-1847-42d2-bed6-5a9e4fc75653 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 15:48:25.928798 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 521/1018] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Aug 22 15:48:25 2022] POST /identity/v3/auth/tokens => generated 1717 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 22 15:48:25.930544 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c46ef35-1847-42d2-bed6-5a9e4fc75653 None admin] RBAC: Authorization granted {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 15:48:26.059379 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 498/1019] 10.222.0.15 () {60 vars in 1144 bytes} [Mon Aug 22 15:48:25 2022] POST /identity/v3/endpoints => generated 334 bytes in 378 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 22 15:48:26.222572 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-484edba3-c995-4734-8d91-1f3e2e317f5f None None] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 15:48:26.225294 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 15:48:26.379779 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-484edba3-c995-4734-8d91-1f3e2e317f5f None None] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 15:48:26.435094 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba77673f-72ee-412a-9f31-be7faeaba9fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 15:48:26.435992 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba77673f-72ee-412a-9f31-be7faeaba9fe None admin] REQUEST_METHOD: `POST` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:48:26.436442 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba77673f-72ee-412a-9f31-be7faeaba9fe None admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:48:26.436852 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba77673f-72ee-412a-9f31-be7faeaba9fe None admin] PATH_INFO: `/v3/services` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:48:26.437853 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba77673f-72ee-412a-9f31-be7faeaba9fe None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 15:48:26.439544 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba77673f-72ee-412a-9f31-be7faeaba9fe None admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 15:48:26.521182 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 522/1020] 10.222.0.15 () {60 vars in 1141 bytes} [Mon Aug 22 15:48:26 2022] POST /identity/v3/services => generated 242 bytes in 299 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 22 15:48:27.803143 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f7a17aa-8623-4252-a5b7-e56b412bd9e0 None None] REQUEST_METHOD: `GET` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:48:27.803985 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f7a17aa-8623-4252-a5b7-e56b412bd9e0 None None] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:48:27.804125 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f7a17aa-8623-4252-a5b7-e56b412bd9e0 None None] PATH_INFO: `/` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:48:27.807318 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 499/1021] 10.222.0.15 () {54 vars in 927 bytes} [Mon Aug 22 15:48:27 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 15:48:27.812370 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cd6da50-9199-4542-91b9-30a8dfa80a2c None None] REQUEST_METHOD: `POST` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:48:27.813140 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cd6da50-9199-4542-91b9-30a8dfa80a2c None None] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:48:27.813442 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cd6da50-9199-4542-91b9-30a8dfa80a2c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:48:27.833661 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.auth.core [None req-5cd6da50-9199-4542-91b9-30a8dfa80a2c None None] MFA Rules not processed for user `6b2199ae0fb0480aa917ada391f5ec54`. Rule list: `[]` (Enabled: `True`). {{(pid=81768) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 15:48:27.835081 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-5cd6da50-9199-4542-91b9-30a8dfa80a2c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 15:48:27.902907 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 15:48:27.931750 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 523/1022] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Aug 22 15:48:27 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 22 15:48:28.218488 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f7a17aa-8623-4252-a5b7-e56b412bd9e0 None None] Authenticating user token {{(pid=81767) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 15:48:28.243727 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be9dcb2-882b-42cb-84ff-a040ee84a1a6 None None] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:48:28.244074 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be9dcb2-882b-42cb-84ff-a040ee84a1a6 None None] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:48:28.244286 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be9dcb2-882b-42cb-84ff-a040ee84a1a6 None None] PATH_INFO: `/` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:48:28.245769 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 524/1023] 10.222.0.15 () {54 vars in 927 bytes} [Mon Aug 22 15:48:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 15:48:28.250958 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4b6e70d-14b2-44f7-a07a-240b09d6dad3 None None] REQUEST_METHOD: `POST` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:48:28.251239 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4b6e70d-14b2-44f7-a07a-240b09d6dad3 None None] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:48:28.251536 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4b6e70d-14b2-44f7-a07a-240b09d6dad3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:48:28.270926 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.auth.core [None req-b4b6e70d-14b2-44f7-a07a-240b09d6dad3 None None] MFA Rules not processed for user `6b2199ae0fb0480aa917ada391f5ec54`. Rule list: `[]` (Enabled: `True`). {{(pid=81768) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 15:48:28.272190 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-b4b6e70d-14b2-44f7-a07a-240b09d6dad3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 15:48:28.346653 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f7a17aa-8623-4252-a5b7-e56b412bd9e0 None None] Validating token access rules against request {{(pid=81767) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 15:48:28.373221 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 525/1024] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Aug 22 15:48:28 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 22 15:48:28.407910 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acc4c616-3ee0-46c9-b696-0e827f4bf020 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81767) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 15:48:28.408585 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acc4c616-3ee0-46c9-b696-0e827f4bf020 None admin] REQUEST_METHOD: `GET` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:48:28.408859 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acc4c616-3ee0-46c9-b696-0e827f4bf020 None admin] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:48:28.409133 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acc4c616-3ee0-46c9-b696-0e827f4bf020 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:48:28.410341 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acc4c616-3ee0-46c9-b696-0e827f4bf020 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 15:48:28.411818 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acc4c616-3ee0-46c9-b696-0e827f4bf020 None admin] RBAC: Authorization granted {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 15:48:28.413598 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 500/1025] 10.222.0.15 () {56 vars in 1108 bytes} [Mon Aug 22 15:48:28 2022] GET /identity/v3/domains/default => generated 199 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 15:48:28.419677 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4b6e70d-14b2-44f7-a07a-240b09d6dad3 None None] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 15:48:28.595295 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4b6e70d-14b2-44f7-a07a-240b09d6dad3 None None] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 15:48:28.622073 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=100 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 15:48:28.662397 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acc4c616-3ee0-46c9-b696-0e827f4bf020 None admin] Authenticating user token {{(pid=81767) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 15:48:28.663062 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64e749bb-2ad6-4da9-a2e5-b5d9137c4465 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 15:48:28.664133 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64e749bb-2ad6-4da9-a2e5-b5d9137c4465 None admin] REQUEST_METHOD: `POST` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:48:28.664624 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64e749bb-2ad6-4da9-a2e5-b5d9137c4465 None admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:48:28.665175 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64e749bb-2ad6-4da9-a2e5-b5d9137c4465 None admin] PATH_INFO: `/v3/users` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:48:28.666342 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64e749bb-2ad6-4da9-a2e5-b5d9137c4465 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 15:48:28.669216 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64e749bb-2ad6-4da9-a2e5-b5d9137c4465 None admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 15:48:28.712910 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG dbcounter [-] [81767] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81767) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 15:48:28.799550 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acc4c616-3ee0-46c9-b696-0e827f4bf020 None admin] Validating token access rules against request {{(pid=81767) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 15:48:28.825058 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 526/1026] 10.222.0.15 () {60 vars in 1132 bytes} [Mon Aug 22 15:48:28 2022] POST /identity/v3/users => generated 285 bytes in 406 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 22 15:48:28.852562 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56ef5ecf-190f-4bb2-8add-9ff3a1a81d23 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81767) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 15:48:28.853677 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ef5ecf-190f-4bb2-8add-9ff3a1a81d23 None admin] REQUEST_METHOD: `GET` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:48:28.854155 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ef5ecf-190f-4bb2-8add-9ff3a1a81d23 None admin] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:48:28.854621 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ef5ecf-190f-4bb2-8add-9ff3a1a81d23 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:48:28.876764 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56ef5ecf-190f-4bb2-8add-9ff3a1a81d23 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 22 15:48:28.886666 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application [None req-56ef5ecf-190f-4bb2-8add-9ff3a1a81d23 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 22 15:48:28.886666 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 15:48:28.886666 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 15:48:28.886666 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 15:48:28.886666 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 15:48:28.886666 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 15:48:28.886666 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 15:48:28.886666 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 15:48:28.886666 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 15:48:28.886666 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 15:48:28.886666 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 15:48:28.886666 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 15:48:28.886666 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 22 15:48:28.886666 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 22 15:48:28.886666 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 22 15:48:28.886666 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 22 15:48:28.887789 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 22 15:48:28.887789 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 22 15:48:28.887789 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 22 15:48:28.887789 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 22 15:48:28.887789 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 15:48:28.887789 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 15:48:28.887789 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 15:48:28.887789 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 15:48:28.887789 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 15:48:28.887789 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 15:48:28.887789 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 15:48:28.887789 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application with Lock( Aug 22 15:48:28.887789 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 15:48:28.887789 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return self._enter() Aug 22 15:48:28.887789 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 15:48:28.887789 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 15:48:28.888616 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 15:48:28.888616 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return self.creator() Aug 22 15:48:28.888616 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 15:48:28.888616 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application created_value = creator( Aug 22 15:48:28.888616 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 22 15:48:28.888616 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 22 15:48:28.888616 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 22 15:48:28.888616 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 22 15:48:28.888616 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 22 15:48:28.888616 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 22 15:48:28.888616 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 22 15:48:28.888616 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application Aug 22 15:48:28.891697 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 501/1027] 10.222.0.15 () {56 vars in 1120 bytes} [Mon Aug 22 15:48:28 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 230 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 22 15:48:28.898672 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64e749bb-2ad6-4da9-a2e5-b5d9137c4465 None admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 15:48:29.016886 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef454ba1-f92d-4931-9218-05f812cdcc0d None None] REQUEST_METHOD: `GET` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:48:29.017479 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef454ba1-f92d-4931-9218-05f812cdcc0d None None] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:48:29.017962 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef454ba1-f92d-4931-9218-05f812cdcc0d None None] PATH_INFO: `/` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:48:29.019977 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 502/1028] 10.222.0.15 () {54 vars in 927 bytes} [Mon Aug 22 15:48:29 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 22 15:48:29.026695 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1526fd10-f957-4d08-89a5-6a2c8fc53ef5 None None] REQUEST_METHOD: `POST` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:48:29.027008 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1526fd10-f957-4d08-89a5-6a2c8fc53ef5 None None] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:48:29.027239 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1526fd10-f957-4d08-89a5-6a2c8fc53ef5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:48:29.053999 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.auth.core [None req-1526fd10-f957-4d08-89a5-6a2c8fc53ef5 None None] MFA Rules not processed for user `6b2199ae0fb0480aa917ada391f5ec54`. Rule list: `[]` (Enabled: `True`). {{(pid=81767) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 15:48:29.056366 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.fernet_utils [None req-1526fd10-f957-4d08-89a5-6a2c8fc53ef5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81767) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 15:48:29.104066 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64e749bb-2ad6-4da9-a2e5-b5d9137c4465 None admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 15:48:29.166928 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a2a0f16-dcbe-494e-a292-18765b617d26 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 15:48:29.167937 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a2a0f16-dcbe-494e-a292-18765b617d26 None admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:48:29.168502 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a2a0f16-dcbe-494e-a292-18765b617d26 None admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:48:29.168996 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a2a0f16-dcbe-494e-a292-18765b617d26 None admin] PATH_INFO: `/v3/roles` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:48:29.169891 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a2a0f16-dcbe-494e-a292-18765b617d26 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81768) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 15:48:29.170431 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a2a0f16-dcbe-494e-a292-18765b617d26 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 15:48:29.170596 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 503/1029] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Aug 22 15:48:29 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 22 15:48:29.173382 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a2a0f16-dcbe-494e-a292-18765b617d26 None admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 15:48:29.194344 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 527/1030] 10.222.0.15 () {56 vars in 1115 bytes} [Mon Aug 22 15:48:28 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 292 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 15:48:29.198736 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1526fd10-f957-4d08-89a5-6a2c8fc53ef5 None None] Authenticating user token {{(pid=81767) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 15:48:29.235450 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG dbcounter [-] [81767] Writing DB stats keystone:SELECT=100 {{(pid=81767) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 15:48:29.361711 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1526fd10-f957-4d08-89a5-6a2c8fc53ef5 None None] Validating token access rules against request {{(pid=81767) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 15:48:29.429194 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf647196-7cbe-4f58-a208-ae6caa881d3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81767) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 15:48:29.430236 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf647196-7cbe-4f58-a208-ae6caa881d3c None admin] REQUEST_METHOD: `GET` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:48:29.431252 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf647196-7cbe-4f58-a208-ae6caa881d3c None admin] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:48:29.432475 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf647196-7cbe-4f58-a208-ae6caa881d3c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:48:29.452484 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf647196-7cbe-4f58-a208-ae6caa881d3c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 15:48:29.454896 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf647196-7cbe-4f58-a208-ae6caa881d3c None admin] RBAC: Authorization granted {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 15:48:29.481458 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application [None req-bf647196-7cbe-4f58-a208-ae6caa881d3c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 22 15:48:29.481458 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 15:48:29.481458 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 15:48:29.481458 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 15:48:29.481458 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 15:48:29.481458 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 15:48:29.481458 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 15:48:29.481458 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 15:48:29.481458 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 15:48:29.481458 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 15:48:29.481458 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 15:48:29.481458 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 15:48:29.481458 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 22 15:48:29.481458 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 22 15:48:29.481458 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 22 15:48:29.481458 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 22 15:48:29.483449 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 15:48:29.483449 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 15:48:29.483449 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 15:48:29.483449 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 15:48:29.483449 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 15:48:29.483449 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 15:48:29.483449 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 15:48:29.483449 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application with Lock( Aug 22 15:48:29.483449 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 15:48:29.483449 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return self._enter() Aug 22 15:48:29.483449 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 15:48:29.483449 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 15:48:29.483449 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 15:48:29.483449 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return self.creator() Aug 22 15:48:29.483449 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 15:48:29.483449 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application created_value = creator( Aug 22 15:48:29.483449 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 22 15:48:29.491451 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 22 15:48:29.491451 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 22 15:48:29.491451 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application Aug 22 15:48:29.491451 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 504/1031] 10.222.0.15 () {56 vars in 1108 bytes} [Mon Aug 22 15:48:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 286 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 15:48:29.496544 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a2a0f16-dcbe-494e-a292-18765b617d26 None admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 15:48:29.536866 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 15:48:29.602165 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf647196-7cbe-4f58-a208-ae6caa881d3c None admin] Authenticating user token {{(pid=81767) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 15:48:29.617691 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a2a0f16-dcbe-494e-a292-18765b617d26 None admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 15:48:29.668987 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c56a8405-4da8-471c-b192-61f75a551c6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 15:48:29.670976 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c56a8405-4da8-471c-b192-61f75a551c6b None admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:48:29.672055 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c56a8405-4da8-471c-b192-61f75a551c6b None admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:48:29.672966 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c56a8405-4da8-471c-b192-61f75a551c6b None admin] PATH_INFO: `/v3/domains` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:48:29.674592 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c56a8405-4da8-471c-b192-61f75a551c6b None admin] RBAC: Adding query filter params (name=Default) {{(pid=81768) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 15:48:29.675126 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c56a8405-4da8-471c-b192-61f75a551c6b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 15:48:29.678003 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c56a8405-4da8-471c-b192-61f75a551c6b None admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 15:48:29.702105 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 528/1032] 10.222.0.15 () {56 vars in 1109 bytes} [Mon Aug 22 15:48:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 15:48:29.711019 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c56a8405-4da8-471c-b192-61f75a551c6b None admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 15:48:29.718066 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG dbcounter [-] [81767] Writing DB stats keystone:SELECT=100 {{(pid=81767) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 15:48:29.766395 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf647196-7cbe-4f58-a208-ae6caa881d3c None admin] Validating token access rules against request {{(pid=81767) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 15:48:29.805762 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-872d91ad-d021-4be2-ab39-bfdb2cd94ccd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81767) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 15:48:29.806898 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-872d91ad-d021-4be2-ab39-bfdb2cd94ccd None admin] REQUEST_METHOD: `GET` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:48:29.807209 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-872d91ad-d021-4be2-ab39-bfdb2cd94ccd None admin] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:48:29.807486 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-872d91ad-d021-4be2-ab39-bfdb2cd94ccd None admin] PATH_INFO: `/v3/services/image` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:48:29.808270 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-872d91ad-d021-4be2-ab39-bfdb2cd94ccd None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 15:48:29.809644 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-872d91ad-d021-4be2-ab39-bfdb2cd94ccd None admin] RBAC: Authorization granted {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 15:48:29.820901 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application [None req-872d91ad-d021-4be2-ab39-bfdb2cd94ccd None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Aug 22 15:48:29.820901 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 15:48:29.820901 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 22 15:48:29.820901 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 22 15:48:29.820901 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 22 15:48:29.820901 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 22 15:48:29.820901 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 22 15:48:29.820901 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 22 15:48:29.820901 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 22 15:48:29.820901 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application Aug 22 15:48:29.820901 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 22 15:48:29.820901 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application Aug 22 15:48:29.820901 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 15:48:29.820901 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 15:48:29.820901 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 15:48:29.820901 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 15:48:29.820901 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 15:48:29.824149 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 15:48:29.824149 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 15:48:29.824149 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 15:48:29.824149 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 15:48:29.824149 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 15:48:29.824149 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 15:48:29.824149 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 22 15:48:29.824149 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 22 15:48:29.824149 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 22 15:48:29.824149 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 22 15:48:29.824149 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 15:48:29.824149 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 15:48:29.824149 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 15:48:29.824149 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 15:48:29.824149 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 15:48:29.824149 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 15:48:29.826569 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 15:48:29.826569 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application with Lock( Aug 22 15:48:29.826569 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 15:48:29.826569 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return self._enter() Aug 22 15:48:29.826569 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 15:48:29.826569 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 15:48:29.826569 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 15:48:29.826569 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return self.creator() Aug 22 15:48:29.826569 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 15:48:29.826569 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application created_value = creator( Aug 22 15:48:29.826569 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 22 15:48:29.826569 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 22 15:48:29.826569 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 22 15:48:29.826569 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application Aug 22 15:48:29.829541 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 505/1033] 10.222.0.15 () {56 vars in 1105 bytes} [Mon Aug 22 15:48:29 2022] GET /identity/v3/services/image => generated 86 bytes in 226 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 22 15:48:29.829541 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-872d91ad-d021-4be2-ab39-bfdb2cd94ccd None admin] Authenticating user token {{(pid=81767) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 15:48:29.843041 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=100 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 15:48:29.858527 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c56a8405-4da8-471c-b192-61f75a551c6b None admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 15:48:29.916594 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e522392d-07fc-4f9c-9720-938a881b0492 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 15:48:29.917147 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e522392d-07fc-4f9c-9720-938a881b0492 None admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:48:29.917527 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e522392d-07fc-4f9c-9720-938a881b0492 None admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:48:29.917813 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e522392d-07fc-4f9c-9720-938a881b0492 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:48:29.928483 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e522392d-07fc-4f9c-9720-938a881b0492 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 15:48:29.930036 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e522392d-07fc-4f9c-9720-938a881b0492 None admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 15:48:29.945543 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application [None req-e522392d-07fc-4f9c-9720-938a881b0492 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 22 15:48:29.945543 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 15:48:29.945543 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 15:48:29.945543 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 15:48:29.945543 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 15:48:29.945543 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 15:48:29.945543 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 15:48:29.945543 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 15:48:29.945543 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 15:48:29.945543 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 15:48:29.945543 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 15:48:29.945543 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 15:48:29.945543 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 22 15:48:29.945543 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 22 15:48:29.945543 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 22 15:48:29.945543 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 22 15:48:29.946832 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 15:48:29.946832 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 15:48:29.946832 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 22 15:48:29.946832 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 22 15:48:29.946832 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 22 15:48:29.946832 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 22 15:48:29.946832 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 15:48:29.946832 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 15:48:29.946832 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 15:48:29.946832 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 15:48:29.946832 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 15:48:29.946832 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application with Lock( Aug 22 15:48:29.946832 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 15:48:29.946832 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self._enter() Aug 22 15:48:29.946832 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 15:48:29.946832 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 15:48:29.946832 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 15:48:29.946832 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self.creator() Aug 22 15:48:29.947935 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 15:48:29.947935 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application created_value = creator( Aug 22 15:48:29.947935 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 22 15:48:29.947935 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 22 15:48:29.947935 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 22 15:48:29.947935 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 22 15:48:29.947935 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 22 15:48:29.947935 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 22 15:48:29.947935 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 22 15:48:29.947935 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application Aug 22 15:48:29.947935 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 529/1034] 10.222.0.15 () {56 vars in 1093 bytes} [Mon Aug 22 15:48:29 2022] GET /identity/v3/users/nova => generated 82 bytes in 237 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 22 15:48:29.952605 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e522392d-07fc-4f9c-9720-938a881b0492 None admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 15:48:29.990735 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-872d91ad-d021-4be2-ab39-bfdb2cd94ccd None admin] Validating token access rules against request {{(pid=81767) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 15:48:30.039332 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fda75200-8fff-4d3f-9968-92d95e57d99f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81767) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 15:48:30.039332 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fda75200-8fff-4d3f-9968-92d95e57d99f None admin] REQUEST_METHOD: `GET` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:48:30.039332 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fda75200-8fff-4d3f-9968-92d95e57d99f None admin] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:48:30.039332 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fda75200-8fff-4d3f-9968-92d95e57d99f None admin] PATH_INFO: `/v3/services` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:48:30.040344 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fda75200-8fff-4d3f-9968-92d95e57d99f None admin] RBAC: Adding query filter params (name=image) {{(pid=81767) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 15:48:30.040344 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fda75200-8fff-4d3f-9968-92d95e57d99f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 15:48:30.040344 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fda75200-8fff-4d3f-9968-92d95e57d99f None admin] RBAC: Authorization granted {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 15:48:30.041443 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG dbcounter [-] [81767] Writing DB stats keystone:SELECT=100 {{(pid=81767) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 15:48:30.047812 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 506/1035] 10.222.0.15 () {56 vars in 1108 bytes} [Mon Aug 22 15:48:29 2022] GET /identity/v3/services?name=image => generated 122 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 15:48:30.053353 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fda75200-8fff-4d3f-9968-92d95e57d99f None admin] Authenticating user token {{(pid=81767) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 15:48:30.076994 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e522392d-07fc-4f9c-9720-938a881b0492 None admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 15:48:30.115876 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42ed162c-2de8-4cb5-a12b-319b6be59e80 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 15:48:30.116876 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42ed162c-2de8-4cb5-a12b-319b6be59e80 None admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:48:30.117151 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42ed162c-2de8-4cb5-a12b-319b6be59e80 None admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:48:30.117423 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42ed162c-2de8-4cb5-a12b-319b6be59e80 None admin] PATH_INFO: `/v3/users` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:48:30.118710 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42ed162c-2de8-4cb5-a12b-319b6be59e80 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81768) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 15:48:30.119014 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42ed162c-2de8-4cb5-a12b-319b6be59e80 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 15:48:30.120556 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42ed162c-2de8-4cb5-a12b-319b6be59e80 None admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 15:48:30.140240 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 530/1036] 10.222.0.15 () {56 vars in 1133 bytes} [Mon Aug 22 15:48:29 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 22 15:48:30.145989 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42ed162c-2de8-4cb5-a12b-319b6be59e80 None admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 15:48:30.160043 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=100 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 15:48:30.180599 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fda75200-8fff-4d3f-9968-92d95e57d99f None admin] Validating token access rules against request {{(pid=81767) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 15:48:30.213491 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24ce1919-5be7-4f4c-98a4-8587748bee07 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81767) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 15:48:30.214153 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24ce1919-5be7-4f4c-98a4-8587748bee07 None admin] REQUEST_METHOD: `GET` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:48:30.214354 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24ce1919-5be7-4f4c-98a4-8587748bee07 None admin] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:48:30.214634 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24ce1919-5be7-4f4c-98a4-8587748bee07 None admin] PATH_INFO: `/v3/services` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:48:30.215290 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24ce1919-5be7-4f4c-98a4-8587748bee07 None admin] RBAC: Adding query filter params (type=image) {{(pid=81767) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 15:48:30.215590 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24ce1919-5be7-4f4c-98a4-8587748bee07 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 15:48:30.217090 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24ce1919-5be7-4f4c-98a4-8587748bee07 None admin] RBAC: Authorization granted {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 15:48:30.222628 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 507/1037] 10.222.0.15 () {56 vars in 1108 bytes} [Mon Aug 22 15:48:30 2022] GET /identity/v3/services?type=image => generated 350 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 22 15:48:30.229150 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24ce1919-5be7-4f4c-98a4-8587748bee07 None admin] Authenticating user token {{(pid=81767) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 15:48:30.259105 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42ed162c-2de8-4cb5-a12b-319b6be59e80 None admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 15:48:30.300091 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d967916-3981-4a51-a8a9-ba2762f5a88f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 15:48:30.301309 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d967916-3981-4a51-a8a9-ba2762f5a88f None admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:48:30.301309 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d967916-3981-4a51-a8a9-ba2762f5a88f None admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:48:30.301309 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d967916-3981-4a51-a8a9-ba2762f5a88f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:48:30.301622 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG dbcounter [-] [81767] Writing DB stats keystone:SELECT=100 {{(pid=81767) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 15:48:30.349421 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24ce1919-5be7-4f4c-98a4-8587748bee07 None admin] Validating token access rules against request {{(pid=81767) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 15:48:30.386968 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5e86e45-8076-4b36-a1eb-ec5913aee2d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81767) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 15:48:30.387614 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5e86e45-8076-4b36-a1eb-ec5913aee2d0 None admin] REQUEST_METHOD: `GET` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:48:30.387929 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5e86e45-8076-4b36-a1eb-ec5913aee2d0 None admin] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:48:30.388167 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5e86e45-8076-4b36-a1eb-ec5913aee2d0 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:48:30.388884 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5e86e45-8076-4b36-a1eb-ec5913aee2d0 None admin] RBAC: Adding query filter params (interface=public, service_id=c707a302ec2f4849b238049a6373be0b, region_id=RegionOne) {{(pid=81767) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 15:48:30.389194 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5e86e45-8076-4b36-a1eb-ec5913aee2d0 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 15:48:30.390678 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5e86e45-8076-4b36-a1eb-ec5913aee2d0 None admin] RBAC: Authorization granted {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 15:48:30.401000 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 508/1038] 10.222.0.15 () {56 vars in 1251 bytes} [Mon Aug 22 15:48:30 2022] GET /identity/v3/endpoints?service_id=c707a302ec2f4849b238049a6373be0b&interface=public®ion_id=RegionOne => generated 194 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 22 15:48:30.402540 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5e86e45-8076-4b36-a1eb-ec5913aee2d0 None admin] Authenticating user token {{(pid=81767) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 15:48:30.407504 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d967916-3981-4a51-a8a9-ba2762f5a88f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 15:48:30.409447 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d967916-3981-4a51-a8a9-ba2762f5a88f None admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 15:48:30.434446 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=100 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 15:48:30.560337 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 531/1039] 10.222.0.15 () {58 vars in 1282 bytes} [Mon Aug 22 15:48:30 2022] GET /identity/v3/auth/tokens => generated 1969 bytes in 415 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Aug 22 15:48:30.566902 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d967916-3981-4a51-a8a9-ba2762f5a88f None admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 15:48:30.579414 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5e86e45-8076-4b36-a1eb-ec5913aee2d0 None admin] Validating token access rules against request {{(pid=81767) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 15:48:30.624101 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG dbcounter [-] [81767] Writing DB stats keystone:SELECT=100 {{(pid=81767) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 15:48:30.635725 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5429c541-c866-494f-abbd-e8000bd08397 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81767) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 15:48:30.636540 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5429c541-c866-494f-abbd-e8000bd08397 None admin] REQUEST_METHOD: `GET` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:48:30.637733 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5429c541-c866-494f-abbd-e8000bd08397 None admin] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:48:30.637733 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5429c541-c866-494f-abbd-e8000bd08397 None admin] PATH_INFO: `/v3/services` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:48:30.638409 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5429c541-c866-494f-abbd-e8000bd08397 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 15:48:30.640145 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5429c541-c866-494f-abbd-e8000bd08397 None admin] RBAC: Authorization granted {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 15:48:30.655616 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 509/1040] 10.222.0.15 () {56 vars in 1087 bytes} [Mon Aug 22 15:48:30 2022] GET /identity/v3/services => generated 1957 bytes in 254 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 22 15:48:30.722342 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d967916-3981-4a51-a8a9-ba2762f5a88f None admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 15:48:30.737033 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=100 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 15:48:30.767177 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c832bb2-82dc-4e78-9925-1436d90a1784 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 15:48:30.767971 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c832bb2-82dc-4e78-9925-1436d90a1784 None admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:48:30.768327 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c832bb2-82dc-4e78-9925-1436d90a1784 None admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:48:30.768675 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c832bb2-82dc-4e78-9925-1436d90a1784 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:48:30.782545 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c832bb2-82dc-4e78-9925-1436d90a1784 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 15:48:30.784282 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c832bb2-82dc-4e78-9925-1436d90a1784 None admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 15:48:30.802607 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application [None req-3c832bb2-82dc-4e78-9925-1436d90a1784 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 22 15:48:30.802607 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 15:48:30.802607 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 15:48:30.802607 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 15:48:30.802607 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 15:48:30.802607 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 15:48:30.802607 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 15:48:30.802607 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 15:48:30.802607 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 15:48:30.802607 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 15:48:30.802607 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 15:48:30.802607 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 15:48:30.802607 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 22 15:48:30.802607 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 22 15:48:30.802607 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 22 15:48:30.802607 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 22 15:48:30.804766 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 15:48:30.804766 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 15:48:30.804766 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 15:48:30.804766 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 15:48:30.804766 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 15:48:30.804766 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 15:48:30.804766 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 15:48:30.804766 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application with Lock( Aug 22 15:48:30.804766 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 15:48:30.804766 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self._enter() Aug 22 15:48:30.804766 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 15:48:30.804766 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 15:48:30.804766 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 15:48:30.804766 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self.creator() Aug 22 15:48:30.804766 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 15:48:30.804766 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application created_value = creator( Aug 22 15:48:30.804766 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 22 15:48:30.806970 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 22 15:48:30.806970 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 22 15:48:30.806970 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application Aug 22 15:48:30.806970 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 532/1041] 10.222.0.15 () {56 vars in 1108 bytes} [Mon Aug 22 15:48:30 2022] GET /identity/v3/domains/Default => generated 87 bytes in 239 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 22 15:48:30.813816 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5429c541-c866-494f-abbd-e8000bd08397 None admin] Authenticating user token {{(pid=81767) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 15:48:30.982251 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5429c541-c866-494f-abbd-e8000bd08397 None admin] Validating token access rules against request {{(pid=81767) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 15:48:31.024725 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6953cc77-db05-40d7-805f-ebc387344181 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81767) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 15:48:31.025385 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6953cc77-db05-40d7-805f-ebc387344181 None admin] REQUEST_METHOD: `GET` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:48:31.025597 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6953cc77-db05-40d7-805f-ebc387344181 None admin] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:48:31.025863 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6953cc77-db05-40d7-805f-ebc387344181 None admin] PATH_INFO: `/v3/domains` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:48:31.026509 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6953cc77-db05-40d7-805f-ebc387344181 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81767) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 15:48:31.026810 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6953cc77-db05-40d7-805f-ebc387344181 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 15:48:31.028199 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6953cc77-db05-40d7-805f-ebc387344181 None admin] RBAC: Authorization granted {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 15:48:31.047313 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 510/1042] 10.222.0.15 () {56 vars in 1109 bytes} [Mon Aug 22 15:48:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 15:48:31.051644 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c832bb2-82dc-4e78-9925-1436d90a1784 None admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 15:48:31.130071 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b548f1-8a6e-4633-97aa-4d2a94741f37 None None] REQUEST_METHOD: `GET` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:48:31.131057 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b548f1-8a6e-4633-97aa-4d2a94741f37 None None] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:48:31.131682 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b548f1-8a6e-4633-97aa-4d2a94741f37 None None] PATH_INFO: `/` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:48:31.134253 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 511/1043] 10.222.0.15 () {54 vars in 927 bytes} [Mon Aug 22 15:48:31 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 15:48:31.142130 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-874b7a1c-6e92-4d7e-a0a5-69e3e51e8a6e None None] REQUEST_METHOD: `POST` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:48:31.142771 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-874b7a1c-6e92-4d7e-a0a5-69e3e51e8a6e None None] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:48:31.143772 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-874b7a1c-6e92-4d7e-a0a5-69e3e51e8a6e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:48:31.173485 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.auth.core [None req-874b7a1c-6e92-4d7e-a0a5-69e3e51e8a6e None None] MFA Rules not processed for user `6b2199ae0fb0480aa917ada391f5ec54`. Rule list: `[]` (Enabled: `True`). {{(pid=81767) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 15:48:31.174885 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.fernet_utils [None req-874b7a1c-6e92-4d7e-a0a5-69e3e51e8a6e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81767) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 15:48:31.214129 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c832bb2-82dc-4e78-9925-1436d90a1784 None admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 15:48:31.257953 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-126af32a-c220-4d32-af74-153be3afed52 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 15:48:31.259118 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-126af32a-c220-4d32-af74-153be3afed52 None admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:48:31.259532 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-126af32a-c220-4d32-af74-153be3afed52 None admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:48:31.259894 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-126af32a-c220-4d32-af74-153be3afed52 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:48:31.262170 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG dbcounter [-] [81767] Writing DB stats keystone:SELECT=100 {{(pid=81767) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 15:48:31.270548 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-126af32a-c220-4d32-af74-153be3afed52 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 15:48:31.272225 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-126af32a-c220-4d32-af74-153be3afed52 None admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 15:48:31.288297 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application [None req-126af32a-c220-4d32-af74-153be3afed52 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 22 15:48:31.288297 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 15:48:31.288297 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 15:48:31.288297 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 15:48:31.288297 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 15:48:31.288297 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 15:48:31.288297 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 15:48:31.288297 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 15:48:31.288297 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 15:48:31.288297 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 15:48:31.288297 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 15:48:31.288297 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 15:48:31.288297 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 22 15:48:31.288297 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 22 15:48:31.288297 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 22 15:48:31.289271 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 22 15:48:31.289271 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 15:48:31.289271 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 15:48:31.289271 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 15:48:31.289271 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 15:48:31.289271 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 15:48:31.289271 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 15:48:31.289271 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 15:48:31.289271 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application with Lock( Aug 22 15:48:31.289271 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 15:48:31.289271 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self._enter() Aug 22 15:48:31.289271 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 15:48:31.289271 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 15:48:31.289271 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 15:48:31.289271 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self.creator() Aug 22 15:48:31.289271 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 15:48:31.289271 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application created_value = creator( Aug 22 15:48:31.290665 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 22 15:48:31.290665 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 22 15:48:31.290665 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 22 15:48:31.290665 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 22 15:48:31.290665 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 22 15:48:31.290665 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 22 15:48:31.290665 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 22 15:48:31.290665 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application Aug 22 15:48:31.290665 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 533/1044] 10.222.0.15 () {56 vars in 1111 bytes} [Mon Aug 22 15:48:31 2022] GET /identity/v3/projects/service => generated 88 bytes in 239 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 15:48:31.296260 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-126af32a-c220-4d32-af74-153be3afed52 None admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 15:48:31.302875 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 512/1045] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Aug 22 15:48:31 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 22 15:48:31.340651 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=100 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 15:48:31.473023 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-126af32a-c220-4d32-af74-153be3afed52 None admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 15:48:31.532386 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc0e20d0-1c19-453f-b4bc-29c91d3a6906 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 15:48:31.533300 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc0e20d0-1c19-453f-b4bc-29c91d3a6906 None admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:48:31.533763 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc0e20d0-1c19-453f-b4bc-29c91d3a6906 None admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:48:31.534229 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc0e20d0-1c19-453f-b4bc-29c91d3a6906 None admin] PATH_INFO: `/v3/projects` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:48:31.535529 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc0e20d0-1c19-453f-b4bc-29c91d3a6906 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81768) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 15:48:31.535967 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc0e20d0-1c19-453f-b4bc-29c91d3a6906 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 15:48:31.538101 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc0e20d0-1c19-453f-b4bc-29c91d3a6906 None admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 15:48:31.559319 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 534/1046] 10.222.0.15 () {56 vars in 1148 bytes} [Mon Aug 22 15:48:31 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 259 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 22 15:48:31.565973 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-874b7a1c-6e92-4d7e-a0a5-69e3e51e8a6e None None] Authenticating user token {{(pid=81767) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 15:48:31.601370 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc0e20d0-1c19-453f-b4bc-29c91d3a6906 None admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 15:48:31.710324 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-874b7a1c-6e92-4d7e-a0a5-69e3e51e8a6e None None] Validating token access rules against request {{(pid=81767) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 15:48:31.712281 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=100 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 15:48:31.737526 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc0e20d0-1c19-453f-b4bc-29c91d3a6906 None admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 15:48:31.758566 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d2743bc-e18f-446b-9845-ccd6010c5465 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81767) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 15:48:31.759196 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d2743bc-e18f-446b-9845-ccd6010c5465 None admin] REQUEST_METHOD: `GET` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:48:31.759526 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d2743bc-e18f-446b-9845-ccd6010c5465 None admin] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:48:31.759800 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d2743bc-e18f-446b-9845-ccd6010c5465 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:48:31.760605 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d2743bc-e18f-446b-9845-ccd6010c5465 None admin] RBAC: Adding query filter params (role.id=806eb8803ece4a1fa1bc2487b4cd7c02, scope.project.id=c4ee0413f9ef4d4fb1f95a660babe090, user.id=e4ef569e41fb46d0b721bb1c7663616f) {{(pid=81767) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 15:48:31.760934 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d2743bc-e18f-446b-9845-ccd6010c5465 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 15:48:31.762384 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d2743bc-e18f-446b-9845-ccd6010c5465 None admin] RBAC: Authorization granted {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 15:48:31.779593 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 513/1047] 10.222.0.15 () {56 vars in 1374 bytes} [Mon Aug 22 15:48:31 2022] GET /identity/v3/role_assignments?user.id=e4ef569e41fb46d0b721bb1c7663616f&scope.project.id=c4ee0413f9ef4d4fb1f95a660babe090&role.id=806eb8803ece4a1fa1bc2487b4cd7c02 => generated 259 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 15:48:31.787749 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb58a831-a467-4957-a703-02e79de5dae1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 15:48:31.788829 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb58a831-a467-4957-a703-02e79de5dae1 None admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:48:31.789299 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb58a831-a467-4957-a703-02e79de5dae1 None admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:48:31.790308 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb58a831-a467-4957-a703-02e79de5dae1 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:48:31.804863 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb58a831-a467-4957-a703-02e79de5dae1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 22 15:48:31.811722 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application [None req-cb58a831-a467-4957-a703-02e79de5dae1 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 22 15:48:31.811722 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 15:48:31.811722 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 15:48:31.811722 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 15:48:31.811722 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 15:48:31.811722 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 15:48:31.811722 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 15:48:31.811722 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 15:48:31.811722 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 15:48:31.811722 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 15:48:31.811722 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 15:48:31.811722 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 15:48:31.811722 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 22 15:48:31.811722 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 22 15:48:31.811722 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 22 15:48:31.811722 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 22 15:48:31.812817 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 22 15:48:31.812817 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 22 15:48:31.812817 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 22 15:48:31.812817 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 22 15:48:31.812817 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 15:48:31.812817 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 15:48:31.812817 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 15:48:31.812817 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 15:48:31.812817 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 15:48:31.812817 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 15:48:31.812817 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 15:48:31.812817 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application with Lock( Aug 22 15:48:31.812817 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 15:48:31.812817 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self._enter() Aug 22 15:48:31.812817 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 15:48:31.812817 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 15:48:31.813733 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 15:48:31.813733 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self.creator() Aug 22 15:48:31.813733 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 15:48:31.813733 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application created_value = creator( Aug 22 15:48:31.813733 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 22 15:48:31.813733 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 22 15:48:31.813733 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 22 15:48:31.813733 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 22 15:48:31.813733 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 22 15:48:31.813733 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 22 15:48:31.813733 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 22 15:48:31.813733 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application Aug 22 15:48:31.816337 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 535/1048] 10.222.0.15 () {56 vars in 1096 bytes} [Mon Aug 22 15:48:31 2022] GET /identity/v3/roles/admin => generated 83 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 22 15:48:31.823604 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d2743bc-e18f-446b-9845-ccd6010c5465 None admin] Authenticating user token {{(pid=81767) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 15:48:31.888162 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG dbcounter [-] [81767] Writing DB stats keystone:SELECT=100 {{(pid=81767) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 15:48:31.977293 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d2743bc-e18f-446b-9845-ccd6010c5465 None admin] Validating token access rules against request {{(pid=81767) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 15:48:32.046884 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea3080a8-9e13-424d-97a4-3fdcb1eee4fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81767) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 15:48:32.048241 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea3080a8-9e13-424d-97a4-3fdcb1eee4fe None admin] REQUEST_METHOD: `GET` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:48:32.049000 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea3080a8-9e13-424d-97a4-3fdcb1eee4fe None admin] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:48:32.049681 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea3080a8-9e13-424d-97a4-3fdcb1eee4fe None admin] PATH_INFO: `/v3/roles` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:48:32.051290 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea3080a8-9e13-424d-97a4-3fdcb1eee4fe None admin] RBAC: Adding query filter params (name=admin) {{(pid=81767) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 15:48:32.052269 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea3080a8-9e13-424d-97a4-3fdcb1eee4fe None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 15:48:32.055021 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea3080a8-9e13-424d-97a4-3fdcb1eee4fe None admin] RBAC: Authorization granted {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 15:48:32.078986 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 514/1049] 10.222.0.15 () {56 vars in 1099 bytes} [Mon Aug 22 15:48:31 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 251 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 22 15:48:32.081215 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb58a831-a467-4957-a703-02e79de5dae1 None admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 15:48:32.238592 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb58a831-a467-4957-a703-02e79de5dae1 None admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 15:48:32.275836 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b7723a5-441d-4a74-acb4-d5d9b35c5d00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 15:48:32.276631 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b7723a5-441d-4a74-acb4-d5d9b35c5d00 None admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:48:32.276904 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b7723a5-441d-4a74-acb4-d5d9b35c5d00 None admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:48:32.277165 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b7723a5-441d-4a74-acb4-d5d9b35c5d00 None admin] PATH_INFO: `/v3/users/45eb7849209b48c6bbcf379cab77e02a` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:48:32.299111 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=100 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 15:48:32.302374 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b7723a5-441d-4a74-acb4-d5d9b35c5d00 None admin] RBAC: Authorizing `identity:get_user(user_id=45eb7849209b48c6bbcf379cab77e02a)` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 15:48:32.303892 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b7723a5-441d-4a74-acb4-d5d9b35c5d00 None admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 15:48:32.306006 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 536/1050] 10.222.0.15 () {56 vars in 1177 bytes} [Mon Aug 22 15:48:32 2022] GET /identity/v3/users/45eb7849209b48c6bbcf379cab77e02a => generated 285 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 15:48:32.312121 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea3080a8-9e13-424d-97a4-3fdcb1eee4fe None admin] Authenticating user token {{(pid=81767) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 15:48:32.431676 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea3080a8-9e13-424d-97a4-3fdcb1eee4fe None admin] Validating token access rules against request {{(pid=81767) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 15:48:32.440871 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG dbcounter [-] [81767] Writing DB stats keystone:SELECT=100 {{(pid=81767) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 15:48:32.469038 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8f31298-9747-44ac-82ac-3708b779ac3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81767) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 15:48:32.469652 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8f31298-9747-44ac-82ac-3708b779ac3f None admin] REQUEST_METHOD: `GET` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:48:32.469928 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8f31298-9747-44ac-82ac-3708b779ac3f None admin] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:48:32.470197 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8f31298-9747-44ac-82ac-3708b779ac3f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:48:32.548273 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8f31298-9747-44ac-82ac-3708b779ac3f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 15:48:32.549773 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8f31298-9747-44ac-82ac-3708b779ac3f None admin] RBAC: Authorization granted {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 15:48:32.659472 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 515/1051] 10.222.0.15 () {58 vars in 1282 bytes} [Mon Aug 22 15:48:32 2022] GET /identity/v3/auth/tokens => generated 1969 bytes in 343 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 22 15:48:32.661479 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b7723a5-441d-4a74-acb4-d5d9b35c5d00 None admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 15:48:32.846027 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b7723a5-441d-4a74-acb4-d5d9b35c5d00 None admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 15:48:32.910708 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf3923d1-b230-4c73-a8ca-323353e947d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 15:48:32.911869 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf3923d1-b230-4c73-a8ca-323353e947d1 None admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:48:32.912376 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf3923d1-b230-4c73-a8ca-323353e947d1 None admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:48:32.912874 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf3923d1-b230-4c73-a8ca-323353e947d1 None admin] PATH_INFO: `/v3/projects/2b65932e1345451c880e9d46bd66f7f3` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:48:32.915555 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf3923d1-b230-4c73-a8ca-323353e947d1 None admin] RBAC: Authorizing `identity:get_project(project_id=2b65932e1345451c880e9d46bd66f7f3)` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 15:48:32.918322 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf3923d1-b230-4c73-a8ca-323353e947d1 None admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 15:48:32.921568 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 537/1052] 10.222.0.15 () {56 vars in 1186 bytes} [Mon Aug 22 15:48:32 2022] GET /identity/v3/projects/2b65932e1345451c880e9d46bd66f7f3 => generated 311 bytes in 261 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 22 15:48:32.929259 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8f31298-9747-44ac-82ac-3708b779ac3f None admin] Authenticating user token {{(pid=81767) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 15:48:32.941649 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG dbcounter [-] [81767] Writing DB stats keystone:SELECT=100 {{(pid=81767) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 15:48:33.067961 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8f31298-9747-44ac-82ac-3708b779ac3f None admin] Validating token access rules against request {{(pid=81767) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 15:48:33.127412 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60df1a33-d2e8-4732-ac2b-6ef83883cc09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81767) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 15:48:33.128253 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60df1a33-d2e8-4732-ac2b-6ef83883cc09 None admin] REQUEST_METHOD: `GET` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:48:33.128718 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60df1a33-d2e8-4732-ac2b-6ef83883cc09 None admin] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:48:33.129161 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60df1a33-d2e8-4732-ac2b-6ef83883cc09 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:48:33.130419 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60df1a33-d2e8-4732-ac2b-6ef83883cc09 None admin] RBAC: Adding query filter params (role.id=9d9848cfd22e4b089db443c350e56670, scope.project.id=2b65932e1345451c880e9d46bd66f7f3, user.id=45eb7849209b48c6bbcf379cab77e02a) {{(pid=81767) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 15:48:33.130948 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60df1a33-d2e8-4732-ac2b-6ef83883cc09 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 15:48:33.133634 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60df1a33-d2e8-4732-ac2b-6ef83883cc09 None admin] RBAC: Authorization granted {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 15:48:33.151634 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 516/1053] 10.222.0.15 () {56 vars in 1374 bytes} [Mon Aug 22 15:48:32 2022] GET /identity/v3/role_assignments?user.id=45eb7849209b48c6bbcf379cab77e02a&scope.project.id=2b65932e1345451c880e9d46bd66f7f3&role.id=9d9848cfd22e4b089db443c350e56670 => generated 259 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 22 15:48:33.886514 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a3c7e2-f5b4-4c15-8985-0dc8cc513ebd None None] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:48:33.887357 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a3c7e2-f5b4-4c15-8985-0dc8cc513ebd None None] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:48:33.887357 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a3c7e2-f5b4-4c15-8985-0dc8cc513ebd None None] PATH_INFO: `/` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:48:33.888694 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 538/1054] 10.222.0.15 () {54 vars in 927 bytes} [Mon Aug 22 15:48:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 22 15:48:33.893998 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca01e7ef-eb6b-4498-88ac-5e69d36a830d None None] REQUEST_METHOD: `POST` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:48:33.894350 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca01e7ef-eb6b-4498-88ac-5e69d36a830d None None] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:48:33.894593 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca01e7ef-eb6b-4498-88ac-5e69d36a830d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:48:33.918795 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ee185a-f002-4e03-9db2-456f7ba31b7d None None] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:48:33.919058 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ee185a-f002-4e03-9db2-456f7ba31b7d None None] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:48:33.919364 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ee185a-f002-4e03-9db2-456f7ba31b7d None None] PATH_INFO: `/` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:48:33.920754 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 539/1055] 10.222.0.15 () {54 vars in 927 bytes} [Mon Aug 22 15:48:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 22 15:48:33.922504 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.auth.core [None req-ca01e7ef-eb6b-4498-88ac-5e69d36a830d None None] MFA Rules not processed for user `6b2199ae0fb0480aa917ada391f5ec54`. Rule list: `[]` (Enabled: `True`). {{(pid=81767) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 15:48:33.924704 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.fernet_utils [None req-ca01e7ef-eb6b-4498-88ac-5e69d36a830d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81767) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 15:48:33.929747 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13eba676-45b0-4813-9c6a-e128bcf0cb6f None None] REQUEST_METHOD: `POST` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:48:33.930345 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13eba676-45b0-4813-9c6a-e128bcf0cb6f None None] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:48:33.930818 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13eba676-45b0-4813-9c6a-e128bcf0cb6f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:48:33.961087 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.auth.core [None req-13eba676-45b0-4813-9c6a-e128bcf0cb6f None None] MFA Rules not processed for user `6b2199ae0fb0480aa917ada391f5ec54`. Rule list: `[]` (Enabled: `True`). {{(pid=81768) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 15:48:33.963682 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-13eba676-45b0-4813-9c6a-e128bcf0cb6f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 15:48:34.033811 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG dbcounter [-] [81767] Writing DB stats keystone:SELECT=100 {{(pid=81767) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 15:48:34.039892 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 517/1056] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Aug 22 15:48:33 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 22 15:48:34.073611 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=100 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 15:48:34.080510 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 540/1057] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Aug 22 15:48:33 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 22 15:48:34.306174 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca01e7ef-eb6b-4498-88ac-5e69d36a830d None None] Authenticating user token {{(pid=81767) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 15:48:34.362715 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13eba676-45b0-4813-9c6a-e128bcf0cb6f None None] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 15:48:34.479307 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca01e7ef-eb6b-4498-88ac-5e69d36a830d None None] Validating token access rules against request {{(pid=81767) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 15:48:34.482650 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13eba676-45b0-4813-9c6a-e128bcf0cb6f None None] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 15:48:34.529827 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6344c56b-5463-4adf-89ff-9f120d474efe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81767) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 15:48:34.530458 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6344c56b-5463-4adf-89ff-9f120d474efe None admin] REQUEST_METHOD: `GET` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:48:34.530739 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6344c56b-5463-4adf-89ff-9f120d474efe None admin] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:48:34.531005 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6344c56b-5463-4adf-89ff-9f120d474efe None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:48:34.531298 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b19a387-1ddd-42af-be9a-946e0f7c5e4b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 15:48:34.532348 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b19a387-1ddd-42af-be9a-946e0f7c5e4b None admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:48:34.532827 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b19a387-1ddd-42af-be9a-946e0f7c5e4b None admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:48:34.533354 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b19a387-1ddd-42af-be9a-946e0f7c5e4b None admin] PATH_INFO: `/v3/services/image` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:48:34.534362 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b19a387-1ddd-42af-be9a-946e0f7c5e4b None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 15:48:34.536867 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b19a387-1ddd-42af-be9a-946e0f7c5e4b None admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 15:48:34.543766 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6344c56b-5463-4adf-89ff-9f120d474efe None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 22 15:48:34.548509 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application [None req-6344c56b-5463-4adf-89ff-9f120d474efe None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 22 15:48:34.548509 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 15:48:34.548509 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 15:48:34.548509 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 15:48:34.548509 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 15:48:34.548509 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 15:48:34.548509 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 15:48:34.548509 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 15:48:34.548509 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 15:48:34.548509 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 15:48:34.548509 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 15:48:34.548509 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 15:48:34.548509 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 22 15:48:34.548509 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 22 15:48:34.548509 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 22 15:48:34.548509 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 22 15:48:34.549504 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 22 15:48:34.549504 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 22 15:48:34.549504 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 22 15:48:34.549504 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 22 15:48:34.549504 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 15:48:34.549504 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 15:48:34.549504 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 15:48:34.549504 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 15:48:34.549504 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 15:48:34.549504 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 15:48:34.549504 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 15:48:34.549504 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application with Lock( Aug 22 15:48:34.549504 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 15:48:34.549504 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return self._enter() Aug 22 15:48:34.549504 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 15:48:34.549504 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 15:48:34.550335 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 15:48:34.550335 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return self.creator() Aug 22 15:48:34.550335 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 15:48:34.550335 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application created_value = creator( Aug 22 15:48:34.550335 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 22 15:48:34.550335 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 22 15:48:34.550335 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 22 15:48:34.550335 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 22 15:48:34.550335 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 22 15:48:34.550335 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 22 15:48:34.550335 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 22 15:48:34.550335 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application Aug 22 15:48:34.554291 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application [None req-2b19a387-1ddd-42af-be9a-946e0f7c5e4b None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Aug 22 15:48:34.554291 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 15:48:34.554291 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 22 15:48:34.554291 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 22 15:48:34.554291 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 22 15:48:34.554291 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 22 15:48:34.554291 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 22 15:48:34.554291 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 22 15:48:34.554291 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 22 15:48:34.554291 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application Aug 22 15:48:34.554291 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 22 15:48:34.554291 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application Aug 22 15:48:34.554291 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 15:48:34.554291 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 15:48:34.554291 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 15:48:34.554291 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 15:48:34.554291 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 15:48:34.555783 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 15:48:34.555783 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 15:48:34.555783 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 15:48:34.555783 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 15:48:34.555783 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 15:48:34.555783 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 15:48:34.555783 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 22 15:48:34.555783 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 22 15:48:34.555783 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 22 15:48:34.555783 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 22 15:48:34.555783 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 15:48:34.555783 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 15:48:34.555783 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 15:48:34.555783 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 15:48:34.555783 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 15:48:34.555783 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 15:48:34.560224 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 15:48:34.560224 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application with Lock( Aug 22 15:48:34.560224 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 15:48:34.560224 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self._enter() Aug 22 15:48:34.560224 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 15:48:34.560224 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 15:48:34.560224 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 15:48:34.560224 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self.creator() Aug 22 15:48:34.560224 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 15:48:34.560224 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application created_value = creator( Aug 22 15:48:34.560224 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 22 15:48:34.560224 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 22 15:48:34.560224 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 22 15:48:34.560224 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application Aug 22 15:48:34.561976 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 518/1058] 10.222.0.15 () {56 vars in 1120 bytes} [Mon Aug 22 15:48:34 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 250 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 22 15:48:34.561976 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6344c56b-5463-4adf-89ff-9f120d474efe None admin] Authenticating user token {{(pid=81767) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 15:48:34.562241 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 541/1059] 10.222.0.15 () {56 vars in 1105 bytes} [Mon Aug 22 15:48:34 2022] GET /identity/v3/services/image => generated 86 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 15:48:34.564406 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b19a387-1ddd-42af-be9a-946e0f7c5e4b None admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 15:48:34.632127 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG dbcounter [-] [81767] Writing DB stats keystone:SELECT=100 {{(pid=81767) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 15:48:34.667160 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=100 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 15:48:34.683473 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6344c56b-5463-4adf-89ff-9f120d474efe None admin] Validating token access rules against request {{(pid=81767) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 15:48:34.720616 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b19a387-1ddd-42af-be9a-946e0f7c5e4b None admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 15:48:34.723972 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e3c201b-a0c3-4118-a869-edae70634538 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81767) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 15:48:34.724645 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e3c201b-a0c3-4118-a869-edae70634538 None admin] REQUEST_METHOD: `GET` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:48:34.724931 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e3c201b-a0c3-4118-a869-edae70634538 None admin] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:48:34.725195 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e3c201b-a0c3-4118-a869-edae70634538 None admin] PATH_INFO: `/v3/roles` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:48:34.725870 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e3c201b-a0c3-4118-a869-edae70634538 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81767) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 15:48:34.726167 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e3c201b-a0c3-4118-a869-edae70634538 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 15:48:34.727663 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e3c201b-a0c3-4118-a869-edae70634538 None admin] RBAC: Authorization granted {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 15:48:34.741394 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 519/1060] 10.222.0.15 () {56 vars in 1115 bytes} [Mon Aug 22 15:48:34 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 22 15:48:34.743374 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e3c201b-a0c3-4118-a869-edae70634538 None admin] Authenticating user token {{(pid=81767) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 15:48:34.781099 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7b4a943-7c48-47c4-916c-5aabe61f9c42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 15:48:34.782095 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7b4a943-7c48-47c4-916c-5aabe61f9c42 None admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:48:34.782516 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7b4a943-7c48-47c4-916c-5aabe61f9c42 None admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:48:34.782951 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7b4a943-7c48-47c4-916c-5aabe61f9c42 None admin] PATH_INFO: `/v3/services` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:48:34.784009 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7b4a943-7c48-47c4-916c-5aabe61f9c42 None admin] RBAC: Adding query filter params (name=image) {{(pid=81768) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 15:48:34.784549 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7b4a943-7c48-47c4-916c-5aabe61f9c42 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 15:48:34.786740 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7b4a943-7c48-47c4-916c-5aabe61f9c42 None admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 15:48:34.799677 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 542/1061] 10.222.0.15 () {56 vars in 1108 bytes} [Mon Aug 22 15:48:34 2022] GET /identity/v3/services?name=image => generated 122 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 15:48:34.801363 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7b4a943-7c48-47c4-916c-5aabe61f9c42 None admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 15:48:34.888533 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e3c201b-a0c3-4118-a869-edae70634538 None admin] Validating token access rules against request {{(pid=81767) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 15:48:34.917900 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG dbcounter [-] [81767] Writing DB stats keystone:SELECT=100 {{(pid=81767) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 15:48:34.930170 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c1665af-9c48-4b78-811b-3e659151fbd4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81767) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 15:48:34.930836 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c1665af-9c48-4b78-811b-3e659151fbd4 None admin] REQUEST_METHOD: `GET` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:48:34.939479 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c1665af-9c48-4b78-811b-3e659151fbd4 None admin] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:48:34.939479 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c1665af-9c48-4b78-811b-3e659151fbd4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:48:34.939683 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7b4a943-7c48-47c4-916c-5aabe61f9c42 None admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 15:48:34.944435 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c1665af-9c48-4b78-811b-3e659151fbd4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 15:48:34.945937 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c1665af-9c48-4b78-811b-3e659151fbd4 None admin] RBAC: Authorization granted {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 15:48:34.964513 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application [None req-4c1665af-9c48-4b78-811b-3e659151fbd4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 22 15:48:34.964513 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 15:48:34.964513 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 15:48:34.964513 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 15:48:34.964513 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 15:48:34.964513 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 15:48:34.964513 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 15:48:34.964513 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 15:48:34.964513 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 15:48:34.964513 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 15:48:34.964513 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 15:48:34.964513 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 15:48:34.964513 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 22 15:48:34.964513 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 22 15:48:34.964513 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 22 15:48:34.964513 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 22 15:48:34.966422 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 15:48:34.966422 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 15:48:34.966422 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 15:48:34.966422 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 15:48:34.966422 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 15:48:34.966422 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 15:48:34.966422 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 15:48:34.966422 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application with Lock( Aug 22 15:48:34.966422 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 15:48:34.966422 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return self._enter() Aug 22 15:48:34.966422 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 15:48:34.966422 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 15:48:34.966422 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 15:48:34.966422 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return self.creator() Aug 22 15:48:34.966422 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 15:48:34.966422 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application created_value = creator( Aug 22 15:48:34.966422 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 22 15:48:34.968193 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 22 15:48:34.968193 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 22 15:48:34.968193 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application Aug 22 15:48:34.968193 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 520/1062] 10.222.0.15 () {56 vars in 1108 bytes} [Mon Aug 22 15:48:34 2022] GET /identity/v3/domains/Default => generated 87 bytes in 225 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 15:48:34.972688 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c1665af-9c48-4b78-811b-3e659151fbd4 None admin] Authenticating user token {{(pid=81767) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 15:48:34.980777 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=100 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 15:48:34.996496 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5146b3b2-b8d0-47a6-b965-b6d533ccf41f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 15:48:34.997372 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5146b3b2-b8d0-47a6-b965-b6d533ccf41f None admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:48:34.997824 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5146b3b2-b8d0-47a6-b965-b6d533ccf41f None admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:48:34.998262 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5146b3b2-b8d0-47a6-b965-b6d533ccf41f None admin] PATH_INFO: `/v3/services` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:48:34.999090 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5146b3b2-b8d0-47a6-b965-b6d533ccf41f None admin] RBAC: Adding query filter params (type=image) {{(pid=81768) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 15:48:34.999591 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5146b3b2-b8d0-47a6-b965-b6d533ccf41f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 15:48:35.001907 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5146b3b2-b8d0-47a6-b965-b6d533ccf41f None admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 15:48:35.009648 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 543/1063] 10.222.0.15 () {56 vars in 1108 bytes} [Mon Aug 22 15:48:34 2022] GET /identity/v3/services?type=image => generated 350 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 15:48:35.017706 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5146b3b2-b8d0-47a6-b965-b6d533ccf41f None admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 15:48:35.146689 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c1665af-9c48-4b78-811b-3e659151fbd4 None admin] Validating token access rules against request {{(pid=81767) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 15:48:35.202569 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5146b3b2-b8d0-47a6-b965-b6d533ccf41f None admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 15:48:35.211376 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e2c1daa-bf4d-4176-8fb0-a70c4039e240 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81767) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 15:48:35.212554 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e2c1daa-bf4d-4176-8fb0-a70c4039e240 None admin] REQUEST_METHOD: `GET` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:48:35.213016 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e2c1daa-bf4d-4176-8fb0-a70c4039e240 None admin] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:48:35.213465 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e2c1daa-bf4d-4176-8fb0-a70c4039e240 None admin] PATH_INFO: `/v3/domains` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:48:35.214639 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e2c1daa-bf4d-4176-8fb0-a70c4039e240 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81767) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 15:48:35.215134 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e2c1daa-bf4d-4176-8fb0-a70c4039e240 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 15:48:35.217342 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e2c1daa-bf4d-4176-8fb0-a70c4039e240 None admin] RBAC: Authorization granted {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 15:48:35.244503 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 521/1064] 10.222.0.15 () {56 vars in 1109 bytes} [Mon Aug 22 15:48:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 267 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 22 15:48:35.245925 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e2c1daa-bf4d-4176-8fb0-a70c4039e240 None admin] Authenticating user token {{(pid=81767) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 15:48:35.270575 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3ba6c21-99c8-4252-8bfd-3ab2383ff17b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 15:48:35.271633 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ba6c21-99c8-4252-8bfd-3ab2383ff17b None admin] REQUEST_METHOD: `POST` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:48:35.272151 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ba6c21-99c8-4252-8bfd-3ab2383ff17b None admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:48:35.272670 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ba6c21-99c8-4252-8bfd-3ab2383ff17b None admin] PATH_INFO: `/v3/endpoints` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:48:35.273598 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3ba6c21-99c8-4252-8bfd-3ab2383ff17b None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 15:48:35.276243 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3ba6c21-99c8-4252-8bfd-3ab2383ff17b None admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 15:48:35.304097 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG dbcounter [-] [81767] Writing DB stats keystone:SELECT=100 {{(pid=81767) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 15:48:35.372656 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 544/1065] 10.222.0.15 () {60 vars in 1144 bytes} [Mon Aug 22 15:48:35 2022] POST /identity/v3/endpoints => generated 327 bytes in 356 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 22 15:48:35.393288 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e2c1daa-bf4d-4176-8fb0-a70c4039e240 None admin] Validating token access rules against request {{(pid=81767) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 15:48:35.453522 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d93172f2-d0a2-41f3-9ce1-74bbb6f6cf36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81767) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 15:48:35.454546 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d93172f2-d0a2-41f3-9ce1-74bbb6f6cf36 None admin] REQUEST_METHOD: `GET` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:48:35.455072 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d93172f2-d0a2-41f3-9ce1-74bbb6f6cf36 None admin] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:48:35.455665 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d93172f2-d0a2-41f3-9ce1-74bbb6f6cf36 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:48:35.466227 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d93172f2-d0a2-41f3-9ce1-74bbb6f6cf36 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 15:48:35.468084 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d93172f2-d0a2-41f3-9ce1-74bbb6f6cf36 None admin] RBAC: Authorization granted {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 15:48:35.483584 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application [None req-d93172f2-d0a2-41f3-9ce1-74bbb6f6cf36 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 22 15:48:35.483584 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 15:48:35.483584 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 15:48:35.483584 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 15:48:35.483584 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 15:48:35.483584 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 15:48:35.483584 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 15:48:35.483584 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 15:48:35.483584 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 15:48:35.483584 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 15:48:35.483584 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 15:48:35.483584 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 15:48:35.483584 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 22 15:48:35.483584 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 22 15:48:35.483584 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 22 15:48:35.483584 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 22 15:48:35.485800 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 15:48:35.485800 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 15:48:35.485800 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 22 15:48:35.485800 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 22 15:48:35.485800 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 22 15:48:35.485800 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 22 15:48:35.485800 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 15:48:35.485800 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 15:48:35.485800 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 15:48:35.485800 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 15:48:35.485800 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 15:48:35.485800 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application with Lock( Aug 22 15:48:35.485800 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 15:48:35.485800 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return self._enter() Aug 22 15:48:35.485800 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 15:48:35.485800 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 15:48:35.485800 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 15:48:35.485800 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return self.creator() Aug 22 15:48:35.488265 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 15:48:35.488265 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application created_value = creator( Aug 22 15:48:35.488265 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 22 15:48:35.488265 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 22 15:48:35.488265 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 22 15:48:35.488265 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 22 15:48:35.488265 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 22 15:48:35.488265 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 22 15:48:35.488265 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 22 15:48:35.488265 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application Aug 22 15:48:35.490072 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 522/1066] 10.222.0.15 () {56 vars in 1093 bytes} [Mon Aug 22 15:48:35 2022] GET /identity/v3/users/nova => generated 82 bytes in 245 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 15:48:35.496998 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3ba6c21-99c8-4252-8bfd-3ab2383ff17b None admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 15:48:35.562779 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 15:48:35.634070 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3ba6c21-99c8-4252-8bfd-3ab2383ff17b None admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 15:48:35.680938 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73cfdf36-c126-48e1-91e7-61e06e778508 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 15:48:35.681553 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73cfdf36-c126-48e1-91e7-61e06e778508 None admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:48:35.681837 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73cfdf36-c126-48e1-91e7-61e06e778508 None admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:48:35.682106 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73cfdf36-c126-48e1-91e7-61e06e778508 None admin] PATH_INFO: `/v3/users` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:48:35.683082 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73cfdf36-c126-48e1-91e7-61e06e778508 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81768) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 15:48:35.683417 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73cfdf36-c126-48e1-91e7-61e06e778508 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 15:48:35.684853 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73cfdf36-c126-48e1-91e7-61e06e778508 None admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 15:48:35.705577 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 545/1067] 10.222.0.15 () {56 vars in 1133 bytes} [Mon Aug 22 15:48:35 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 22 15:48:35.712288 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d93172f2-d0a2-41f3-9ce1-74bbb6f6cf36 None admin] Authenticating user token {{(pid=81767) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 15:48:35.876728 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG dbcounter [-] [81767] Writing DB stats keystone:SELECT=100 {{(pid=81767) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 15:48:35.894421 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d93172f2-d0a2-41f3-9ce1-74bbb6f6cf36 None admin] Validating token access rules against request {{(pid=81767) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 15:48:35.954622 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15f9f2ee-f7ea-4373-a19e-dbef8817aa90 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81767) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 15:48:35.956158 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f9f2ee-f7ea-4373-a19e-dbef8817aa90 None admin] REQUEST_METHOD: `GET` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:48:35.956881 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f9f2ee-f7ea-4373-a19e-dbef8817aa90 None admin] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:48:35.957542 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f9f2ee-f7ea-4373-a19e-dbef8817aa90 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:48:35.974428 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15f9f2ee-f7ea-4373-a19e-dbef8817aa90 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 15:48:35.976485 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15f9f2ee-f7ea-4373-a19e-dbef8817aa90 None admin] RBAC: Authorization granted {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 15:48:35.989386 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75590030-a34a-4847-936f-98f3b198744f None None] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:48:35.990255 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75590030-a34a-4847-936f-98f3b198744f None None] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:48:35.990729 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75590030-a34a-4847-936f-98f3b198744f None None] PATH_INFO: `/` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:48:35.993918 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 546/1068] 10.222.0.15 () {54 vars in 927 bytes} [Mon Aug 22 15:48:35 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 22 15:48:35.995717 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application [None req-15f9f2ee-f7ea-4373-a19e-dbef8817aa90 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 22 15:48:35.995717 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 15:48:35.995717 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 15:48:35.995717 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 15:48:35.995717 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 15:48:35.995717 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 15:48:35.995717 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 15:48:35.995717 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 15:48:35.995717 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 15:48:35.995717 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 15:48:35.995717 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 15:48:35.995717 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 15:48:35.995717 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 22 15:48:35.995717 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 22 15:48:35.995717 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 22 15:48:35.995717 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 22 15:48:35.997519 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 15:48:35.997519 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 15:48:35.997519 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 15:48:35.997519 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 15:48:35.997519 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 15:48:35.997519 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 15:48:35.997519 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 15:48:35.997519 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application with Lock( Aug 22 15:48:35.997519 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 15:48:35.997519 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return self._enter() Aug 22 15:48:35.997519 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 15:48:35.997519 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 15:48:35.997519 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 15:48:35.997519 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return self.creator() Aug 22 15:48:35.997519 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 15:48:35.997519 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application created_value = creator( Aug 22 15:48:35.997519 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 22 15:48:35.999321 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 22 15:48:35.999321 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 22 15:48:35.999321 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application Aug 22 15:48:36.002875 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 523/1069] 10.222.0.15 () {56 vars in 1108 bytes} [Mon Aug 22 15:48:35 2022] GET /identity/v3/domains/Default => generated 87 bytes in 292 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 15:48:36.003619 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-832f0dad-30fc-40db-9b80-16d13f4717e6 None None] REQUEST_METHOD: `POST` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:48:36.004019 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15f9f2ee-f7ea-4373-a19e-dbef8817aa90 None admin] Authenticating user token {{(pid=81767) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 15:48:36.004470 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-832f0dad-30fc-40db-9b80-16d13f4717e6 None None] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:48:36.004818 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-832f0dad-30fc-40db-9b80-16d13f4717e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:48:36.035324 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.auth.core [None req-832f0dad-30fc-40db-9b80-16d13f4717e6 None None] MFA Rules not processed for user `6b2199ae0fb0480aa917ada391f5ec54`. Rule list: `[]` (Enabled: `True`). {{(pid=81768) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 15:48:36.037361 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-832f0dad-30fc-40db-9b80-16d13f4717e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 15:48:36.142227 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15f9f2ee-f7ea-4373-a19e-dbef8817aa90 None admin] Validating token access rules against request {{(pid=81767) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 15:48:36.150031 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 547/1070] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Aug 22 15:48:36 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 22 15:48:36.203635 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bb723e7-1eb5-46c1-9585-b2e1ac03e667 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81767) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 15:48:36.204662 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb723e7-1eb5-46c1-9585-b2e1ac03e667 None admin] REQUEST_METHOD: `GET` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:48:36.205100 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb723e7-1eb5-46c1-9585-b2e1ac03e667 None admin] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:48:36.205568 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb723e7-1eb5-46c1-9585-b2e1ac03e667 None admin] PATH_INFO: `/v3/domains` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:48:36.206721 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bb723e7-1eb5-46c1-9585-b2e1ac03e667 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81767) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 15:48:36.207200 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bb723e7-1eb5-46c1-9585-b2e1ac03e667 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 15:48:36.209783 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bb723e7-1eb5-46c1-9585-b2e1ac03e667 None admin] RBAC: Authorization granted {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 15:48:36.238754 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 524/1071] 10.222.0.15 () {56 vars in 1109 bytes} [Mon Aug 22 15:48:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 22 15:48:36.240843 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-832f0dad-30fc-40db-9b80-16d13f4717e6 None None] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 15:48:36.306116 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=100 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 15:48:36.443777 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-832f0dad-30fc-40db-9b80-16d13f4717e6 None None] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 15:48:36.510698 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88038698-0b9c-47b0-a35a-48f77c92a6cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 15:48:36.511848 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88038698-0b9c-47b0-a35a-48f77c92a6cb None admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:48:36.512345 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88038698-0b9c-47b0-a35a-48f77c92a6cb None admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:48:36.512841 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88038698-0b9c-47b0-a35a-48f77c92a6cb None admin] PATH_INFO: `/v3/projects/service` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:48:36.525546 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bb723e7-1eb5-46c1-9585-b2e1ac03e667 None admin] Authenticating user token {{(pid=81767) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 15:48:36.526335 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88038698-0b9c-47b0-a35a-48f77c92a6cb None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 15:48:36.529087 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG dbcounter [-] [81767] Writing DB stats keystone:SELECT=100 {{(pid=81767) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 15:48:36.529240 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88038698-0b9c-47b0-a35a-48f77c92a6cb None admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 15:48:36.550961 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application [None req-88038698-0b9c-47b0-a35a-48f77c92a6cb None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 22 15:48:36.550961 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 15:48:36.550961 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 15:48:36.550961 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 15:48:36.550961 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 15:48:36.550961 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 15:48:36.550961 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 15:48:36.550961 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 15:48:36.550961 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 15:48:36.550961 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 15:48:36.550961 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 15:48:36.550961 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 15:48:36.550961 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 22 15:48:36.550961 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 22 15:48:36.550961 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 22 15:48:36.559232 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 22 15:48:36.559232 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 15:48:36.559232 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 15:48:36.559232 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 15:48:36.559232 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 15:48:36.559232 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 15:48:36.559232 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 15:48:36.559232 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 15:48:36.559232 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application with Lock( Aug 22 15:48:36.559232 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 15:48:36.559232 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self._enter() Aug 22 15:48:36.559232 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 15:48:36.559232 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 15:48:36.559232 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 15:48:36.559232 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self.creator() Aug 22 15:48:36.559232 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 15:48:36.559232 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application created_value = creator( Aug 22 15:48:36.561144 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 22 15:48:36.561144 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 22 15:48:36.561144 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 22 15:48:36.561144 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 22 15:48:36.561144 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 22 15:48:36.561144 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 22 15:48:36.561144 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 22 15:48:36.561144 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application Aug 22 15:48:36.561144 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 548/1072] 10.222.0.15 () {56 vars in 1111 bytes} [Mon Aug 22 15:48:36 2022] GET /identity/v3/projects/service => generated 88 bytes in 313 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 15:48:36.561144 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88038698-0b9c-47b0-a35a-48f77c92a6cb None admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 15:48:36.686453 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=100 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 15:48:36.694482 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88038698-0b9c-47b0-a35a-48f77c92a6cb None admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 15:48:36.702945 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bb723e7-1eb5-46c1-9585-b2e1ac03e667 None admin] Validating token access rules against request {{(pid=81767) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 15:48:36.737343 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf3a9b7b-8770-495e-964c-b6a6decdc7eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 15:48:36.738056 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf3a9b7b-8770-495e-964c-b6a6decdc7eb None admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:48:36.738336 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf3a9b7b-8770-495e-964c-b6a6decdc7eb None admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:48:36.738604 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf3a9b7b-8770-495e-964c-b6a6decdc7eb None admin] PATH_INFO: `/v3/projects` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:48:36.739368 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf3a9b7b-8770-495e-964c-b6a6decdc7eb None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81768) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 15:48:36.739671 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf3a9b7b-8770-495e-964c-b6a6decdc7eb None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 15:48:36.741111 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf3a9b7b-8770-495e-964c-b6a6decdc7eb None admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 15:48:36.754039 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 549/1073] 10.222.0.15 () {56 vars in 1148 bytes} [Mon Aug 22 15:48:36 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 15:48:36.759016 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1516d598-830c-4243-92c2-b49ef819ad88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81767) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 15:48:36.760048 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf3a9b7b-8770-495e-964c-b6a6decdc7eb None admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 15:48:36.760990 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1516d598-830c-4243-92c2-b49ef819ad88 None admin] REQUEST_METHOD: `GET` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:48:36.761199 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1516d598-830c-4243-92c2-b49ef819ad88 None admin] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:48:36.761468 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1516d598-830c-4243-92c2-b49ef819ad88 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:48:36.772440 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1516d598-830c-4243-92c2-b49ef819ad88 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 22 15:48:36.777810 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application [None req-1516d598-830c-4243-92c2-b49ef819ad88 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 22 15:48:36.777810 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 15:48:36.777810 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 15:48:36.777810 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 15:48:36.777810 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 15:48:36.777810 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 15:48:36.777810 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 15:48:36.777810 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 15:48:36.777810 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 15:48:36.777810 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 15:48:36.777810 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 15:48:36.777810 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 15:48:36.777810 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 22 15:48:36.777810 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 22 15:48:36.777810 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 22 15:48:36.777810 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 22 15:48:36.783683 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 22 15:48:36.783683 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 22 15:48:36.783683 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 22 15:48:36.783683 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 22 15:48:36.783683 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 15:48:36.783683 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 15:48:36.783683 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 15:48:36.783683 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 15:48:36.783683 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 15:48:36.783683 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 15:48:36.783683 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 15:48:36.783683 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application with Lock( Aug 22 15:48:36.783683 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 15:48:36.783683 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return self._enter() Aug 22 15:48:36.783683 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 15:48:36.783683 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 15:48:36.784566 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 15:48:36.784566 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return self.creator() Aug 22 15:48:36.784566 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 15:48:36.784566 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application created_value = creator( Aug 22 15:48:36.784566 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 22 15:48:36.784566 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 22 15:48:36.784566 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 22 15:48:36.784566 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 22 15:48:36.784566 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 22 15:48:36.784566 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 22 15:48:36.784566 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 22 15:48:36.784566 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application Aug 22 15:48:36.784566 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 525/1074] 10.222.0.15 () {56 vars in 1096 bytes} [Mon Aug 22 15:48:36 2022] GET /identity/v3/roles/admin => generated 83 bytes in 255 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 22 15:48:36.786489 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1516d598-830c-4243-92c2-b49ef819ad88 None admin] Authenticating user token {{(pid=81767) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 15:48:36.865464 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG dbcounter [-] [81767] Writing DB stats keystone:SELECT=100 {{(pid=81767) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 15:48:36.922613 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1516d598-830c-4243-92c2-b49ef819ad88 None admin] Validating token access rules against request {{(pid=81767) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 15:48:36.925554 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf3a9b7b-8770-495e-964c-b6a6decdc7eb None admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 15:48:36.967072 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f066b85-3ea0-4ea9-97eb-f26865d3e77a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81767) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 15:48:36.968010 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f066b85-3ea0-4ea9-97eb-f26865d3e77a None admin] REQUEST_METHOD: `GET` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:48:36.968298 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f066b85-3ea0-4ea9-97eb-f26865d3e77a None admin] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:48:36.969637 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f066b85-3ea0-4ea9-97eb-f26865d3e77a None admin] PATH_INFO: `/v3/roles` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:48:36.970333 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f066b85-3ea0-4ea9-97eb-f26865d3e77a None admin] RBAC: Adding query filter params (name=admin) {{(pid=81767) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 15:48:36.970634 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f066b85-3ea0-4ea9-97eb-f26865d3e77a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 15:48:36.972055 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f066b85-3ea0-4ea9-97eb-f26865d3e77a None admin] RBAC: Authorization granted {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 15:48:36.976814 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51f1881b-fcbc-4aa6-a354-2c88932c9168 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 15:48:36.977957 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51f1881b-fcbc-4aa6-a354-2c88932c9168 None admin] REQUEST_METHOD: `PUT` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:48:36.978616 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51f1881b-fcbc-4aa6-a354-2c88932c9168 None admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:48:36.979091 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51f1881b-fcbc-4aa6-a354-2c88932c9168 None admin] PATH_INFO: `/v3/projects/c4ee0413f9ef4d4fb1f95a660babe090/users/e4ef569e41fb46d0b721bb1c7663616f/roles/806eb8803ece4a1fa1bc2487b4cd7c02` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:48:36.982242 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 526/1075] 10.222.0.15 () {56 vars in 1099 bytes} [Mon Aug 22 15:48:36 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 22 15:48:36.983302 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51f1881b-fcbc-4aa6-a354-2c88932c9168 None admin] RBAC: Authorizing `identity:create_grant(project_id=c4ee0413f9ef4d4fb1f95a660babe090, user_id=e4ef569e41fb46d0b721bb1c7663616f, role_id=806eb8803ece4a1fa1bc2487b4cd7c02)` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 15:48:36.986084 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51f1881b-fcbc-4aa6-a354-2c88932c9168 None admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 15:48:36.987992 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f066b85-3ea0-4ea9-97eb-f26865d3e77a None admin] Authenticating user token {{(pid=81767) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 15:48:37.098559 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f066b85-3ea0-4ea9-97eb-f26865d3e77a None admin] Validating token access rules against request {{(pid=81767) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 15:48:37.122666 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG dbcounter [-] [81767] Writing DB stats keystone:SELECT=100 {{(pid=81767) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 15:48:37.134218 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ae8e0cb-6de4-4198-a4d9-0650b6ef68d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81767) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 15:48:37.134873 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ae8e0cb-6de4-4198-a4d9-0650b6ef68d5 None admin] REQUEST_METHOD: `GET` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:48:37.135149 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ae8e0cb-6de4-4198-a4d9-0650b6ef68d5 None admin] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:48:37.135449 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ae8e0cb-6de4-4198-a4d9-0650b6ef68d5 None admin] PATH_INFO: `/v3/users/45eb7849209b48c6bbcf379cab77e02a` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:48:37.136714 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ae8e0cb-6de4-4198-a4d9-0650b6ef68d5 None admin] RBAC: Authorizing `identity:get_user(user_id=45eb7849209b48c6bbcf379cab77e02a)` {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 15:48:37.138166 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ae8e0cb-6de4-4198-a4d9-0650b6ef68d5 None admin] RBAC: Authorization granted {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 15:48:37.139911 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 527/1076] 10.222.0.15 () {56 vars in 1177 bytes} [Mon Aug 22 15:48:36 2022] GET /identity/v3/users/45eb7849209b48c6bbcf379cab77e02a => generated 285 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 22 15:48:37.146145 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ae8e0cb-6de4-4198-a4d9-0650b6ef68d5 None admin] Authenticating user token {{(pid=81767) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 15:48:37.223153 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 550/1077] 10.222.0.15 () {58 vars in 1439 bytes} [Mon Aug 22 15:48:36 2022] PUT /identity/v3/projects/c4ee0413f9ef4d4fb1f95a660babe090/users/e4ef569e41fb46d0b721bb1c7663616f/roles/806eb8803ece4a1fa1bc2487b4cd7c02 => generated 0 bytes in 463 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 22 15:48:37.266156 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ae8e0cb-6de4-4198-a4d9-0650b6ef68d5 None admin] Validating token access rules against request {{(pid=81767) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 15:48:37.316350 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1b7ca25-c3fc-43e0-8fd5-c62f25d031fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81767) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 15:48:37.317010 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1b7ca25-c3fc-43e0-8fd5-c62f25d031fa None admin] REQUEST_METHOD: `GET` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:48:37.317369 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1b7ca25-c3fc-43e0-8fd5-c62f25d031fa None admin] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:48:37.317671 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1b7ca25-c3fc-43e0-8fd5-c62f25d031fa None admin] PATH_INFO: `/v3/projects/2b65932e1345451c880e9d46bd66f7f3` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:48:37.319498 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1b7ca25-c3fc-43e0-8fd5-c62f25d031fa None admin] RBAC: Authorizing `identity:get_project(project_id=2b65932e1345451c880e9d46bd66f7f3)` {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 15:48:37.321554 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1b7ca25-c3fc-43e0-8fd5-c62f25d031fa None admin] RBAC: Authorization granted {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 15:48:37.326131 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 528/1078] 10.222.0.15 () {56 vars in 1186 bytes} [Mon Aug 22 15:48:37 2022] GET /identity/v3/projects/2b65932e1345451c880e9d46bd66f7f3 => generated 311 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 15:48:37.331733 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51f1881b-fcbc-4aa6-a354-2c88932c9168 None admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 15:48:37.365352 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 15:48:37.469699 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51f1881b-fcbc-4aa6-a354-2c88932c9168 None admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 15:48:37.513552 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-742ab65a-3e61-4684-9102-38df425ba271 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 15:48:37.514222 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-742ab65a-3e61-4684-9102-38df425ba271 None admin] REQUEST_METHOD: `PUT` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:48:37.514535 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-742ab65a-3e61-4684-9102-38df425ba271 None admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:48:37.514809 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-742ab65a-3e61-4684-9102-38df425ba271 None admin] PATH_INFO: `/v3/projects/2b65932e1345451c880e9d46bd66f7f3/users/45eb7849209b48c6bbcf379cab77e02a/roles/9d9848cfd22e4b089db443c350e56670` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:48:37.517056 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-742ab65a-3e61-4684-9102-38df425ba271 None admin] RBAC: Authorizing `identity:create_grant(project_id=2b65932e1345451c880e9d46bd66f7f3, user_id=45eb7849209b48c6bbcf379cab77e02a, role_id=9d9848cfd22e4b089db443c350e56670)` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 15:48:37.518660 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-742ab65a-3e61-4684-9102-38df425ba271 None admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 15:48:37.533864 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1738adcb-65d2-4edd-843d-b17d2b23abf9 None None] REQUEST_METHOD: `GET` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:48:37.536801 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1738adcb-65d2-4edd-843d-b17d2b23abf9 None None] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:48:37.536801 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1738adcb-65d2-4edd-843d-b17d2b23abf9 None None] PATH_INFO: `/` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:48:37.536801 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 529/1079] 10.222.0.15 () {54 vars in 927 bytes} [Mon Aug 22 15:48:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 22 15:48:37.543946 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae72efc5-35da-4797-a4d1-ceb9b90e02f5 None None] REQUEST_METHOD: `POST` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:48:37.544470 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae72efc5-35da-4797-a4d1-ceb9b90e02f5 None None] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:48:37.544950 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae72efc5-35da-4797-a4d1-ceb9b90e02f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:48:37.566428 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.auth.core [None req-ae72efc5-35da-4797-a4d1-ceb9b90e02f5 None None] MFA Rules not processed for user `6b2199ae0fb0480aa917ada391f5ec54`. Rule list: `[]` (Enabled: `True`). {{(pid=81767) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 15:48:37.567691 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.fernet_utils [None req-ae72efc5-35da-4797-a4d1-ceb9b90e02f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81767) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 15:48:37.612022 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 551/1080] 10.222.0.15 () {58 vars in 1439 bytes} [Mon Aug 22 15:48:37 2022] PUT /identity/v3/projects/2b65932e1345451c880e9d46bd66f7f3/users/45eb7849209b48c6bbcf379cab77e02a/roles/9d9848cfd22e4b089db443c350e56670 => generated 0 bytes in 281 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 15:48:37.642242 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG dbcounter [-] [81767] Writing DB stats keystone:SELECT=100 {{(pid=81767) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 15:48:37.657366 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 530/1081] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Aug 22 15:48:37 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 22 15:48:37.939612 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-742ab65a-3e61-4684-9102-38df425ba271 None admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 15:48:38.020528 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 15:48:38.047184 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-742ab65a-3e61-4684-9102-38df425ba271 None admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 15:48:38.083981 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-380f0f04-9006-4589-bd81-c283c12ba92f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 15:48:38.084601 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-380f0f04-9006-4589-bd81-c283c12ba92f None admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:48:38.084875 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-380f0f04-9006-4589-bd81-c283c12ba92f None admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:48:38.085138 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-380f0f04-9006-4589-bd81-c283c12ba92f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:48:38.161765 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-380f0f04-9006-4589-bd81-c283c12ba92f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 15:48:38.163224 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-380f0f04-9006-4589-bd81-c283c12ba92f None admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 15:48:38.246433 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=100 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 15:48:38.279325 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 552/1082] 10.222.0.15 () {58 vars in 1282 bytes} [Mon Aug 22 15:48:37 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 338 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 22 15:48:38.282578 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae72efc5-35da-4797-a4d1-ceb9b90e02f5 None None] Authenticating user token {{(pid=81767) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 15:48:38.427293 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae72efc5-35da-4797-a4d1-ceb9b90e02f5 None None] Validating token access rules against request {{(pid=81767) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 15:48:38.467360 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8965bb5-e12a-4af9-a9e0-7038df3557ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81767) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 15:48:38.467993 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8965bb5-e12a-4af9-a9e0-7038df3557ad None admin] REQUEST_METHOD: `GET` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:48:38.468269 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8965bb5-e12a-4af9-a9e0-7038df3557ad None admin] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:48:38.468538 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8965bb5-e12a-4af9-a9e0-7038df3557ad None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:48:38.481447 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8965bb5-e12a-4af9-a9e0-7038df3557ad None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 15:48:38.482909 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8965bb5-e12a-4af9-a9e0-7038df3557ad None admin] RBAC: Authorization granted {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 15:48:38.503822 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application [None req-e8965bb5-e12a-4af9-a9e0-7038df3557ad None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 22 15:48:38.503822 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 15:48:38.503822 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 15:48:38.503822 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 15:48:38.503822 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 15:48:38.503822 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 15:48:38.503822 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 15:48:38.503822 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 15:48:38.503822 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 15:48:38.503822 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 15:48:38.503822 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 15:48:38.503822 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 15:48:38.503822 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 22 15:48:38.503822 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 22 15:48:38.503822 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 22 15:48:38.503822 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 22 15:48:38.504852 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 15:48:38.504852 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 15:48:38.504852 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 15:48:38.504852 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 15:48:38.504852 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 15:48:38.504852 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 15:48:38.504852 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 15:48:38.504852 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application with Lock( Aug 22 15:48:38.504852 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 15:48:38.504852 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return self._enter() Aug 22 15:48:38.504852 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 15:48:38.504852 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 15:48:38.504852 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 15:48:38.504852 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return self.creator() Aug 22 15:48:38.504852 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 15:48:38.504852 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application created_value = creator( Aug 22 15:48:38.504852 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 22 15:48:38.505781 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 22 15:48:38.505781 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 22 15:48:38.505781 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application Aug 22 15:48:38.506678 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 531/1083] 10.222.0.15 () {56 vars in 1108 bytes} [Mon Aug 22 15:48:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 225 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 15:48:38.511485 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-380f0f04-9006-4589-bd81-c283c12ba92f None admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 15:48:38.633984 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-380f0f04-9006-4589-bd81-c283c12ba92f None admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 15:48:38.675103 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0657074a-460c-4f7a-b901-a18534db063f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 15:48:38.675814 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0657074a-460c-4f7a-b901-a18534db063f None admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:48:38.676093 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0657074a-460c-4f7a-b901-a18534db063f None admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:48:38.676360 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0657074a-460c-4f7a-b901-a18534db063f None admin] PATH_INFO: `/v3/domains` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:48:38.677008 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0657074a-460c-4f7a-b901-a18534db063f None admin] RBAC: Adding query filter params (name=Default) {{(pid=81768) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 15:48:38.677309 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0657074a-460c-4f7a-b901-a18534db063f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 15:48:38.678800 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0657074a-460c-4f7a-b901-a18534db063f None admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 15:48:38.691632 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 553/1084] 10.222.0.15 () {56 vars in 1109 bytes} [Mon Aug 22 15:48:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 15:48:39.266070 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7609c7c-7b74-476f-b547-d642810dc7f5 None None] REQUEST_METHOD: `GET` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:48:39.266641 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7609c7c-7b74-476f-b547-d642810dc7f5 None None] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:48:39.266641 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7609c7c-7b74-476f-b547-d642810dc7f5 None None] PATH_INFO: `/` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:48:39.268111 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 532/1085] 10.222.0.15 () {54 vars in 927 bytes} [Mon Aug 22 15:48:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 15:48:39.274704 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e26a9eb-b551-4837-ac69-0bc1cc7aad91 None None] REQUEST_METHOD: `POST` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:48:39.275247 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e26a9eb-b551-4837-ac69-0bc1cc7aad91 None None] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:48:39.275366 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e26a9eb-b551-4837-ac69-0bc1cc7aad91 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:48:39.294956 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.auth.core [None req-2e26a9eb-b551-4837-ac69-0bc1cc7aad91 None None] MFA Rules not processed for user `6b2199ae0fb0480aa917ada391f5ec54`. Rule list: `[]` (Enabled: `True`). {{(pid=81768) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 15:48:39.296194 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-2e26a9eb-b551-4837-ac69-0bc1cc7aad91 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 15:48:39.329520 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=100 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 15:48:39.376089 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 554/1086] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Aug 22 15:48:39 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 22 15:48:39.608979 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40799770-2451-4635-b869-69d93ed044a8 None None] REQUEST_METHOD: `GET` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:48:39.609326 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40799770-2451-4635-b869-69d93ed044a8 None None] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:48:39.609597 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40799770-2451-4635-b869-69d93ed044a8 None None] PATH_INFO: `/` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:48:39.611081 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 533/1087] 10.222.0.15 () {54 vars in 927 bytes} [Mon Aug 22 15:48:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 15:48:39.617046 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c63d77c7-0382-448c-88eb-a3b33e989637 None None] REQUEST_METHOD: `POST` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:48:39.617324 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c63d77c7-0382-448c-88eb-a3b33e989637 None None] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:48:39.617631 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c63d77c7-0382-448c-88eb-a3b33e989637 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:48:39.636141 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.auth.core [None req-c63d77c7-0382-448c-88eb-a3b33e989637 None None] MFA Rules not processed for user `6b2199ae0fb0480aa917ada391f5ec54`. Rule list: `[]` (Enabled: `True`). {{(pid=81768) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 15:48:39.637301 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-c63d77c7-0382-448c-88eb-a3b33e989637 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 15:48:39.649911 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40799770-2451-4635-b869-69d93ed044a8 None None] Authenticating user token {{(pid=81767) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 15:48:39.696869 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG dbcounter [-] [81767] Writing DB stats keystone:SELECT=100 {{(pid=81767) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 15:48:39.709174 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 555/1088] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Aug 22 15:48:39 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 22 15:48:39.757940 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40799770-2451-4635-b869-69d93ed044a8 None None] Validating token access rules against request {{(pid=81767) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 15:48:39.791084 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a36ca604-a162-49fd-a05c-98d79cdb3ae3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81767) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 15:48:39.791725 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a36ca604-a162-49fd-a05c-98d79cdb3ae3 None admin] REQUEST_METHOD: `GET` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:48:39.792052 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a36ca604-a162-49fd-a05c-98d79cdb3ae3 None admin] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:48:39.792261 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a36ca604-a162-49fd-a05c-98d79cdb3ae3 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:48:39.803097 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a36ca604-a162-49fd-a05c-98d79cdb3ae3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 22 15:48:39.808619 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application [None req-a36ca604-a162-49fd-a05c-98d79cdb3ae3 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 22 15:48:39.808619 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 15:48:39.808619 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 15:48:39.808619 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 15:48:39.808619 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 15:48:39.808619 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 15:48:39.808619 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 15:48:39.808619 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 15:48:39.808619 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 15:48:39.808619 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 15:48:39.808619 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 15:48:39.808619 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 15:48:39.808619 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 22 15:48:39.808619 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 22 15:48:39.808619 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 22 15:48:39.808619 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 22 15:48:39.809889 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 22 15:48:39.809889 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 22 15:48:39.809889 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 22 15:48:39.809889 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 22 15:48:39.809889 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 15:48:39.809889 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 15:48:39.809889 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 15:48:39.809889 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 15:48:39.809889 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 15:48:39.809889 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 15:48:39.809889 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 15:48:39.809889 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application with Lock( Aug 22 15:48:39.809889 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 15:48:39.809889 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return self._enter() Aug 22 15:48:39.809889 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 15:48:39.809889 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 15:48:39.811067 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 15:48:39.811067 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return self.creator() Aug 22 15:48:39.811067 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 15:48:39.811067 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application created_value = creator( Aug 22 15:48:39.811067 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 22 15:48:39.811067 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 22 15:48:39.811067 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 22 15:48:39.811067 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 22 15:48:39.811067 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 22 15:48:39.811067 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 22 15:48:39.811067 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 22 15:48:39.811067 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application Aug 22 15:48:39.811067 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 534/1089] 10.222.0.15 () {56 vars in 1120 bytes} [Mon Aug 22 15:48:39 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 15:48:39.815710 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c63d77c7-0382-448c-88eb-a3b33e989637 None None] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 15:48:39.910600 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=100 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 15:48:39.917918 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c63d77c7-0382-448c-88eb-a3b33e989637 None None] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 15:48:39.952261 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a399b93c-a8dc-42f2-b3c4-4e4c758fa21d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 15:48:39.952858 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a399b93c-a8dc-42f2-b3c4-4e4c758fa21d None admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:48:39.953125 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a399b93c-a8dc-42f2-b3c4-4e4c758fa21d None admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:48:39.953387 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a399b93c-a8dc-42f2-b3c4-4e4c758fa21d None admin] PATH_INFO: `/v3/roles` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:48:39.954040 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a399b93c-a8dc-42f2-b3c4-4e4c758fa21d None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81768) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 15:48:39.954340 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a399b93c-a8dc-42f2-b3c4-4e4c758fa21d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 15:48:39.955742 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a399b93c-a8dc-42f2-b3c4-4e4c758fa21d None admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 15:48:39.964605 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 556/1090] 10.222.0.15 () {56 vars in 1115 bytes} [Mon Aug 22 15:48:39 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 15:48:39.970473 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a36ca604-a162-49fd-a05c-98d79cdb3ae3 None admin] Authenticating user token {{(pid=81767) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 15:48:39.985080 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a399b93c-a8dc-42f2-b3c4-4e4c758fa21d None admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 15:48:40.078241 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a36ca604-a162-49fd-a05c-98d79cdb3ae3 None admin] Validating token access rules against request {{(pid=81767) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 15:48:40.079880 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG dbcounter [-] [81767] Writing DB stats keystone:SELECT=100 {{(pid=81767) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 15:48:40.089407 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a399b93c-a8dc-42f2-b3c4-4e4c758fa21d None admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 15:48:40.113593 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47ee434e-1783-41d6-9644-82157904f7f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81767) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 15:48:40.114220 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47ee434e-1783-41d6-9644-82157904f7f2 None admin] REQUEST_METHOD: `GET` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:48:40.114496 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47ee434e-1783-41d6-9644-82157904f7f2 None admin] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:48:40.114763 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47ee434e-1783-41d6-9644-82157904f7f2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:48:40.124598 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09633d04-e579-41e7-b21e-5944851a50fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 15:48:40.125219 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09633d04-e579-41e7-b21e-5944851a50fe None admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:48:40.125488 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09633d04-e579-41e7-b21e-5944851a50fe None admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:48:40.125752 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09633d04-e579-41e7-b21e-5944851a50fe None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:48:40.125843 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47ee434e-1783-41d6-9644-82157904f7f2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 15:48:40.127229 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47ee434e-1783-41d6-9644-82157904f7f2 None admin] RBAC: Authorization granted {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 15:48:40.135771 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09633d04-e579-41e7-b21e-5944851a50fe None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 22 15:48:40.140857 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application [None req-09633d04-e579-41e7-b21e-5944851a50fe None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 22 15:48:40.140857 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 15:48:40.140857 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 15:48:40.140857 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 15:48:40.140857 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 15:48:40.140857 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 15:48:40.140857 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 15:48:40.140857 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 15:48:40.140857 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 15:48:40.140857 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 15:48:40.140857 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 15:48:40.140857 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 15:48:40.140857 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 22 15:48:40.140857 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 22 15:48:40.140857 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 22 15:48:40.140857 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 22 15:48:40.141822 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 22 15:48:40.141822 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 22 15:48:40.141822 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 22 15:48:40.141822 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 22 15:48:40.141822 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 15:48:40.141822 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 15:48:40.141822 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 15:48:40.141822 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 15:48:40.141822 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 15:48:40.141822 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 15:48:40.141822 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 15:48:40.141822 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application with Lock( Aug 22 15:48:40.141822 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 15:48:40.141822 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self._enter() Aug 22 15:48:40.141822 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 15:48:40.141822 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 15:48:40.142678 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 15:48:40.142678 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self.creator() Aug 22 15:48:40.142678 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 15:48:40.142678 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application created_value = creator( Aug 22 15:48:40.142678 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 22 15:48:40.142678 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 22 15:48:40.142678 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 22 15:48:40.142678 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 22 15:48:40.142678 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 22 15:48:40.142678 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 22 15:48:40.142678 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 22 15:48:40.142678 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application Aug 22 15:48:40.142678 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 557/1091] 10.222.0.15 () {56 vars in 1096 bytes} [Mon Aug 22 15:48:39 2022] GET /identity/v3/roles/admin => generated 83 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 15:48:40.143410 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application [None req-47ee434e-1783-41d6-9644-82157904f7f2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 22 15:48:40.143410 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 15:48:40.143410 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 15:48:40.143410 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 15:48:40.143410 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 15:48:40.143410 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 15:48:40.143410 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 15:48:40.143410 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 15:48:40.143410 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 15:48:40.143410 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 15:48:40.143410 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 15:48:40.143410 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 15:48:40.143410 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 22 15:48:40.143410 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 22 15:48:40.143410 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 22 15:48:40.143410 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 22 15:48:40.144278 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 15:48:40.144278 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 15:48:40.144278 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 15:48:40.144278 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 15:48:40.144278 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 15:48:40.144278 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 15:48:40.144278 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 15:48:40.144278 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application with Lock( Aug 22 15:48:40.144278 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 15:48:40.144278 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return self._enter() Aug 22 15:48:40.144278 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 15:48:40.144278 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 15:48:40.144278 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 15:48:40.144278 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return self.creator() Aug 22 15:48:40.144278 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 15:48:40.144278 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application created_value = creator( Aug 22 15:48:40.144278 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 22 15:48:40.145179 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 22 15:48:40.145179 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 22 15:48:40.145179 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application Aug 22 15:48:40.145179 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 535/1092] 10.222.0.15 () {56 vars in 1108 bytes} [Mon Aug 22 15:48:39 2022] GET /identity/v3/domains/Default => generated 87 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 22 15:48:40.150804 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09633d04-e579-41e7-b21e-5944851a50fe None admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 15:48:40.151053 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47ee434e-1783-41d6-9644-82157904f7f2 None admin] Authenticating user token {{(pid=81767) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 15:48:40.170146 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=100 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 15:48:40.259394 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09633d04-e579-41e7-b21e-5944851a50fe None admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 15:48:40.259938 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47ee434e-1783-41d6-9644-82157904f7f2 None admin] Validating token access rules against request {{(pid=81767) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 15:48:40.294481 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e5deb1a-c151-4163-8f52-9b7747e06e12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81767) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 15:48:40.295167 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e5deb1a-c151-4163-8f52-9b7747e06e12 None admin] REQUEST_METHOD: `GET` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:48:40.295369 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e5deb1a-c151-4163-8f52-9b7747e06e12 None admin] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:48:40.295629 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e5deb1a-c151-4163-8f52-9b7747e06e12 None admin] PATH_INFO: `/v3/roles` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:48:40.295848 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bce59a7c-73be-42f3-9140-d701f363b5fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 15:48:40.296278 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e5deb1a-c151-4163-8f52-9b7747e06e12 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81767) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 15:48:40.296451 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bce59a7c-73be-42f3-9140-d701f363b5fb None admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:48:40.296556 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e5deb1a-c151-4163-8f52-9b7747e06e12 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 15:48:40.296729 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bce59a7c-73be-42f3-9140-d701f363b5fb None admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:48:40.297001 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bce59a7c-73be-42f3-9140-d701f363b5fb None admin] PATH_INFO: `/v3/domains` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:48:40.297638 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bce59a7c-73be-42f3-9140-d701f363b5fb None admin] RBAC: Adding query filter params (name=Default) {{(pid=81768) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 15:48:40.297912 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e5deb1a-c151-4163-8f52-9b7747e06e12 None admin] RBAC: Authorization granted {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 15:48:40.298014 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bce59a7c-73be-42f3-9140-d701f363b5fb None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 15:48:40.299295 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bce59a7c-73be-42f3-9140-d701f363b5fb None admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 15:48:40.306220 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 536/1093] 10.222.0.15 () {56 vars in 1099 bytes} [Mon Aug 22 15:48:40 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 22 15:48:40.311093 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 558/1094] 10.222.0.15 () {56 vars in 1109 bytes} [Mon Aug 22 15:48:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 15:48:40.312338 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e5deb1a-c151-4163-8f52-9b7747e06e12 None admin] Authenticating user token {{(pid=81767) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 15:48:40.318075 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bce59a7c-73be-42f3-9140-d701f363b5fb None admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 15:48:40.332670 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG dbcounter [-] [81767] Writing DB stats keystone:SELECT=100 {{(pid=81767) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 15:48:40.401041 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=100 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 15:48:40.418319 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e5deb1a-c151-4163-8f52-9b7747e06e12 None admin] Validating token access rules against request {{(pid=81767) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 15:48:40.423580 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bce59a7c-73be-42f3-9140-d701f363b5fb None admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 15:48:40.454153 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-766ab673-95e1-4e0c-8ae7-ac7a25060dde None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81767) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 15:48:40.454777 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-766ab673-95e1-4e0c-8ae7-ac7a25060dde None admin] REQUEST_METHOD: `GET` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:48:40.455053 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-766ab673-95e1-4e0c-8ae7-ac7a25060dde None admin] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:48:40.455356 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-766ab673-95e1-4e0c-8ae7-ac7a25060dde None admin] PATH_INFO: `/v3/users/45eb7849209b48c6bbcf379cab77e02a` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:48:40.456512 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-766ab673-95e1-4e0c-8ae7-ac7a25060dde None admin] RBAC: Authorizing `identity:get_user(user_id=45eb7849209b48c6bbcf379cab77e02a)` {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 15:48:40.457927 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-766ab673-95e1-4e0c-8ae7-ac7a25060dde None admin] RBAC: Authorization granted {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 15:48:40.459535 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 537/1095] 10.222.0.15 () {56 vars in 1177 bytes} [Mon Aug 22 15:48:40 2022] GET /identity/v3/users/45eb7849209b48c6bbcf379cab77e02a => generated 285 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 15:48:40.465713 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9ec7103-c4cf-41d4-ba4a-8ae05b86fb63 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 15:48:40.466165 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-766ab673-95e1-4e0c-8ae7-ac7a25060dde None admin] Authenticating user token {{(pid=81767) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 15:48:40.466337 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9ec7103-c4cf-41d4-ba4a-8ae05b86fb63 None admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:48:40.466611 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9ec7103-c4cf-41d4-ba4a-8ae05b86fb63 None admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:48:40.466881 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9ec7103-c4cf-41d4-ba4a-8ae05b86fb63 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:48:40.474710 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9ec7103-c4cf-41d4-ba4a-8ae05b86fb63 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 15:48:40.476167 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9ec7103-c4cf-41d4-ba4a-8ae05b86fb63 None admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 15:48:40.490384 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application [None req-f9ec7103-c4cf-41d4-ba4a-8ae05b86fb63 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 22 15:48:40.490384 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 15:48:40.490384 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 15:48:40.490384 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 15:48:40.490384 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 15:48:40.490384 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 15:48:40.490384 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 15:48:40.490384 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 15:48:40.490384 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 15:48:40.490384 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 15:48:40.490384 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 15:48:40.490384 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 15:48:40.490384 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 22 15:48:40.490384 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 22 15:48:40.490384 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 22 15:48:40.490384 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 22 15:48:40.491665 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 15:48:40.491665 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 15:48:40.491665 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 22 15:48:40.491665 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 22 15:48:40.491665 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 22 15:48:40.491665 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 22 15:48:40.491665 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 15:48:40.491665 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 15:48:40.491665 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 15:48:40.491665 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 15:48:40.491665 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 15:48:40.491665 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application with Lock( Aug 22 15:48:40.491665 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 15:48:40.491665 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self._enter() Aug 22 15:48:40.491665 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 15:48:40.491665 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 15:48:40.491665 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 15:48:40.491665 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self.creator() Aug 22 15:48:40.492779 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 15:48:40.492779 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application created_value = creator( Aug 22 15:48:40.492779 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 22 15:48:40.492779 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 22 15:48:40.492779 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 22 15:48:40.492779 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 22 15:48:40.492779 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 22 15:48:40.492779 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 22 15:48:40.492779 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 22 15:48:40.492779 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application Aug 22 15:48:40.492779 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 559/1096] 10.222.0.15 () {56 vars in 1093 bytes} [Mon Aug 22 15:48:40 2022] GET /identity/v3/users/nova => generated 82 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 22 15:48:40.497282 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9ec7103-c4cf-41d4-ba4a-8ae05b86fb63 None admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 15:48:40.555228 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG dbcounter [-] [81767] Writing DB stats keystone:SELECT=100 {{(pid=81767) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 15:48:40.571884 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-766ab673-95e1-4e0c-8ae7-ac7a25060dde None admin] Validating token access rules against request {{(pid=81767) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 15:48:40.602628 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9ec7103-c4cf-41d4-ba4a-8ae05b86fb63 None admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 15:48:40.611197 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f81ef4bb-8e2f-4c2e-84cc-82c0600e9f6e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81767) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 15:48:40.611834 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f81ef4bb-8e2f-4c2e-84cc-82c0600e9f6e None admin] REQUEST_METHOD: `GET` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:48:40.612107 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f81ef4bb-8e2f-4c2e-84cc-82c0600e9f6e None admin] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:48:40.612371 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f81ef4bb-8e2f-4c2e-84cc-82c0600e9f6e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:48:40.637513 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dffa5950-29eb-401e-bd81-2831e39a088d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 15:48:40.638120 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dffa5950-29eb-401e-bd81-2831e39a088d None admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:48:40.638392 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dffa5950-29eb-401e-bd81-2831e39a088d None admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:48:40.638662 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dffa5950-29eb-401e-bd81-2831e39a088d None admin] PATH_INFO: `/v3/users` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:48:40.639573 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dffa5950-29eb-401e-bd81-2831e39a088d None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81768) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 15:48:40.639701 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dffa5950-29eb-401e-bd81-2831e39a088d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 15:48:40.641128 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dffa5950-29eb-401e-bd81-2831e39a088d None admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 15:48:40.643838 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=100 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 15:48:40.662538 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 560/1097] 10.222.0.15 () {56 vars in 1133 bytes} [Mon Aug 22 15:48:40 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 22 15:48:40.668273 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dffa5950-29eb-401e-bd81-2831e39a088d None admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 15:48:40.691969 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f81ef4bb-8e2f-4c2e-84cc-82c0600e9f6e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 15:48:40.693334 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f81ef4bb-8e2f-4c2e-84cc-82c0600e9f6e None admin] RBAC: Authorization granted {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 15:48:40.779987 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dffa5950-29eb-401e-bd81-2831e39a088d None admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 15:48:40.780200 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG dbcounter [-] [81767] Writing DB stats keystone:SELECT=100 {{(pid=81767) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 15:48:40.802223 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 538/1098] 10.222.0.15 () {58 vars in 1282 bytes} [Mon Aug 22 15:48:40 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 332 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Aug 22 15:48:40.803716 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f81ef4bb-8e2f-4c2e-84cc-82c0600e9f6e None admin] Authenticating user token {{(pid=81767) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 15:48:40.817087 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8ee448d-7b8d-40a9-9699-2169fbe71b73 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 15:48:40.817697 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8ee448d-7b8d-40a9-9699-2169fbe71b73 None admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:48:40.817969 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8ee448d-7b8d-40a9-9699-2169fbe71b73 None admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:48:40.818234 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8ee448d-7b8d-40a9-9699-2169fbe71b73 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:48:40.883356 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=100 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 15:48:40.891194 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8ee448d-7b8d-40a9-9699-2169fbe71b73 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 15:48:40.892775 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8ee448d-7b8d-40a9-9699-2169fbe71b73 None admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 15:48:40.913790 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f81ef4bb-8e2f-4c2e-84cc-82c0600e9f6e None admin] Validating token access rules against request {{(pid=81767) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 15:48:40.952551 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62b412e6-442e-4091-a395-ea3e6d674a28 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81767) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 15:48:40.952825 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62b412e6-442e-4091-a395-ea3e6d674a28 None admin] REQUEST_METHOD: `GET` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:48:40.953068 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62b412e6-442e-4091-a395-ea3e6d674a28 None admin] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:48:40.953331 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62b412e6-442e-4091-a395-ea3e6d674a28 None admin] PATH_INFO: `/v3/projects/2b65932e1345451c880e9d46bd66f7f3` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:48:40.954533 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62b412e6-442e-4091-a395-ea3e6d674a28 None admin] RBAC: Authorizing `identity:get_project(project_id=2b65932e1345451c880e9d46bd66f7f3)` {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 15:48:40.956009 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62b412e6-442e-4091-a395-ea3e6d674a28 None admin] RBAC: Authorization granted {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 15:48:40.957826 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 539/1099] 10.222.0.15 () {56 vars in 1186 bytes} [Mon Aug 22 15:48:40 2022] GET /identity/v3/projects/2b65932e1345451c880e9d46bd66f7f3 => generated 311 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 22 15:48:40.963770 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62b412e6-442e-4091-a395-ea3e6d674a28 None admin] Authenticating user token {{(pid=81767) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 15:48:41.000155 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 561/1100] 10.222.0.15 () {58 vars in 1282 bytes} [Mon Aug 22 15:48:40 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 332 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Aug 22 15:48:41.007642 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8ee448d-7b8d-40a9-9699-2169fbe71b73 None admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 15:48:41.027465 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG dbcounter [-] [81767] Writing DB stats keystone:SELECT=100 {{(pid=81767) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 15:48:41.092239 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62b412e6-442e-4091-a395-ea3e6d674a28 None admin] Validating token access rules against request {{(pid=81767) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 15:48:41.105398 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=100 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 15:48:41.110645 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8ee448d-7b8d-40a9-9699-2169fbe71b73 None admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 15:48:41.135113 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf53a59c-ebac-4c0d-8637-af4a7fc82712 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81767) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 15:48:41.163575 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf53a59c-ebac-4c0d-8637-af4a7fc82712 None admin] REQUEST_METHOD: `GET` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:48:41.163575 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf53a59c-ebac-4c0d-8637-af4a7fc82712 None admin] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:48:41.163575 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf53a59c-ebac-4c0d-8637-af4a7fc82712 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:48:41.163575 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf53a59c-ebac-4c0d-8637-af4a7fc82712 None admin] RBAC: Adding query filter params (role.id=9d9848cfd22e4b089db443c350e56670, scope.project.id=2b65932e1345451c880e9d46bd66f7f3, user.id=45eb7849209b48c6bbcf379cab77e02a) {{(pid=81767) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 15:48:41.163575 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf53a59c-ebac-4c0d-8637-af4a7fc82712 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 15:48:41.163575 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf53a59c-ebac-4c0d-8637-af4a7fc82712 None admin] RBAC: Authorization granted {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 15:48:41.164104 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-097a0663-e94a-4718-a214-db5f9361d96c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 15:48:41.164104 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-097a0663-e94a-4718-a214-db5f9361d96c None admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:48:41.164104 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-097a0663-e94a-4718-a214-db5f9361d96c None admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:48:41.164104 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-097a0663-e94a-4718-a214-db5f9361d96c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:48:41.164910 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 540/1101] 10.222.0.15 () {56 vars in 1374 bytes} [Mon Aug 22 15:48:40 2022] GET /identity/v3/role_assignments?user.id=45eb7849209b48c6bbcf379cab77e02a&scope.project.id=2b65932e1345451c880e9d46bd66f7f3&role.id=9d9848cfd22e4b089db443c350e56670 => generated 608 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 22 15:48:41.164994 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-097a0663-e94a-4718-a214-db5f9361d96c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 15:48:41.164994 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-097a0663-e94a-4718-a214-db5f9361d96c None admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 15:48:41.175998 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application [None req-097a0663-e94a-4718-a214-db5f9361d96c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 22 15:48:41.175998 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 15:48:41.175998 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 15:48:41.175998 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 15:48:41.175998 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 15:48:41.175998 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 15:48:41.175998 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 15:48:41.175998 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 15:48:41.175998 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 15:48:41.175998 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 15:48:41.175998 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 15:48:41.175998 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 15:48:41.175998 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 22 15:48:41.175998 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 22 15:48:41.175998 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 22 15:48:41.175998 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 22 15:48:41.177243 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 15:48:41.177243 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 15:48:41.177243 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 15:48:41.177243 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 15:48:41.177243 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 15:48:41.177243 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 15:48:41.177243 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 15:48:41.177243 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application with Lock( Aug 22 15:48:41.177243 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 15:48:41.177243 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self._enter() Aug 22 15:48:41.177243 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 15:48:41.177243 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 15:48:41.177243 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 15:48:41.177243 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self.creator() Aug 22 15:48:41.177243 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 15:48:41.177243 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application created_value = creator( Aug 22 15:48:41.177243 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 22 15:48:41.178353 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 22 15:48:41.178353 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 22 15:48:41.178353 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application Aug 22 15:48:41.178782 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 562/1102] 10.222.0.15 () {56 vars in 1108 bytes} [Mon Aug 22 15:48:41 2022] GET /identity/v3/domains/Default => generated 87 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 22 15:48:41.185614 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf53a59c-ebac-4c0d-8637-af4a7fc82712 None admin] Authenticating user token {{(pid=81767) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 15:48:41.302800 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf53a59c-ebac-4c0d-8637-af4a7fc82712 None admin] Validating token access rules against request {{(pid=81767) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 15:48:41.321517 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG dbcounter [-] [81767] Writing DB stats keystone:SELECT=100 {{(pid=81767) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 15:48:41.340517 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecf2bbcf-a262-445e-95da-c2bd10014310 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81767) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 15:48:41.341130 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecf2bbcf-a262-445e-95da-c2bd10014310 None admin] REQUEST_METHOD: `GET` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:48:41.341401 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecf2bbcf-a262-445e-95da-c2bd10014310 None admin] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:48:41.341672 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecf2bbcf-a262-445e-95da-c2bd10014310 None admin] PATH_INFO: `/v3/domains` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:48:41.342312 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecf2bbcf-a262-445e-95da-c2bd10014310 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81767) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 15:48:41.342612 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecf2bbcf-a262-445e-95da-c2bd10014310 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 15:48:41.344003 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecf2bbcf-a262-445e-95da-c2bd10014310 None admin] RBAC: Authorization granted {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 15:48:41.358482 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 541/1103] 10.222.0.15 () {56 vars in 1109 bytes} [Mon Aug 22 15:48:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 15:48:41.364381 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-097a0663-e94a-4718-a214-db5f9361d96c None admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 15:48:41.481199 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-097a0663-e94a-4718-a214-db5f9361d96c None admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 15:48:41.516696 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5159def5-6856-4180-a872-812fd6d40c0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 15:48:41.517316 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5159def5-6856-4180-a872-812fd6d40c0e None admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:48:41.517592 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5159def5-6856-4180-a872-812fd6d40c0e None admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:48:41.517861 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5159def5-6856-4180-a872-812fd6d40c0e None admin] PATH_INFO: `/v3/projects/service` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:48:41.525695 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5159def5-6856-4180-a872-812fd6d40c0e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 15:48:41.527225 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5159def5-6856-4180-a872-812fd6d40c0e None admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 15:48:41.538574 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application [None req-5159def5-6856-4180-a872-812fd6d40c0e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 22 15:48:41.538574 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 15:48:41.538574 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 15:48:41.538574 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 15:48:41.538574 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 15:48:41.538574 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 15:48:41.538574 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 15:48:41.538574 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 15:48:41.538574 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 15:48:41.538574 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 15:48:41.538574 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 15:48:41.538574 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 15:48:41.538574 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 22 15:48:41.538574 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 22 15:48:41.538574 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 22 15:48:41.540503 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 22 15:48:41.540503 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 15:48:41.540503 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 15:48:41.540503 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 15:48:41.540503 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 15:48:41.540503 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 15:48:41.540503 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 15:48:41.540503 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 15:48:41.540503 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application with Lock( Aug 22 15:48:41.540503 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 15:48:41.540503 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self._enter() Aug 22 15:48:41.540503 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 15:48:41.540503 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 15:48:41.540503 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 15:48:41.540503 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self.creator() Aug 22 15:48:41.540503 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 15:48:41.540503 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application created_value = creator( Aug 22 15:48:41.541797 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 22 15:48:41.541797 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 22 15:48:41.541797 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 22 15:48:41.541797 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 22 15:48:41.541797 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 22 15:48:41.541797 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 22 15:48:41.541797 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 22 15:48:41.541797 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application Aug 22 15:48:41.541797 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 563/1104] 10.222.0.15 () {56 vars in 1111 bytes} [Mon Aug 22 15:48:41 2022] GET /identity/v3/projects/service => generated 88 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 15:48:41.546717 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecf2bbcf-a262-445e-95da-c2bd10014310 None admin] Authenticating user token {{(pid=81767) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 15:48:41.651775 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecf2bbcf-a262-445e-95da-c2bd10014310 None admin] Validating token access rules against request {{(pid=81767) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 15:48:41.691457 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53522734-327e-49bb-a0d3-2b00cbed6411 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81767) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 15:48:41.692071 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53522734-327e-49bb-a0d3-2b00cbed6411 None admin] REQUEST_METHOD: `GET` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:48:41.692392 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53522734-327e-49bb-a0d3-2b00cbed6411 None admin] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:48:41.692648 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53522734-327e-49bb-a0d3-2b00cbed6411 None admin] PATH_INFO: `/v3/projects` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:48:41.693377 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53522734-327e-49bb-a0d3-2b00cbed6411 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81767) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 15:48:41.693672 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53522734-327e-49bb-a0d3-2b00cbed6411 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 15:48:41.695093 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53522734-327e-49bb-a0d3-2b00cbed6411 None admin] RBAC: Authorization granted {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 15:48:41.708819 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 542/1105] 10.222.0.15 () {56 vars in 1148 bytes} [Mon Aug 22 15:48:41 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 15:48:41.715407 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5159def5-6856-4180-a872-812fd6d40c0e None admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 15:48:41.725572 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=100 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 15:48:41.829539 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5159def5-6856-4180-a872-812fd6d40c0e None admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 15:48:41.876487 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37adf62d-60d2-4d3e-95be-47c64ab4913e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 15:48:41.877161 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37adf62d-60d2-4d3e-95be-47c64ab4913e None admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:48:41.877660 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37adf62d-60d2-4d3e-95be-47c64ab4913e None admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:48:41.878157 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37adf62d-60d2-4d3e-95be-47c64ab4913e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:48:41.878897 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37adf62d-60d2-4d3e-95be-47c64ab4913e None admin] RBAC: Adding query filter params (role.id=806eb8803ece4a1fa1bc2487b4cd7c02, scope.project.id=c4ee0413f9ef4d4fb1f95a660babe090, user.id=e4ef569e41fb46d0b721bb1c7663616f) {{(pid=81768) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 15:48:41.879203 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37adf62d-60d2-4d3e-95be-47c64ab4913e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 15:48:41.880683 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37adf62d-60d2-4d3e-95be-47c64ab4913e None admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 15:48:41.887895 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 564/1106] 10.222.0.15 () {56 vars in 1374 bytes} [Mon Aug 22 15:48:41 2022] GET /identity/v3/role_assignments?user.id=e4ef569e41fb46d0b721bb1c7663616f&scope.project.id=c4ee0413f9ef4d4fb1f95a660babe090&role.id=806eb8803ece4a1fa1bc2487b4cd7c02 => generated 608 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 15:48:43.126582 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68f65cdb-13eb-4588-86bc-75a12a6e33b8 None None] REQUEST_METHOD: `GET` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:48:43.127177 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68f65cdb-13eb-4588-86bc-75a12a6e33b8 None None] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:48:43.127177 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68f65cdb-13eb-4588-86bc-75a12a6e33b8 None None] PATH_INFO: `/` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:48:43.128570 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 543/1107] 10.222.0.15 () {54 vars in 927 bytes} [Mon Aug 22 15:48:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 15:48:43.134576 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2964f408-56c6-4b97-a140-8a40f9d8466c None None] REQUEST_METHOD: `POST` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:48:43.135081 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2964f408-56c6-4b97-a140-8a40f9d8466c None None] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:48:43.135175 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2964f408-56c6-4b97-a140-8a40f9d8466c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:48:43.155432 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.auth.core [None req-2964f408-56c6-4b97-a140-8a40f9d8466c None None] MFA Rules not processed for user `6b2199ae0fb0480aa917ada391f5ec54`. Rule list: `[]` (Enabled: `True`). {{(pid=81768) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 15:48:43.156626 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-2964f408-56c6-4b97-a140-8a40f9d8466c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 15:48:43.224089 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=100 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 15:48:43.229325 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 565/1108] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Aug 22 15:48:43 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 22 15:48:43.495780 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68f65cdb-13eb-4588-86bc-75a12a6e33b8 None None] Authenticating user token {{(pid=81767) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 15:48:43.543349 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG dbcounter [-] [81767] Writing DB stats keystone:SELECT=100 {{(pid=81767) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 15:48:43.622907 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68f65cdb-13eb-4588-86bc-75a12a6e33b8 None None] Validating token access rules against request {{(pid=81767) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 15:48:43.663466 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4439683-4fb9-4ddf-b991-cb311cdea580 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81767) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 15:48:43.664127 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4439683-4fb9-4ddf-b991-cb311cdea580 None admin] REQUEST_METHOD: `GET` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:48:43.664426 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4439683-4fb9-4ddf-b991-cb311cdea580 None admin] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:48:43.664698 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4439683-4fb9-4ddf-b991-cb311cdea580 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:48:43.665928 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4439683-4fb9-4ddf-b991-cb311cdea580 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 15:48:43.667397 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4439683-4fb9-4ddf-b991-cb311cdea580 None admin] RBAC: Authorization granted {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 15:48:43.669022 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 544/1109] 10.222.0.15 () {56 vars in 1108 bytes} [Mon Aug 22 15:48:43 2022] GET /identity/v3/domains/default => generated 199 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 15:48:43.675039 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2964f408-56c6-4b97-a140-8a40f9d8466c None None] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 15:48:43.779952 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2964f408-56c6-4b97-a140-8a40f9d8466c None None] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 15:48:43.816163 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d72a3841-7de1-4f70-b321-473f33e8e318 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 15:48:43.816775 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d72a3841-7de1-4f70-b321-473f33e8e318 None admin] REQUEST_METHOD: `POST` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:48:43.817051 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d72a3841-7de1-4f70-b321-473f33e8e318 None admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:48:43.817319 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d72a3841-7de1-4f70-b321-473f33e8e318 None admin] PATH_INFO: `/v3/users` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:48:43.817974 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d72a3841-7de1-4f70-b321-473f33e8e318 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 15:48:43.819435 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d72a3841-7de1-4f70-b321-473f33e8e318 None admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 15:48:43.941562 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 566/1110] 10.222.0.15 () {60 vars in 1132 bytes} [Mon Aug 22 15:48:43 2022] POST /identity/v3/users => generated 286 bytes in 267 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 22 15:48:45.905896 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dd3ab34-d21a-4cb1-9eb5-2951fab6e90f None None] REQUEST_METHOD: `GET` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:48:45.906479 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dd3ab34-d21a-4cb1-9eb5-2951fab6e90f None None] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:48:45.906479 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dd3ab34-d21a-4cb1-9eb5-2951fab6e90f None None] PATH_INFO: `/` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:48:45.907794 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 545/1111] 10.222.0.15 () {54 vars in 927 bytes} [Mon Aug 22 15:48:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 15:48:45.913004 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c7749e5-8802-49d5-8b34-d1cd0fee60ab None None] REQUEST_METHOD: `POST` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:48:45.913425 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c7749e5-8802-49d5-8b34-d1cd0fee60ab None None] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:48:45.913541 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c7749e5-8802-49d5-8b34-d1cd0fee60ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:48:45.933885 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.auth.core [None req-0c7749e5-8802-49d5-8b34-d1cd0fee60ab None None] MFA Rules not processed for user `6b2199ae0fb0480aa917ada391f5ec54`. Rule list: `[]` (Enabled: `True`). {{(pid=81768) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 15:48:45.935086 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-0c7749e5-8802-49d5-8b34-d1cd0fee60ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 15:48:45.987000 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 15:48:46.003764 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 567/1112] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Aug 22 15:48:45 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 22 15:48:46.275598 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dd3ab34-d21a-4cb1-9eb5-2951fab6e90f None None] Authenticating user token {{(pid=81767) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 15:48:46.387131 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dd3ab34-d21a-4cb1-9eb5-2951fab6e90f None None] Validating token access rules against request {{(pid=81767) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 15:48:46.389532 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG dbcounter [-] [81767] Writing DB stats keystone:SELECT=100 {{(pid=81767) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 15:48:46.422278 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f866e10e-c081-4a53-aa37-5cebd61c547f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81767) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 15:48:46.422907 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f866e10e-c081-4a53-aa37-5cebd61c547f None admin] REQUEST_METHOD: `GET` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:48:46.423185 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f866e10e-c081-4a53-aa37-5cebd61c547f None admin] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:48:46.423490 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f866e10e-c081-4a53-aa37-5cebd61c547f None admin] PATH_INFO: `/v3/roles/1ce6b924c9604048906c07dbfb126da9` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:48:46.424884 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f866e10e-c081-4a53-aa37-5cebd61c547f None admin] RBAC: Authorizing `identity:get_role(role_id=1ce6b924c9604048906c07dbfb126da9)` {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 15:48:46.426240 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f866e10e-c081-4a53-aa37-5cebd61c547f None admin] RBAC: Authorization granted {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 15:48:46.427618 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 546/1113] 10.222.0.15 () {56 vars in 1177 bytes} [Mon Aug 22 15:48:46 2022] GET /identity/v3/roles/1ce6b924c9604048906c07dbfb126da9 => generated 223 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 15:48:46.434548 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c7749e5-8802-49d5-8b34-d1cd0fee60ab None None] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 15:48:46.535256 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c7749e5-8802-49d5-8b34-d1cd0fee60ab None None] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 15:48:46.570780 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30535be8-e6b1-4525-b47c-ced5e1dd7749 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 15:48:46.571485 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30535be8-e6b1-4525-b47c-ced5e1dd7749 None admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:48:46.571766 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30535be8-e6b1-4525-b47c-ced5e1dd7749 None admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:48:46.572039 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30535be8-e6b1-4525-b47c-ced5e1dd7749 None admin] PATH_INFO: `/v3/users/2fcb2940519c450abb13a094244271e7` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:48:46.585564 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30535be8-e6b1-4525-b47c-ced5e1dd7749 None admin] RBAC: Authorizing `identity:get_user(user_id=2fcb2940519c450abb13a094244271e7)` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 15:48:46.587123 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30535be8-e6b1-4525-b47c-ced5e1dd7749 None admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 15:48:46.588836 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 568/1114] 10.222.0.15 () {56 vars in 1177 bytes} [Mon Aug 22 15:48:46 2022] GET /identity/v3/users/2fcb2940519c450abb13a094244271e7 => generated 286 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 15:48:46.594819 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f866e10e-c081-4a53-aa37-5cebd61c547f None admin] Authenticating user token {{(pid=81767) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 15:48:46.697702 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f866e10e-c081-4a53-aa37-5cebd61c547f None admin] Validating token access rules against request {{(pid=81767) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 15:48:46.733496 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8504123-f458-49dd-8408-c95683d184a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81767) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 15:48:46.734099 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8504123-f458-49dd-8408-c95683d184a7 None admin] REQUEST_METHOD: `GET` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:48:46.734378 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8504123-f458-49dd-8408-c95683d184a7 None admin] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:48:46.734648 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8504123-f458-49dd-8408-c95683d184a7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:48:46.777655 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG dbcounter [-] [81767] Writing DB stats keystone:SELECT=100 {{(pid=81767) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 15:48:46.811237 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8504123-f458-49dd-8408-c95683d184a7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 15:48:46.812660 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8504123-f458-49dd-8408-c95683d184a7 None admin] RBAC: Authorization granted {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 15:48:46.914469 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 547/1115] 10.222.0.15 () {58 vars in 1282 bytes} [Mon Aug 22 15:48:46 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 320 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 22 15:48:46.920469 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30535be8-e6b1-4525-b47c-ced5e1dd7749 None admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 15:48:46.966959 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=100 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 15:48:47.022823 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30535be8-e6b1-4525-b47c-ced5e1dd7749 None admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 15:48:47.062101 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-698483d0-d88d-421a-adca-4736ee8c7ea1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 15:48:47.062638 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-698483d0-d88d-421a-adca-4736ee8c7ea1 None admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:48:47.062912 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-698483d0-d88d-421a-adca-4736ee8c7ea1 None admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:48:47.063179 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-698483d0-d88d-421a-adca-4736ee8c7ea1 None admin] PATH_INFO: `/v3/projects/2b65932e1345451c880e9d46bd66f7f3` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:48:47.064311 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-698483d0-d88d-421a-adca-4736ee8c7ea1 None admin] RBAC: Authorizing `identity:get_project(project_id=2b65932e1345451c880e9d46bd66f7f3)` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 15:48:47.065741 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-698483d0-d88d-421a-adca-4736ee8c7ea1 None admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 15:48:47.067386 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 569/1116] 10.222.0.15 () {56 vars in 1186 bytes} [Mon Aug 22 15:48:46 2022] GET /identity/v3/projects/2b65932e1345451c880e9d46bd66f7f3 => generated 311 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 15:48:47.074347 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8504123-f458-49dd-8408-c95683d184a7 None admin] Authenticating user token {{(pid=81767) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 15:48:47.146375 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG dbcounter [-] [81767] Writing DB stats keystone:SELECT=100 {{(pid=81767) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 15:48:47.175576 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8504123-f458-49dd-8408-c95683d184a7 None admin] Validating token access rules against request {{(pid=81767) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 15:48:47.211906 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce495881-4fba-4b5a-937e-dec7eb6e84b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81767) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 15:48:47.212503 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce495881-4fba-4b5a-937e-dec7eb6e84b9 None admin] REQUEST_METHOD: `GET` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:48:47.212777 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce495881-4fba-4b5a-937e-dec7eb6e84b9 None admin] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:48:47.213042 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce495881-4fba-4b5a-937e-dec7eb6e84b9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:48:47.213764 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce495881-4fba-4b5a-937e-dec7eb6e84b9 None admin] RBAC: Adding query filter params (role.id=1ce6b924c9604048906c07dbfb126da9, scope.project.id=2b65932e1345451c880e9d46bd66f7f3, user.id=2fcb2940519c450abb13a094244271e7) {{(pid=81767) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 15:48:47.214063 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce495881-4fba-4b5a-937e-dec7eb6e84b9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 15:48:47.215514 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce495881-4fba-4b5a-937e-dec7eb6e84b9 None admin] RBAC: Authorization granted {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 15:48:47.220814 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 548/1117] 10.222.0.15 () {56 vars in 1374 bytes} [Mon Aug 22 15:48:47 2022] GET /identity/v3/role_assignments?user.id=2fcb2940519c450abb13a094244271e7&scope.project.id=2b65932e1345451c880e9d46bd66f7f3&role.id=1ce6b924c9604048906c07dbfb126da9 => generated 259 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 15:48:49.122352 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f5d8773-c730-4419-ba01-eb69700db06b None None] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:48:49.122890 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f5d8773-c730-4419-ba01-eb69700db06b None None] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:48:49.122890 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f5d8773-c730-4419-ba01-eb69700db06b None None] PATH_INFO: `/` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:48:49.124293 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 570/1118] 10.222.0.15 () {54 vars in 927 bytes} [Mon Aug 22 15:48:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 15:48:49.129550 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75fadbcd-5c65-4ae3-9d72-fbe64af7f226 None None] REQUEST_METHOD: `POST` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:48:49.130038 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75fadbcd-5c65-4ae3-9d72-fbe64af7f226 None None] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:48:49.130128 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75fadbcd-5c65-4ae3-9d72-fbe64af7f226 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:48:49.150806 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.auth.core [None req-75fadbcd-5c65-4ae3-9d72-fbe64af7f226 None None] MFA Rules not processed for user `6b2199ae0fb0480aa917ada391f5ec54`. Rule list: `[]` (Enabled: `True`). {{(pid=81767) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 15:48:49.152051 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.fernet_utils [None req-75fadbcd-5c65-4ae3-9d72-fbe64af7f226 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81767) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 15:48:49.221896 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 549/1119] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Aug 22 15:48:49 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 22 15:48:49.489798 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f5d8773-c730-4419-ba01-eb69700db06b None None] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 15:48:49.597722 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f5d8773-c730-4419-ba01-eb69700db06b None None] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 15:48:49.612846 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=100 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 15:48:49.635183 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46b05e7c-1ace-49ce-b4c9-110c791a089a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 15:48:49.635894 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b05e7c-1ace-49ce-b4c9-110c791a089a None admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:48:49.636125 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b05e7c-1ace-49ce-b4c9-110c791a089a None admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:48:49.636394 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b05e7c-1ace-49ce-b4c9-110c791a089a None admin] PATH_INFO: `/v3/roles/1ce6b924c9604048906c07dbfb126da9` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:48:49.637838 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46b05e7c-1ace-49ce-b4c9-110c791a089a None admin] RBAC: Authorizing `identity:get_role(role_id=1ce6b924c9604048906c07dbfb126da9)` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 15:48:49.639215 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46b05e7c-1ace-49ce-b4c9-110c791a089a None admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 15:48:49.640572 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 571/1120] 10.222.0.15 () {56 vars in 1177 bytes} [Mon Aug 22 15:48:49 2022] GET /identity/v3/roles/1ce6b924c9604048906c07dbfb126da9 => generated 223 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 15:48:49.646681 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75fadbcd-5c65-4ae3-9d72-fbe64af7f226 None None] Authenticating user token {{(pid=81767) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 15:48:49.715706 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG dbcounter [-] [81767] Writing DB stats keystone:SELECT=100 {{(pid=81767) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 15:48:49.751375 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75fadbcd-5c65-4ae3-9d72-fbe64af7f226 None None] Validating token access rules against request {{(pid=81767) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 15:48:49.786115 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdd7177c-0bee-4309-a357-615712f11d38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81767) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 15:48:49.786689 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdd7177c-0bee-4309-a357-615712f11d38 None admin] REQUEST_METHOD: `GET` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:48:49.786963 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdd7177c-0bee-4309-a357-615712f11d38 None admin] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:48:49.787219 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdd7177c-0bee-4309-a357-615712f11d38 None admin] PATH_INFO: `/v3/users/2fcb2940519c450abb13a094244271e7` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:48:49.788452 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdd7177c-0bee-4309-a357-615712f11d38 None admin] RBAC: Authorizing `identity:get_user(user_id=2fcb2940519c450abb13a094244271e7)` {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 15:48:49.790416 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdd7177c-0bee-4309-a357-615712f11d38 None admin] RBAC: Authorization granted {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 15:48:49.792050 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 550/1121] 10.222.0.15 () {56 vars in 1177 bytes} [Mon Aug 22 15:48:49 2022] GET /identity/v3/users/2fcb2940519c450abb13a094244271e7 => generated 286 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 15:48:49.798481 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46b05e7c-1ace-49ce-b4c9-110c791a089a None admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 15:48:49.897389 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46b05e7c-1ace-49ce-b4c9-110c791a089a None admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 15:48:49.933267 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05589dcb-b40c-4bdd-9765-d217b5c9a65e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 15:48:49.933894 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05589dcb-b40c-4bdd-9765-d217b5c9a65e None admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:48:49.934167 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05589dcb-b40c-4bdd-9765-d217b5c9a65e None admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:48:49.934434 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05589dcb-b40c-4bdd-9765-d217b5c9a65e None admin] PATH_INFO: `/v3/projects/2b65932e1345451c880e9d46bd66f7f3` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:48:49.935572 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05589dcb-b40c-4bdd-9765-d217b5c9a65e None admin] RBAC: Authorizing `identity:get_project(project_id=2b65932e1345451c880e9d46bd66f7f3)` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 15:48:49.937038 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05589dcb-b40c-4bdd-9765-d217b5c9a65e None admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 15:48:49.938706 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 572/1122] 10.222.0.15 () {56 vars in 1186 bytes} [Mon Aug 22 15:48:49 2022] GET /identity/v3/projects/2b65932e1345451c880e9d46bd66f7f3 => generated 311 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 22 15:48:49.944891 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdd7177c-0bee-4309-a357-615712f11d38 None admin] Authenticating user token {{(pid=81767) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 15:48:50.061580 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdd7177c-0bee-4309-a357-615712f11d38 None admin] Validating token access rules against request {{(pid=81767) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 15:48:50.099112 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e2eaef4-3dbe-4e10-bf81-5ec0bb1f5b3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81767) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 15:48:50.099812 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e2eaef4-3dbe-4e10-bf81-5ec0bb1f5b3f None admin] REQUEST_METHOD: `PUT` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:48:50.100084 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e2eaef4-3dbe-4e10-bf81-5ec0bb1f5b3f None admin] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:48:50.100345 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e2eaef4-3dbe-4e10-bf81-5ec0bb1f5b3f None admin] PATH_INFO: `/v3/projects/2b65932e1345451c880e9d46bd66f7f3/users/2fcb2940519c450abb13a094244271e7/roles/1ce6b924c9604048906c07dbfb126da9` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:48:50.102606 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e2eaef4-3dbe-4e10-bf81-5ec0bb1f5b3f None admin] RBAC: Authorizing `identity:create_grant(project_id=2b65932e1345451c880e9d46bd66f7f3, user_id=2fcb2940519c450abb13a094244271e7, role_id=1ce6b924c9604048906c07dbfb126da9)` {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 15:48:50.104083 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e2eaef4-3dbe-4e10-bf81-5ec0bb1f5b3f None admin] RBAC: Authorization granted {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 15:48:50.106544 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG dbcounter [-] [81767] Writing DB stats keystone:SELECT=100 {{(pid=81767) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 15:48:50.229428 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 551/1123] 10.222.0.15 () {58 vars in 1439 bytes} [Mon Aug 22 15:48:49 2022] PUT /identity/v3/projects/2b65932e1345451c880e9d46bd66f7f3/users/2fcb2940519c450abb13a094244271e7/roles/1ce6b924c9604048906c07dbfb126da9 => generated 0 bytes in 285 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 15:48:53.392890 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed95a8af-52a0-4bfc-a6e2-b2fdddaef066 None None] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:48:53.393685 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed95a8af-52a0-4bfc-a6e2-b2fdddaef066 None None] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:48:53.393864 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed95a8af-52a0-4bfc-a6e2-b2fdddaef066 None None] PATH_INFO: `/` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:48:53.395429 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 573/1124] 10.222.0.15 () {54 vars in 927 bytes} [Mon Aug 22 15:48:53 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 15:48:53.404463 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e398b3c-8a21-47b3-ae98-898370d8b34b None None] REQUEST_METHOD: `POST` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:48:53.405011 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e398b3c-8a21-47b3-ae98-898370d8b34b None None] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:48:53.405293 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e398b3c-8a21-47b3-ae98-898370d8b34b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:48:53.430004 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.auth.core [None req-0e398b3c-8a21-47b3-ae98-898370d8b34b None None] MFA Rules not processed for user `6b2199ae0fb0480aa917ada391f5ec54`. Rule list: `[]` (Enabled: `True`). {{(pid=81767) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 15:48:53.431442 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.fernet_utils [None req-0e398b3c-8a21-47b3-ae98-898370d8b34b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81767) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 15:48:53.513714 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 552/1125] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Aug 22 15:48:53 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 22 15:48:53.866986 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed95a8af-52a0-4bfc-a6e2-b2fdddaef066 None None] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 15:48:53.943392 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=100 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 15:48:54.024737 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed95a8af-52a0-4bfc-a6e2-b2fdddaef066 None None] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 15:48:54.079258 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-725ff339-b252-4e50-b4f4-68a2a75ebbdb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 15:48:54.080242 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-725ff339-b252-4e50-b4f4-68a2a75ebbdb None admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:48:54.080676 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-725ff339-b252-4e50-b4f4-68a2a75ebbdb None admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:48:54.081105 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-725ff339-b252-4e50-b4f4-68a2a75ebbdb None admin] PATH_INFO: `/v3/roles/1ce6b924c9604048906c07dbfb126da9` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:48:54.083338 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-725ff339-b252-4e50-b4f4-68a2a75ebbdb None admin] RBAC: Authorizing `identity:get_role(role_id=1ce6b924c9604048906c07dbfb126da9)` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 15:48:54.085522 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-725ff339-b252-4e50-b4f4-68a2a75ebbdb None admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 15:48:54.088325 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 574/1126] 10.222.0.15 () {56 vars in 1177 bytes} [Mon Aug 22 15:48:53 2022] GET /identity/v3/roles/1ce6b924c9604048906c07dbfb126da9 => generated 223 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 22 15:48:54.098284 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e398b3c-8a21-47b3-ae98-898370d8b34b None None] Authenticating user token {{(pid=81767) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 15:48:54.217904 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e398b3c-8a21-47b3-ae98-898370d8b34b None None] Validating token access rules against request {{(pid=81767) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 15:48:54.245024 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG dbcounter [-] [81767] Writing DB stats keystone:INSERT=1,keystone:SELECT=99 {{(pid=81767) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 15:48:54.259942 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfb0952a-dc2a-4794-9517-ca2fa83aa8a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81767) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 15:48:54.260695 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb0952a-dc2a-4794-9517-ca2fa83aa8a3 None admin] REQUEST_METHOD: `GET` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:48:54.261069 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb0952a-dc2a-4794-9517-ca2fa83aa8a3 None admin] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:48:54.261434 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb0952a-dc2a-4794-9517-ca2fa83aa8a3 None admin] PATH_INFO: `/v3/users/2fcb2940519c450abb13a094244271e7` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:48:54.262674 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfb0952a-dc2a-4794-9517-ca2fa83aa8a3 None admin] RBAC: Authorizing `identity:get_user(user_id=2fcb2940519c450abb13a094244271e7)` {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 15:48:54.264154 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfb0952a-dc2a-4794-9517-ca2fa83aa8a3 None admin] RBAC: Authorization granted {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 15:48:54.265780 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 553/1127] 10.222.0.15 () {56 vars in 1177 bytes} [Mon Aug 22 15:48:54 2022] GET /identity/v3/users/2fcb2940519c450abb13a094244271e7 => generated 286 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 15:48:54.272291 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-725ff339-b252-4e50-b4f4-68a2a75ebbdb None admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 15:48:54.398392 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-725ff339-b252-4e50-b4f4-68a2a75ebbdb None admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 15:48:54.420612 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=100 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 15:48:54.441405 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a853ada-460d-4745-b045-52dcba0e591b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 15:48:54.442025 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a853ada-460d-4745-b045-52dcba0e591b None admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:48:54.442425 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a853ada-460d-4745-b045-52dcba0e591b None admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:48:54.442752 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a853ada-460d-4745-b045-52dcba0e591b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:48:54.562929 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a853ada-460d-4745-b045-52dcba0e591b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 15:48:54.564626 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a853ada-460d-4745-b045-52dcba0e591b None admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 15:48:54.684698 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 575/1128] 10.222.0.15 () {58 vars in 1282 bytes} [Mon Aug 22 15:48:54 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 413 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 22 15:48:54.691697 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfb0952a-dc2a-4794-9517-ca2fa83aa8a3 None admin] Authenticating user token {{(pid=81767) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 15:48:54.836960 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfb0952a-dc2a-4794-9517-ca2fa83aa8a3 None admin] Validating token access rules against request {{(pid=81767) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 15:48:54.881708 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b5adce0-fc1b-46d1-b330-4d7be0333b05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81767) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 15:48:54.882418 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b5adce0-fc1b-46d1-b330-4d7be0333b05 None admin] REQUEST_METHOD: `GET` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:48:54.882747 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b5adce0-fc1b-46d1-b330-4d7be0333b05 None admin] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:48:54.883066 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b5adce0-fc1b-46d1-b330-4d7be0333b05 None admin] PATH_INFO: `/v3/projects/2b65932e1345451c880e9d46bd66f7f3` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:48:54.884360 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b5adce0-fc1b-46d1-b330-4d7be0333b05 None admin] RBAC: Authorizing `identity:get_project(project_id=2b65932e1345451c880e9d46bd66f7f3)` {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 15:48:54.885842 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b5adce0-fc1b-46d1-b330-4d7be0333b05 None admin] RBAC: Authorization granted {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 15:48:54.887625 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 554/1129] 10.222.0.15 () {56 vars in 1186 bytes} [Mon Aug 22 15:48:54 2022] GET /identity/v3/projects/2b65932e1345451c880e9d46bd66f7f3 => generated 311 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 15:48:54.894909 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a853ada-460d-4745-b045-52dcba0e591b None admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 15:48:54.917491 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=100 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 15:48:55.050823 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a853ada-460d-4745-b045-52dcba0e591b None admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 15:48:55.097735 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72e70f4e-0fa4-4e4c-a902-15b9b23f678b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 15:48:55.098314 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e70f4e-0fa4-4e4c-a902-15b9b23f678b None admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:48:55.098636 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e70f4e-0fa4-4e4c-a902-15b9b23f678b None admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:48:55.098950 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e70f4e-0fa4-4e4c-a902-15b9b23f678b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:48:55.099911 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72e70f4e-0fa4-4e4c-a902-15b9b23f678b None admin] RBAC: Adding query filter params (role.id=1ce6b924c9604048906c07dbfb126da9, scope.project.id=2b65932e1345451c880e9d46bd66f7f3, user.id=2fcb2940519c450abb13a094244271e7) {{(pid=81768) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 15:48:55.100351 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72e70f4e-0fa4-4e4c-a902-15b9b23f678b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 15:48:55.102460 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72e70f4e-0fa4-4e4c-a902-15b9b23f678b None admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 15:48:55.109272 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 576/1130] 10.222.0.15 () {56 vars in 1374 bytes} [Mon Aug 22 15:48:54 2022] GET /identity/v3/role_assignments?user.id=2fcb2940519c450abb13a094244271e7&scope.project.id=2b65932e1345451c880e9d46bd66f7f3&role.id=1ce6b924c9604048906c07dbfb126da9 => generated 608 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 15:48:57.180787 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24fb0167-8a89-4cf8-9b18-96ae234ff79f None None] REQUEST_METHOD: `GET` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:48:57.181538 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24fb0167-8a89-4cf8-9b18-96ae234ff79f None None] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:48:57.181538 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24fb0167-8a89-4cf8-9b18-96ae234ff79f None None] PATH_INFO: `/` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:48:57.182840 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 555/1131] 10.222.0.15 () {54 vars in 927 bytes} [Mon Aug 22 15:48:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 15:48:57.189838 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5b148cf-9349-46c6-bfdb-7de61fd65a5f None None] REQUEST_METHOD: `POST` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:48:57.190319 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5b148cf-9349-46c6-bfdb-7de61fd65a5f None None] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:48:57.190787 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5b148cf-9349-46c6-bfdb-7de61fd65a5f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:48:57.221904 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.auth.core [None req-d5b148cf-9349-46c6-bfdb-7de61fd65a5f None None] MFA Rules not processed for user `6b2199ae0fb0480aa917ada391f5ec54`. Rule list: `[]` (Enabled: `True`). {{(pid=81768) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 15:48:57.223945 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-d5b148cf-9349-46c6-bfdb-7de61fd65a5f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 15:48:57.342648 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 577/1132] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Aug 22 15:48:57 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 22 15:48:57.826611 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24fb0167-8a89-4cf8-9b18-96ae234ff79f None None] Authenticating user token {{(pid=81767) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 15:48:57.911196 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG dbcounter [-] [81767] Writing DB stats keystone:SELECT=100 {{(pid=81767) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 15:48:57.968318 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24fb0167-8a89-4cf8-9b18-96ae234ff79f None None] Validating token access rules against request {{(pid=81767) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 15:48:58.012477 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bdabfbc-aa0d-42d6-a9c8-c8cf9bada732 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81767) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 15:48:58.013170 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bdabfbc-aa0d-42d6-a9c8-c8cf9bada732 None admin] REQUEST_METHOD: `GET` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:48:58.013497 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bdabfbc-aa0d-42d6-a9c8-c8cf9bada732 None admin] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:48:58.013815 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bdabfbc-aa0d-42d6-a9c8-c8cf9bada732 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:48:58.015086 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bdabfbc-aa0d-42d6-a9c8-c8cf9bada732 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 15:48:58.016642 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bdabfbc-aa0d-42d6-a9c8-c8cf9bada732 None admin] RBAC: Authorization granted {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 15:48:58.018358 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 556/1133] 10.222.0.15 () {56 vars in 1108 bytes} [Mon Aug 22 15:48:57 2022] GET /identity/v3/domains/default => generated 199 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 15:48:58.025191 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5b148cf-9349-46c6-bfdb-7de61fd65a5f None None] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 15:48:58.041479 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=100 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 15:48:58.153245 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5b148cf-9349-46c6-bfdb-7de61fd65a5f None None] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 15:48:58.190684 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbfed47b-0349-4532-ae21-61b551a8c566 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 15:48:58.191408 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbfed47b-0349-4532-ae21-61b551a8c566 None admin] REQUEST_METHOD: `POST` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:48:58.191728 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbfed47b-0349-4532-ae21-61b551a8c566 None admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:48:58.192038 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbfed47b-0349-4532-ae21-61b551a8c566 None admin] PATH_INFO: `/v3/projects` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:48:58.192724 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbfed47b-0349-4532-ae21-61b551a8c566 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 15:48:58.194523 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbfed47b-0349-4532-ae21-61b551a8c566 None admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 15:48:58.340055 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 578/1134] 10.222.0.15 () {60 vars in 1141 bytes} [Mon Aug 22 15:48:58 2022] POST /identity/v3/projects => generated 311 bytes in 316 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 22 15:49:00.369362 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-531d684b-93fa-4fd6-b22f-b0f646409d24 None None] REQUEST_METHOD: `GET` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:49:00.369895 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-531d684b-93fa-4fd6-b22f-b0f646409d24 None None] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:49:00.370230 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-531d684b-93fa-4fd6-b22f-b0f646409d24 None None] PATH_INFO: `/` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:49:00.372360 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 557/1135] 10.222.0.15 () {54 vars in 927 bytes} [Mon Aug 22 15:49:00 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 15:49:00.378044 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c70e6484-b482-4460-84b1-2fdd1e4cd833 None None] REQUEST_METHOD: `POST` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:49:00.378472 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c70e6484-b482-4460-84b1-2fdd1e4cd833 None None] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:49:00.378604 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c70e6484-b482-4460-84b1-2fdd1e4cd833 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:49:00.398771 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.auth.core [None req-c70e6484-b482-4460-84b1-2fdd1e4cd833 None None] MFA Rules not processed for user `6b2199ae0fb0480aa917ada391f5ec54`. Rule list: `[]` (Enabled: `True`). {{(pid=81768) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 15:49:00.400091 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-c70e6484-b482-4460-84b1-2fdd1e4cd833 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 15:49:00.464766 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 15:49:00.475483 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 579/1136] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Aug 22 15:49:00 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 22 15:49:00.754511 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-531d684b-93fa-4fd6-b22f-b0f646409d24 None None] Authenticating user token {{(pid=81767) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 15:49:00.863115 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-531d684b-93fa-4fd6-b22f-b0f646409d24 None None] Validating token access rules against request {{(pid=81767) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 15:49:00.891976 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG dbcounter [-] [81767] Writing DB stats keystone:SELECT=100 {{(pid=81767) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 15:49:00.900834 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e8ab534-780b-49a1-9340-9f450a7e5661 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81767) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 15:49:00.901570 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e8ab534-780b-49a1-9340-9f450a7e5661 None admin] REQUEST_METHOD: `GET` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:49:00.901943 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e8ab534-780b-49a1-9340-9f450a7e5661 None admin] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:49:00.902302 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e8ab534-780b-49a1-9340-9f450a7e5661 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:49:00.903622 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e8ab534-780b-49a1-9340-9f450a7e5661 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 15:49:00.905130 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e8ab534-780b-49a1-9340-9f450a7e5661 None admin] RBAC: Authorization granted {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 15:49:00.906917 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 558/1137] 10.222.0.15 () {56 vars in 1108 bytes} [Mon Aug 22 15:49:00 2022] GET /identity/v3/domains/default => generated 199 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 22 15:49:00.915600 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c70e6484-b482-4460-84b1-2fdd1e4cd833 None None] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 15:49:01.031849 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c70e6484-b482-4460-84b1-2fdd1e4cd833 None None] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 15:49:01.070333 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c1a7f49-b98f-4c72-8da5-dba52bea6ca6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 15:49:01.071026 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c1a7f49-b98f-4c72-8da5-dba52bea6ca6 None admin] REQUEST_METHOD: `POST` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:49:01.071372 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c1a7f49-b98f-4c72-8da5-dba52bea6ca6 None admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:49:01.071692 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c1a7f49-b98f-4c72-8da5-dba52bea6ca6 None admin] PATH_INFO: `/v3/users` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:49:01.072392 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c1a7f49-b98f-4c72-8da5-dba52bea6ca6 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 15:49:01.073927 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c1a7f49-b98f-4c72-8da5-dba52bea6ca6 None admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 15:49:01.221484 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 580/1138] 10.222.0.15 () {60 vars in 1132 bytes} [Mon Aug 22 15:49:00 2022] POST /identity/v3/users => generated 286 bytes in 306 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 22 15:49:03.222389 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55a93519-257e-475e-afee-c88d06cd9211 None None] REQUEST_METHOD: `GET` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:49:03.222968 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55a93519-257e-475e-afee-c88d06cd9211 None None] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:49:03.222968 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55a93519-257e-475e-afee-c88d06cd9211 None None] PATH_INFO: `/` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:49:03.224304 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 559/1139] 10.222.0.15 () {54 vars in 927 bytes} [Mon Aug 22 15:49:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 15:49:03.230177 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc8a09d-1036-48a5-8c1f-e9432e4cf5d7 None None] REQUEST_METHOD: `POST` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:49:03.230664 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc8a09d-1036-48a5-8c1f-e9432e4cf5d7 None None] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:49:03.230762 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc8a09d-1036-48a5-8c1f-e9432e4cf5d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:49:03.249375 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.auth.core [None req-ecc8a09d-1036-48a5-8c1f-e9432e4cf5d7 None None] MFA Rules not processed for user `6b2199ae0fb0480aa917ada391f5ec54`. Rule list: `[]` (Enabled: `True`). {{(pid=81768) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 15:49:03.250594 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-ecc8a09d-1036-48a5-8c1f-e9432e4cf5d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 15:49:03.297016 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 15:49:03.323649 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 581/1140] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Aug 22 15:49:03 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 22 15:49:03.598232 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55a93519-257e-475e-afee-c88d06cd9211 None None] Authenticating user token {{(pid=81767) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 15:49:03.711405 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55a93519-257e-475e-afee-c88d06cd9211 None None] Validating token access rules against request {{(pid=81767) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 15:49:03.746974 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5233d72e-e061-4add-9879-ad9e24c9d0af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81767) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 15:49:03.747617 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5233d72e-e061-4add-9879-ad9e24c9d0af None admin] REQUEST_METHOD: `GET` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:49:03.747893 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5233d72e-e061-4add-9879-ad9e24c9d0af None admin] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:49:03.748161 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5233d72e-e061-4add-9879-ad9e24c9d0af None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:49:03.758495 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5233d72e-e061-4add-9879-ad9e24c9d0af None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 22 15:49:03.763223 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application [None req-5233d72e-e061-4add-9879-ad9e24c9d0af None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 22 15:49:03.763223 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 15:49:03.763223 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 15:49:03.763223 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 15:49:03.763223 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 15:49:03.763223 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 15:49:03.763223 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 15:49:03.763223 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 15:49:03.763223 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 15:49:03.763223 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 15:49:03.763223 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 15:49:03.763223 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 15:49:03.763223 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 22 15:49:03.763223 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 22 15:49:03.763223 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 22 15:49:03.763223 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 22 15:49:03.764585 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 22 15:49:03.764585 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 22 15:49:03.764585 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 22 15:49:03.764585 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 22 15:49:03.764585 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 15:49:03.764585 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 15:49:03.764585 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 15:49:03.764585 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 15:49:03.764585 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 15:49:03.764585 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 15:49:03.764585 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 15:49:03.764585 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application with Lock( Aug 22 15:49:03.764585 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 15:49:03.764585 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return self._enter() Aug 22 15:49:03.764585 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 15:49:03.764585 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 15:49:03.765501 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 15:49:03.765501 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return self.creator() Aug 22 15:49:03.765501 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 15:49:03.765501 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application created_value = creator( Aug 22 15:49:03.765501 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 22 15:49:03.765501 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 22 15:49:03.765501 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 22 15:49:03.765501 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 22 15:49:03.765501 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 22 15:49:03.765501 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 22 15:49:03.765501 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 22 15:49:03.765501 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application Aug 22 15:49:03.765501 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 560/1141] 10.222.0.15 () {56 vars in 1096 bytes} [Mon Aug 22 15:49:03 2022] GET /identity/v3/roles/admin => generated 83 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 15:49:03.770524 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecc8a09d-1036-48a5-8c1f-e9432e4cf5d7 None None] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 15:49:03.880369 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecc8a09d-1036-48a5-8c1f-e9432e4cf5d7 None None] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 15:49:03.921685 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9e1f303-ca86-4ce2-878e-ba994ff33873 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 15:49:03.922379 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9e1f303-ca86-4ce2-878e-ba994ff33873 None admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:49:03.922659 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9e1f303-ca86-4ce2-878e-ba994ff33873 None admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:49:03.922930 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9e1f303-ca86-4ce2-878e-ba994ff33873 None admin] PATH_INFO: `/v3/roles` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:49:03.923739 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9e1f303-ca86-4ce2-878e-ba994ff33873 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81768) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 15:49:03.924073 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9e1f303-ca86-4ce2-878e-ba994ff33873 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 15:49:03.925503 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9e1f303-ca86-4ce2-878e-ba994ff33873 None admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 15:49:03.935602 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 582/1142] 10.222.0.15 () {56 vars in 1099 bytes} [Mon Aug 22 15:49:03 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 22 15:49:03.941643 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5233d72e-e061-4add-9879-ad9e24c9d0af None admin] Authenticating user token {{(pid=81767) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 15:49:04.131030 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG dbcounter [-] [81767] Writing DB stats keystone:SELECT=100 {{(pid=81767) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 15:49:04.177228 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5233d72e-e061-4add-9879-ad9e24c9d0af None admin] Validating token access rules against request {{(pid=81767) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 15:49:04.214769 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3748a6a-2eb3-48d0-bc3e-9f30011d6249 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81767) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 15:49:04.215437 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3748a6a-2eb3-48d0-bc3e-9f30011d6249 None admin] REQUEST_METHOD: `GET` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:49:04.215708 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3748a6a-2eb3-48d0-bc3e-9f30011d6249 None admin] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:49:04.215970 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3748a6a-2eb3-48d0-bc3e-9f30011d6249 None admin] PATH_INFO: `/v3/users/0ca63572bbe34306a150a6d2dae70c28` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:49:04.230958 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3748a6a-2eb3-48d0-bc3e-9f30011d6249 None admin] RBAC: Authorizing `identity:get_user(user_id=0ca63572bbe34306a150a6d2dae70c28)` {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 15:49:04.232479 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3748a6a-2eb3-48d0-bc3e-9f30011d6249 None admin] RBAC: Authorization granted {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 15:49:04.234133 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 561/1143] 10.222.0.15 () {56 vars in 1177 bytes} [Mon Aug 22 15:49:03 2022] GET /identity/v3/users/0ca63572bbe34306a150a6d2dae70c28 => generated 286 bytes in 293 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 22 15:49:04.240422 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9e1f303-ca86-4ce2-878e-ba994ff33873 None admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 15:49:04.286705 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=100 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 15:49:04.347775 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9e1f303-ca86-4ce2-878e-ba994ff33873 None admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 15:49:04.383983 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26b144e6-80ac-4137-b9ac-02a3a244fe55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 15:49:04.384497 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26b144e6-80ac-4137-b9ac-02a3a244fe55 None admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:49:04.384769 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26b144e6-80ac-4137-b9ac-02a3a244fe55 None admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:49:04.385031 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26b144e6-80ac-4137-b9ac-02a3a244fe55 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:49:04.473153 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26b144e6-80ac-4137-b9ac-02a3a244fe55 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 15:49:04.474747 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26b144e6-80ac-4137-b9ac-02a3a244fe55 None admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 15:49:04.521208 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=100 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 15:49:04.597562 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 583/1144] 10.222.0.15 () {58 vars in 1282 bytes} [Mon Aug 22 15:49:04 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 358 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 22 15:49:04.603844 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3748a6a-2eb3-48d0-bc3e-9f30011d6249 None admin] Authenticating user token {{(pid=81767) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 15:49:04.711512 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3748a6a-2eb3-48d0-bc3e-9f30011d6249 None admin] Validating token access rules against request {{(pid=81767) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 15:49:04.726683 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG dbcounter [-] [81767] Writing DB stats keystone:SELECT=100 {{(pid=81767) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 15:49:04.752073 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3398e1f-0940-4917-8dc0-cb4e214b3ba2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81767) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 15:49:04.752715 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3398e1f-0940-4917-8dc0-cb4e214b3ba2 None admin] REQUEST_METHOD: `GET` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:49:04.752994 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3398e1f-0940-4917-8dc0-cb4e214b3ba2 None admin] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:49:04.753269 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3398e1f-0940-4917-8dc0-cb4e214b3ba2 None admin] PATH_INFO: `/v3/projects/207bb32d0fc247a48caa54e3631718f2` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:49:04.754424 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3398e1f-0940-4917-8dc0-cb4e214b3ba2 None admin] RBAC: Authorizing `identity:get_project(project_id=207bb32d0fc247a48caa54e3631718f2)` {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 15:49:04.755892 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3398e1f-0940-4917-8dc0-cb4e214b3ba2 None admin] RBAC: Authorization granted {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 15:49:04.758092 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 562/1145] 10.222.0.15 () {56 vars in 1186 bytes} [Mon Aug 22 15:49:04 2022] GET /identity/v3/projects/207bb32d0fc247a48caa54e3631718f2 => generated 311 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 22 15:49:04.764601 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26b144e6-80ac-4137-b9ac-02a3a244fe55 None admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 15:49:04.892949 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26b144e6-80ac-4137-b9ac-02a3a244fe55 None admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 15:49:04.931413 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-201952a9-a81c-47b4-b8d1-7a84ea1e556f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 15:49:04.932018 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-201952a9-a81c-47b4-b8d1-7a84ea1e556f None admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:49:04.932296 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-201952a9-a81c-47b4-b8d1-7a84ea1e556f None admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:49:04.932568 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-201952a9-a81c-47b4-b8d1-7a84ea1e556f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:49:04.933581 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-201952a9-a81c-47b4-b8d1-7a84ea1e556f None admin] RBAC: Adding query filter params (role.id=9d9848cfd22e4b089db443c350e56670, scope.project.id=207bb32d0fc247a48caa54e3631718f2, user.id=0ca63572bbe34306a150a6d2dae70c28) {{(pid=81768) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 15:49:04.933888 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-201952a9-a81c-47b4-b8d1-7a84ea1e556f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 15:49:04.935412 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-201952a9-a81c-47b4-b8d1-7a84ea1e556f None admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 15:49:04.941353 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 584/1146] 10.222.0.15 () {56 vars in 1374 bytes} [Mon Aug 22 15:49:04 2022] GET /identity/v3/role_assignments?user.id=0ca63572bbe34306a150a6d2dae70c28&scope.project.id=207bb32d0fc247a48caa54e3631718f2&role.id=9d9848cfd22e4b089db443c350e56670 => generated 259 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 15:49:06.932680 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fecb3112-cbd2-46f1-b2f6-4e7d42870830 None None] REQUEST_METHOD: `GET` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:49:06.933464 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fecb3112-cbd2-46f1-b2f6-4e7d42870830 None None] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:49:06.933752 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fecb3112-cbd2-46f1-b2f6-4e7d42870830 None None] PATH_INFO: `/` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:49:06.935879 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 563/1147] 10.222.0.15 () {54 vars in 927 bytes} [Mon Aug 22 15:49:06 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 15:49:06.942122 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceabeffc-3b9e-4b0d-a6bd-fba8d7f7dcca None None] REQUEST_METHOD: `POST` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:49:06.942574 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceabeffc-3b9e-4b0d-a6bd-fba8d7f7dcca None None] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:49:06.942663 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceabeffc-3b9e-4b0d-a6bd-fba8d7f7dcca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:49:06.951187 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=100 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 15:49:06.962757 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.auth.core [None req-ceabeffc-3b9e-4b0d-a6bd-fba8d7f7dcca None None] MFA Rules not processed for user `6b2199ae0fb0480aa917ada391f5ec54`. Rule list: `[]` (Enabled: `True`). {{(pid=81768) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 15:49:06.982267 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-ceabeffc-3b9e-4b0d-a6bd-fba8d7f7dcca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 15:49:07.034138 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 585/1148] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Aug 22 15:49:06 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 22 15:49:07.314636 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fecb3112-cbd2-46f1-b2f6-4e7d42870830 None None] Authenticating user token {{(pid=81767) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 15:49:07.426980 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fecb3112-cbd2-46f1-b2f6-4e7d42870830 None None] Validating token access rules against request {{(pid=81767) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 15:49:07.464020 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bec4a61b-4785-46ae-a959-358542f6c1e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81767) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 15:49:07.464643 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bec4a61b-4785-46ae-a959-358542f6c1e7 None admin] REQUEST_METHOD: `GET` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:49:07.464916 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bec4a61b-4785-46ae-a959-358542f6c1e7 None admin] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:49:07.465186 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bec4a61b-4785-46ae-a959-358542f6c1e7 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:49:07.475676 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bec4a61b-4785-46ae-a959-358542f6c1e7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 22 15:49:07.481108 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application [None req-bec4a61b-4785-46ae-a959-358542f6c1e7 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 22 15:49:07.481108 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 15:49:07.481108 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 15:49:07.481108 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 15:49:07.481108 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 15:49:07.481108 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 15:49:07.481108 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 15:49:07.481108 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 15:49:07.481108 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 15:49:07.481108 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 15:49:07.481108 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 15:49:07.481108 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 15:49:07.481108 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 22 15:49:07.481108 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 22 15:49:07.481108 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 22 15:49:07.481108 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 22 15:49:07.482223 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 22 15:49:07.482223 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 22 15:49:07.482223 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 22 15:49:07.482223 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 22 15:49:07.482223 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 15:49:07.482223 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 15:49:07.482223 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 15:49:07.482223 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 15:49:07.482223 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 15:49:07.482223 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 15:49:07.482223 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 15:49:07.482223 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application with Lock( Aug 22 15:49:07.482223 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 15:49:07.482223 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return self._enter() Aug 22 15:49:07.482223 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 15:49:07.482223 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 15:49:07.483534 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 15:49:07.483534 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return self.creator() Aug 22 15:49:07.483534 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 15:49:07.483534 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application created_value = creator( Aug 22 15:49:07.483534 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 22 15:49:07.483534 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 22 15:49:07.483534 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 22 15:49:07.483534 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 22 15:49:07.483534 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 22 15:49:07.483534 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 22 15:49:07.483534 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 22 15:49:07.483534 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application Aug 22 15:49:07.483534 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 564/1149] 10.222.0.15 () {56 vars in 1096 bytes} [Mon Aug 22 15:49:07 2022] GET /identity/v3/roles/admin => generated 83 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 15:49:07.488617 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceabeffc-3b9e-4b0d-a6bd-fba8d7f7dcca None None] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 15:49:07.592329 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceabeffc-3b9e-4b0d-a6bd-fba8d7f7dcca None None] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 15:49:07.629657 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25f7c2bf-e254-4ca8-ad0b-04076835c819 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 15:49:07.630260 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f7c2bf-e254-4ca8-ad0b-04076835c819 None admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:49:07.630627 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f7c2bf-e254-4ca8-ad0b-04076835c819 None admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:49:07.630902 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f7c2bf-e254-4ca8-ad0b-04076835c819 None admin] PATH_INFO: `/v3/roles` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:49:07.631686 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25f7c2bf-e254-4ca8-ad0b-04076835c819 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81768) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 15:49:07.632040 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25f7c2bf-e254-4ca8-ad0b-04076835c819 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 15:49:07.633552 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25f7c2bf-e254-4ca8-ad0b-04076835c819 None admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 15:49:07.635193 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=100 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 15:49:07.642755 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 586/1150] 10.222.0.15 () {56 vars in 1099 bytes} [Mon Aug 22 15:49:07 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 15:49:07.649760 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bec4a61b-4785-46ae-a959-358542f6c1e7 None admin] Authenticating user token {{(pid=81767) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 15:49:07.699394 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG dbcounter [-] [81767] Writing DB stats keystone:SELECT=100 {{(pid=81767) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 15:49:07.759972 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bec4a61b-4785-46ae-a959-358542f6c1e7 None admin] Validating token access rules against request {{(pid=81767) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 15:49:07.798982 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2961acfb-1b79-4b0b-b564-d53353e488c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81767) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 15:49:07.799655 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2961acfb-1b79-4b0b-b564-d53353e488c9 None admin] REQUEST_METHOD: `GET` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:49:07.799930 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2961acfb-1b79-4b0b-b564-d53353e488c9 None admin] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:49:07.800203 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2961acfb-1b79-4b0b-b564-d53353e488c9 None admin] PATH_INFO: `/v3/users/0ca63572bbe34306a150a6d2dae70c28` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:49:07.801429 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2961acfb-1b79-4b0b-b564-d53353e488c9 None admin] RBAC: Authorizing `identity:get_user(user_id=0ca63572bbe34306a150a6d2dae70c28)` {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 15:49:07.802863 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2961acfb-1b79-4b0b-b564-d53353e488c9 None admin] RBAC: Authorization granted {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 15:49:07.804531 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 565/1151] 10.222.0.15 () {56 vars in 1177 bytes} [Mon Aug 22 15:49:07 2022] GET /identity/v3/users/0ca63572bbe34306a150a6d2dae70c28 => generated 286 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 15:49:07.810756 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25f7c2bf-e254-4ca8-ad0b-04076835c819 None admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 15:49:07.924779 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25f7c2bf-e254-4ca8-ad0b-04076835c819 None admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 15:49:07.965993 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3113c9f3-b299-44df-9313-b2ff96e6157b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 15:49:07.967338 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3113c9f3-b299-44df-9313-b2ff96e6157b None admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:49:07.967620 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3113c9f3-b299-44df-9313-b2ff96e6157b None admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:49:07.968074 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3113c9f3-b299-44df-9313-b2ff96e6157b None admin] PATH_INFO: `/v3/projects/207bb32d0fc247a48caa54e3631718f2` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:49:07.969394 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3113c9f3-b299-44df-9313-b2ff96e6157b None admin] RBAC: Authorizing `identity:get_project(project_id=207bb32d0fc247a48caa54e3631718f2)` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 15:49:07.971081 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3113c9f3-b299-44df-9313-b2ff96e6157b None admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 15:49:07.972790 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 587/1152] 10.222.0.15 () {56 vars in 1186 bytes} [Mon Aug 22 15:49:07 2022] GET /identity/v3/projects/207bb32d0fc247a48caa54e3631718f2 => generated 311 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 15:49:07.979427 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2961acfb-1b79-4b0b-b564-d53353e488c9 None admin] Authenticating user token {{(pid=81767) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 15:49:08.091921 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2961acfb-1b79-4b0b-b564-d53353e488c9 None admin] Validating token access rules against request {{(pid=81767) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 15:49:08.103688 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG dbcounter [-] [81767] Writing DB stats keystone:SELECT=100 {{(pid=81767) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 15:49:08.129341 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43a00812-e7f6-40ec-ac43-db440d123838 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81767) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 15:49:08.140396 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43a00812-e7f6-40ec-ac43-db440d123838 None admin] REQUEST_METHOD: `PUT` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:49:08.140396 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43a00812-e7f6-40ec-ac43-db440d123838 None admin] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:49:08.140396 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43a00812-e7f6-40ec-ac43-db440d123838 None admin] PATH_INFO: `/v3/projects/207bb32d0fc247a48caa54e3631718f2/users/0ca63572bbe34306a150a6d2dae70c28/roles/9d9848cfd22e4b089db443c350e56670` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:49:08.140396 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43a00812-e7f6-40ec-ac43-db440d123838 None admin] RBAC: Authorizing `identity:create_grant(project_id=207bb32d0fc247a48caa54e3631718f2, user_id=0ca63572bbe34306a150a6d2dae70c28, role_id=9d9848cfd22e4b089db443c350e56670)` {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 15:49:08.140396 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43a00812-e7f6-40ec-ac43-db440d123838 None admin] RBAC: Authorization granted {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 15:49:08.309749 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 566/1153] 10.222.0.15 () {58 vars in 1439 bytes} [Mon Aug 22 15:49:07 2022] PUT /identity/v3/projects/207bb32d0fc247a48caa54e3631718f2/users/0ca63572bbe34306a150a6d2dae70c28/roles/9d9848cfd22e4b089db443c350e56670 => generated 0 bytes in 331 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 15:49:10.263925 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-471f6773-cf12-4916-a3a6-fd3fa1d75146 None None] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:49:10.264525 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-471f6773-cf12-4916-a3a6-fd3fa1d75146 None None] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:49:10.264525 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-471f6773-cf12-4916-a3a6-fd3fa1d75146 None None] PATH_INFO: `/` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:49:10.265869 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 588/1154] 10.222.0.15 () {54 vars in 927 bytes} [Mon Aug 22 15:49:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 15:49:10.271302 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-937e876e-0b62-4972-a142-282ac641bff9 None None] REQUEST_METHOD: `POST` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:49:10.271772 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-937e876e-0b62-4972-a142-282ac641bff9 None None] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:49:10.271890 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-937e876e-0b62-4972-a142-282ac641bff9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:49:10.290945 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.auth.core [None req-937e876e-0b62-4972-a142-282ac641bff9 None None] MFA Rules not processed for user `6b2199ae0fb0480aa917ada391f5ec54`. Rule list: `[]` (Enabled: `True`). {{(pid=81767) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 15:49:10.292181 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.fernet_utils [None req-937e876e-0b62-4972-a142-282ac641bff9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81767) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 15:49:10.359866 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 567/1155] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Aug 22 15:49:10 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 22 15:49:10.629804 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-471f6773-cf12-4916-a3a6-fd3fa1d75146 None None] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 15:49:10.693408 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=100 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 15:49:10.732064 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-471f6773-cf12-4916-a3a6-fd3fa1d75146 None None] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 15:49:10.763894 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-053b9ff1-63df-4345-8dac-89b5fec9fc92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 15:49:10.764570 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-053b9ff1-63df-4345-8dac-89b5fec9fc92 None admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:49:10.765034 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-053b9ff1-63df-4345-8dac-89b5fec9fc92 None admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:49:10.765764 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-053b9ff1-63df-4345-8dac-89b5fec9fc92 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:49:10.776906 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-053b9ff1-63df-4345-8dac-89b5fec9fc92 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 22 15:49:10.781619 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application [None req-053b9ff1-63df-4345-8dac-89b5fec9fc92 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 22 15:49:10.781619 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 15:49:10.781619 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 15:49:10.781619 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 15:49:10.781619 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 15:49:10.781619 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 15:49:10.781619 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 15:49:10.781619 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 15:49:10.781619 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 15:49:10.781619 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 15:49:10.781619 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 15:49:10.781619 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 15:49:10.781619 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 22 15:49:10.781619 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 22 15:49:10.781619 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 22 15:49:10.781619 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 22 15:49:10.782745 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 22 15:49:10.782745 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 22 15:49:10.782745 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 22 15:49:10.782745 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 22 15:49:10.782745 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 15:49:10.782745 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 15:49:10.782745 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 15:49:10.782745 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 15:49:10.782745 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 15:49:10.782745 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 15:49:10.782745 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 15:49:10.782745 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application with Lock( Aug 22 15:49:10.782745 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 15:49:10.782745 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self._enter() Aug 22 15:49:10.782745 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 15:49:10.782745 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 15:49:10.783987 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 15:49:10.783987 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self.creator() Aug 22 15:49:10.783987 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 15:49:10.783987 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application created_value = creator( Aug 22 15:49:10.783987 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 22 15:49:10.783987 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 22 15:49:10.783987 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 22 15:49:10.783987 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 22 15:49:10.783987 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 22 15:49:10.783987 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 22 15:49:10.783987 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 22 15:49:10.783987 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application Aug 22 15:49:10.783987 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 589/1156] 10.222.0.15 () {56 vars in 1096 bytes} [Mon Aug 22 15:49:10 2022] GET /identity/v3/roles/admin => generated 83 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 15:49:10.789685 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-937e876e-0b62-4972-a142-282ac641bff9 None None] Authenticating user token {{(pid=81767) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 15:49:10.899825 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG dbcounter [-] [81767] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81767) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 15:49:10.899825 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-937e876e-0b62-4972-a142-282ac641bff9 None None] Validating token access rules against request {{(pid=81767) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 15:49:10.935950 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecfdc4cd-e5cb-46c7-b0f7-2d4d2a8baafb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81767) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 15:49:10.974437 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecfdc4cd-e5cb-46c7-b0f7-2d4d2a8baafb None admin] REQUEST_METHOD: `GET` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:49:10.974437 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecfdc4cd-e5cb-46c7-b0f7-2d4d2a8baafb None admin] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:49:10.974437 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecfdc4cd-e5cb-46c7-b0f7-2d4d2a8baafb None admin] PATH_INFO: `/v3/roles` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:49:10.974437 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecfdc4cd-e5cb-46c7-b0f7-2d4d2a8baafb None admin] RBAC: Adding query filter params (name=admin) {{(pid=81767) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 15:49:10.974437 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecfdc4cd-e5cb-46c7-b0f7-2d4d2a8baafb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 15:49:10.974437 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecfdc4cd-e5cb-46c7-b0f7-2d4d2a8baafb None admin] RBAC: Authorization granted {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 15:49:10.974997 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 568/1157] 10.222.0.15 () {56 vars in 1099 bytes} [Mon Aug 22 15:49:10 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 15:49:10.975090 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-053b9ff1-63df-4345-8dac-89b5fec9fc92 None admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 15:49:11.063504 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-053b9ff1-63df-4345-8dac-89b5fec9fc92 None admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 15:49:11.088654 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=100 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 15:49:11.099039 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae3fa075-0db3-4294-a738-912888ad5382 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 15:49:11.099705 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae3fa075-0db3-4294-a738-912888ad5382 None admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:49:11.099988 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae3fa075-0db3-4294-a738-912888ad5382 None admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:49:11.100258 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae3fa075-0db3-4294-a738-912888ad5382 None admin] PATH_INFO: `/v3/users/0ca63572bbe34306a150a6d2dae70c28` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:49:11.101422 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae3fa075-0db3-4294-a738-912888ad5382 None admin] RBAC: Authorizing `identity:get_user(user_id=0ca63572bbe34306a150a6d2dae70c28)` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 15:49:11.102898 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae3fa075-0db3-4294-a738-912888ad5382 None admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 15:49:11.104519 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 590/1158] 10.222.0.15 () {56 vars in 1177 bytes} [Mon Aug 22 15:49:10 2022] GET /identity/v3/users/0ca63572bbe34306a150a6d2dae70c28 => generated 286 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 15:49:11.110818 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecfdc4cd-e5cb-46c7-b0f7-2d4d2a8baafb None admin] Authenticating user token {{(pid=81767) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 15:49:11.254791 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecfdc4cd-e5cb-46c7-b0f7-2d4d2a8baafb None admin] Validating token access rules against request {{(pid=81767) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 15:49:11.254791 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dedea3d5-d9b8-41dd-8514-df8775ee4edc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81767) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 15:49:11.254791 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dedea3d5-d9b8-41dd-8514-df8775ee4edc None admin] REQUEST_METHOD: `GET` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:49:11.254791 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dedea3d5-d9b8-41dd-8514-df8775ee4edc None admin] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:49:11.255309 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dedea3d5-d9b8-41dd-8514-df8775ee4edc None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:49:11.321894 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dedea3d5-d9b8-41dd-8514-df8775ee4edc None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 15:49:11.323441 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dedea3d5-d9b8-41dd-8514-df8775ee4edc None admin] RBAC: Authorization granted {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 15:49:11.363532 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG dbcounter [-] [81767] Writing DB stats keystone:SELECT=100 {{(pid=81767) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 15:49:11.428452 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 569/1159] 10.222.0.15 () {58 vars in 1282 bytes} [Mon Aug 22 15:49:11 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 318 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 22 15:49:11.435347 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae3fa075-0db3-4294-a738-912888ad5382 None admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 15:49:11.540331 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae3fa075-0db3-4294-a738-912888ad5382 None admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 15:49:11.575466 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48a85ef4-9f12-440f-a5a0-184dbc51c253 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 15:49:11.576093 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48a85ef4-9f12-440f-a5a0-184dbc51c253 None admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:49:11.576370 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48a85ef4-9f12-440f-a5a0-184dbc51c253 None admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:49:11.576641 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48a85ef4-9f12-440f-a5a0-184dbc51c253 None admin] PATH_INFO: `/v3/projects/207bb32d0fc247a48caa54e3631718f2` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:49:11.577761 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48a85ef4-9f12-440f-a5a0-184dbc51c253 None admin] RBAC: Authorizing `identity:get_project(project_id=207bb32d0fc247a48caa54e3631718f2)` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 15:49:11.579233 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48a85ef4-9f12-440f-a5a0-184dbc51c253 None admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 15:49:11.580884 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 591/1160] 10.222.0.15 () {56 vars in 1186 bytes} [Mon Aug 22 15:49:11 2022] GET /identity/v3/projects/207bb32d0fc247a48caa54e3631718f2 => generated 311 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 15:49:11.587098 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dedea3d5-d9b8-41dd-8514-df8775ee4edc None admin] Authenticating user token {{(pid=81767) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 15:49:11.652094 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG dbcounter [-] [81767] Writing DB stats keystone:SELECT=100 {{(pid=81767) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 15:49:11.692798 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dedea3d5-d9b8-41dd-8514-df8775ee4edc None admin] Validating token access rules against request {{(pid=81767) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 15:49:11.728812 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f1ead0a-e466-424b-ab17-23e052e5f57c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81767) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 15:49:11.729409 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f1ead0a-e466-424b-ab17-23e052e5f57c None admin] REQUEST_METHOD: `GET` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:49:11.729682 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f1ead0a-e466-424b-ab17-23e052e5f57c None admin] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:49:11.729946 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f1ead0a-e466-424b-ab17-23e052e5f57c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:49:11.730686 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f1ead0a-e466-424b-ab17-23e052e5f57c None admin] RBAC: Adding query filter params (role.id=9d9848cfd22e4b089db443c350e56670, scope.project.id=207bb32d0fc247a48caa54e3631718f2, user.id=0ca63572bbe34306a150a6d2dae70c28) {{(pid=81767) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 15:49:11.730981 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f1ead0a-e466-424b-ab17-23e052e5f57c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 15:49:11.732416 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f1ead0a-e466-424b-ab17-23e052e5f57c None admin] RBAC: Authorization granted {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 15:49:11.738325 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 570/1161] 10.222.0.15 () {56 vars in 1374 bytes} [Mon Aug 22 15:49:11 2022] GET /identity/v3/role_assignments?user.id=0ca63572bbe34306a150a6d2dae70c28&scope.project.id=207bb32d0fc247a48caa54e3631718f2&role.id=9d9848cfd22e4b089db443c350e56670 => generated 608 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 15:49:13.697070 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70a1e4c8-a520-46a2-9938-1bdc9b3e2abe None None] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:49:13.697586 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70a1e4c8-a520-46a2-9938-1bdc9b3e2abe None None] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:49:13.697586 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70a1e4c8-a520-46a2-9938-1bdc9b3e2abe None None] PATH_INFO: `/` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:49:13.698829 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 592/1162] 10.222.0.15 () {54 vars in 927 bytes} [Mon Aug 22 15:49:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 15:49:13.704645 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0d93fa1-2bea-4f1f-bfde-db284ea43eda None None] REQUEST_METHOD: `POST` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:49:13.705130 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0d93fa1-2bea-4f1f-bfde-db284ea43eda None None] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:49:13.705241 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0d93fa1-2bea-4f1f-bfde-db284ea43eda None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:49:13.723265 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.auth.core [None req-c0d93fa1-2bea-4f1f-bfde-db284ea43eda None None] MFA Rules not processed for user `6b2199ae0fb0480aa917ada391f5ec54`. Rule list: `[]` (Enabled: `True`). {{(pid=81767) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 15:49:13.724705 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.fernet_utils [None req-c0d93fa1-2bea-4f1f-bfde-db284ea43eda None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81767) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 15:49:13.794789 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 571/1163] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Aug 22 15:49:13 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 22 15:49:14.066110 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70a1e4c8-a520-46a2-9938-1bdc9b3e2abe None None] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 15:49:14.128617 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=100 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 15:49:14.172178 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70a1e4c8-a520-46a2-9938-1bdc9b3e2abe None None] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 15:49:14.206846 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b2c8412-fd6d-4de2-b87d-bb2c8eeba675 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 15:49:14.207472 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b2c8412-fd6d-4de2-b87d-bb2c8eeba675 None admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:49:14.207750 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b2c8412-fd6d-4de2-b87d-bb2c8eeba675 None admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:49:14.208019 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b2c8412-fd6d-4de2-b87d-bb2c8eeba675 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:49:14.279632 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b2c8412-fd6d-4de2-b87d-bb2c8eeba675 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 15:49:14.281097 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b2c8412-fd6d-4de2-b87d-bb2c8eeba675 None admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 15:49:14.347230 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=100 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 15:49:14.400648 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 593/1164] 10.222.0.15 () {58 vars in 1282 bytes} [Mon Aug 22 15:49:14 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 335 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 22 15:49:14.407828 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0d93fa1-2bea-4f1f-bfde-db284ea43eda None None] Authenticating user token {{(pid=81767) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 15:49:14.469936 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG dbcounter [-] [81767] Writing DB stats keystone:SELECT=100 {{(pid=81767) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 15:49:14.514194 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0d93fa1-2bea-4f1f-bfde-db284ea43eda None None] Validating token access rules against request {{(pid=81767) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 15:49:14.549414 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-731214fb-dd67-446f-aa89-155223937c90 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81767) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 15:49:14.550027 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-731214fb-dd67-446f-aa89-155223937c90 None admin] REQUEST_METHOD: `GET` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:49:14.550296 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-731214fb-dd67-446f-aa89-155223937c90 None admin] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:49:14.550557 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-731214fb-dd67-446f-aa89-155223937c90 None admin] PATH_INFO: `/v3/domains/swift_test` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:49:14.558631 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-731214fb-dd67-446f-aa89-155223937c90 None admin] RBAC: Authorizing `identity:get_domain(domain_id=swift_test)` {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 15:49:14.560115 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-731214fb-dd67-446f-aa89-155223937c90 None admin] RBAC: Authorization granted {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 15:49:14.572798 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application [None req-731214fb-dd67-446f-aa89-155223937c90 None admin] Could not find domain: swift_test.: keystone.exception.DomainNotFound: Could not find domain: swift_test. Aug 22 15:49:14.572798 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 15:49:14.572798 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 712, in get_domain Aug 22 15:49:14.572798 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application project = self.driver.get_project(domain_id) Aug 22 15:49:14.572798 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 22 15:49:14.572798 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 22 15:49:14.572798 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 22 15:49:14.572798 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 22 15:49:14.572798 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: swift_test. Aug 22 15:49:14.572798 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application Aug 22 15:49:14.572798 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 22 15:49:14.572798 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application Aug 22 15:49:14.572798 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 15:49:14.572798 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 15:49:14.572798 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 15:49:14.572798 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 15:49:14.572798 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 15:49:14.573896 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 15:49:14.573896 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 15:49:14.573896 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 15:49:14.573896 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 15:49:14.573896 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 15:49:14.573896 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 15:49:14.573896 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 22 15:49:14.573896 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 22 15:49:14.573896 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 22 15:49:14.573896 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 22 15:49:14.573896 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 15:49:14.573896 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 15:49:14.573896 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 15:49:14.573896 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 15:49:14.573896 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 15:49:14.573896 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 15:49:14.574764 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 15:49:14.574764 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application with Lock( Aug 22 15:49:14.574764 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 15:49:14.574764 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return self._enter() Aug 22 15:49:14.574764 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 15:49:14.574764 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 15:49:14.574764 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 15:49:14.574764 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return self.creator() Aug 22 15:49:14.574764 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 15:49:14.574764 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application created_value = creator( Aug 22 15:49:14.574764 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 718, in get_domain Aug 22 15:49:14.574764 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 22 15:49:14.574764 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: swift_test. Aug 22 15:49:14.574764 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application Aug 22 15:49:14.574764 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 572/1165] 10.222.0.15 () {56 vars in 1117 bytes} [Mon Aug 22 15:49:14 2022] GET /identity/v3/domains/swift_test => generated 90 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 15:49:14.580107 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b2c8412-fd6d-4de2-b87d-bb2c8eeba675 None admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 15:49:14.681152 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b2c8412-fd6d-4de2-b87d-bb2c8eeba675 None admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 15:49:14.716063 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16c0e3cb-5958-4223-8361-c6e7004ff0a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 15:49:14.716813 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c0e3cb-5958-4223-8361-c6e7004ff0a1 None admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:49:14.716983 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c0e3cb-5958-4223-8361-c6e7004ff0a1 None admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:49:14.717251 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c0e3cb-5958-4223-8361-c6e7004ff0a1 None admin] PATH_INFO: `/v3/domains` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:49:14.717891 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16c0e3cb-5958-4223-8361-c6e7004ff0a1 None admin] RBAC: Adding query filter params (name=swift_test) {{(pid=81768) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 15:49:14.718188 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16c0e3cb-5958-4223-8361-c6e7004ff0a1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 15:49:14.719576 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16c0e3cb-5958-4223-8361-c6e7004ff0a1 None admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 15:49:14.727530 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 594/1166] 10.222.0.15 () {56 vars in 1115 bytes} [Mon Aug 22 15:49:14 2022] GET /identity/v3/domains?name=swift_test => generated 125 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 15:49:16.654340 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73ef5ca8-aae4-4e4c-bb19-116c5b737c7e None None] REQUEST_METHOD: `GET` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:49:16.654912 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73ef5ca8-aae4-4e4c-bb19-116c5b737c7e None None] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:49:16.654912 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73ef5ca8-aae4-4e4c-bb19-116c5b737c7e None None] PATH_INFO: `/` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:49:16.656224 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 573/1167] 10.222.0.15 () {54 vars in 927 bytes} [Mon Aug 22 15:49:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 15:49:16.662632 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7031e269-89d1-4f67-9548-4fde5873cfa9 None None] REQUEST_METHOD: `POST` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:49:16.663094 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7031e269-89d1-4f67-9548-4fde5873cfa9 None None] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:49:16.663192 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7031e269-89d1-4f67-9548-4fde5873cfa9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:49:16.682006 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.auth.core [None req-7031e269-89d1-4f67-9548-4fde5873cfa9 None None] MFA Rules not processed for user `6b2199ae0fb0480aa917ada391f5ec54`. Rule list: `[]` (Enabled: `True`). {{(pid=81768) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 15:49:16.683191 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-7031e269-89d1-4f67-9548-4fde5873cfa9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 15:49:16.701005 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=100 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 15:49:16.753645 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 595/1168] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Aug 22 15:49:16 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 22 15:49:17.031117 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73ef5ca8-aae4-4e4c-bb19-116c5b737c7e None None] Authenticating user token {{(pid=81767) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 15:49:17.138362 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73ef5ca8-aae4-4e4c-bb19-116c5b737c7e None None] Validating token access rules against request {{(pid=81767) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 15:49:17.155718 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG dbcounter [-] [81767] Writing DB stats keystone:SELECT=100 {{(pid=81767) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 15:49:17.185469 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ad7cb25-c71d-4786-84f5-77bfa3b50784 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81767) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 15:49:17.186542 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ad7cb25-c71d-4786-84f5-77bfa3b50784 None admin] REQUEST_METHOD: `POST` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:49:17.187031 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ad7cb25-c71d-4786-84f5-77bfa3b50784 None admin] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:49:17.187548 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ad7cb25-c71d-4786-84f5-77bfa3b50784 None admin] PATH_INFO: `/v3/domains` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:49:17.188732 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ad7cb25-c71d-4786-84f5-77bfa3b50784 None admin] RBAC: Authorizing `identity:create_domain()` {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 15:49:17.191404 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ad7cb25-c71d-4786-84f5-77bfa3b50784 None admin] RBAC: Authorization granted {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 15:49:17.302254 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 574/1169] 10.222.0.15 () {60 vars in 1138 bytes} [Mon Aug 22 15:49:17 2022] POST /identity/v3/domains => generated 267 bytes in 272 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 22 15:49:19.326044 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ca4c24b-1384-48b0-b73c-a296aff69ca8 None None] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:49:19.326617 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ca4c24b-1384-48b0-b73c-a296aff69ca8 None None] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:49:19.326617 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ca4c24b-1384-48b0-b73c-a296aff69ca8 None None] PATH_INFO: `/` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:49:19.327946 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 596/1170] 10.222.0.15 () {54 vars in 927 bytes} [Mon Aug 22 15:49:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 15:49:19.334810 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b94b84d4-5777-44c6-b0f2-9eb465a9d48d None None] REQUEST_METHOD: `POST` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:49:19.335541 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b94b84d4-5777-44c6-b0f2-9eb465a9d48d None None] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:49:19.335541 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b94b84d4-5777-44c6-b0f2-9eb465a9d48d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:49:19.354354 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.auth.core [None req-b94b84d4-5777-44c6-b0f2-9eb465a9d48d None None] MFA Rules not processed for user `6b2199ae0fb0480aa917ada391f5ec54`. Rule list: `[]` (Enabled: `True`). {{(pid=81767) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 15:49:19.355610 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.fernet_utils [None req-b94b84d4-5777-44c6-b0f2-9eb465a9d48d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81767) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 15:49:19.429901 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 575/1171] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Aug 22 15:49:19 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 22 15:49:19.700242 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ca4c24b-1384-48b0-b73c-a296aff69ca8 None None] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 15:49:19.808205 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ca4c24b-1384-48b0-b73c-a296aff69ca8 None None] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 15:49:19.842795 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-001532b1-2aad-417c-ab0b-d83623e9c697 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 15:49:19.843449 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-001532b1-2aad-417c-ab0b-d83623e9c697 None admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:49:19.843730 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-001532b1-2aad-417c-ab0b-d83623e9c697 None admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:49:19.844002 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-001532b1-2aad-417c-ab0b-d83623e9c697 None admin] PATH_INFO: `/v3/domains/eae8ad2d6bfd4a269e61700268708488` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:49:19.856597 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-001532b1-2aad-417c-ab0b-d83623e9c697 None admin] RBAC: Authorizing `identity:get_domain(domain_id=eae8ad2d6bfd4a269e61700268708488)` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 15:49:19.858069 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-001532b1-2aad-417c-ab0b-d83623e9c697 None admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 15:49:19.859773 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 597/1172] 10.222.0.15 () {56 vars in 1183 bytes} [Mon Aug 22 15:49:19 2022] GET /identity/v3/domains/eae8ad2d6bfd4a269e61700268708488 => generated 267 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 15:49:19.865676 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b94b84d4-5777-44c6-b0f2-9eb465a9d48d None None] Authenticating user token {{(pid=81767) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 15:49:19.964499 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b94b84d4-5777-44c6-b0f2-9eb465a9d48d None None] Validating token access rules against request {{(pid=81767) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 15:49:19.972350 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG dbcounter [-] [81767] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81767) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 15:49:20.001646 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89dd1bd3-1f4a-450e-8cdf-fc36056c8389 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81767) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 15:49:20.002249 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89dd1bd3-1f4a-450e-8cdf-fc36056c8389 None admin] REQUEST_METHOD: `POST` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:49:20.002526 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89dd1bd3-1f4a-450e-8cdf-fc36056c8389 None admin] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:49:20.002790 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89dd1bd3-1f4a-450e-8cdf-fc36056c8389 None admin] PATH_INFO: `/v3/projects` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:49:20.003456 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89dd1bd3-1f4a-450e-8cdf-fc36056c8389 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 15:49:20.004903 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89dd1bd3-1f4a-450e-8cdf-fc36056c8389 None admin] RBAC: Authorization granted {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 15:49:20.256619 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 576/1173] 10.222.0.15 () {60 vars in 1141 bytes} [Mon Aug 22 15:49:19 2022] POST /identity/v3/projects => generated 361 bytes in 391 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 22 15:49:22.244445 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce1c1b61-286a-45e9-99b0-bb4ec673a897 None None] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:49:22.245011 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce1c1b61-286a-45e9-99b0-bb4ec673a897 None None] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:49:22.245011 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce1c1b61-286a-45e9-99b0-bb4ec673a897 None None] PATH_INFO: `/` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:49:22.246316 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 598/1174] 10.222.0.15 () {54 vars in 927 bytes} [Mon Aug 22 15:49:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 15:49:22.252967 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37519489-29c9-4d2f-954d-8f63517ff176 None None] REQUEST_METHOD: `POST` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:49:22.252967 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37519489-29c9-4d2f-954d-8f63517ff176 None None] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:49:22.252967 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37519489-29c9-4d2f-954d-8f63517ff176 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:49:22.271384 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.auth.core [None req-37519489-29c9-4d2f-954d-8f63517ff176 None None] MFA Rules not processed for user `6b2199ae0fb0480aa917ada391f5ec54`. Rule list: `[]` (Enabled: `True`). {{(pid=81767) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 15:49:22.272648 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.fernet_utils [None req-37519489-29c9-4d2f-954d-8f63517ff176 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81767) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 15:49:22.343068 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 577/1175] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Aug 22 15:49:22 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 22 15:49:22.619037 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce1c1b61-286a-45e9-99b0-bb4ec673a897 None None] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 15:49:22.636208 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=100 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 15:49:22.727464 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce1c1b61-286a-45e9-99b0-bb4ec673a897 None None] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 15:49:22.763796 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e6a13d9-683c-428a-8141-8c6d8611b428 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 15:49:22.764419 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e6a13d9-683c-428a-8141-8c6d8611b428 None admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:49:22.764695 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e6a13d9-683c-428a-8141-8c6d8611b428 None admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:49:22.764961 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e6a13d9-683c-428a-8141-8c6d8611b428 None admin] PATH_INFO: `/v3/domains/eae8ad2d6bfd4a269e61700268708488` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:49:22.766091 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e6a13d9-683c-428a-8141-8c6d8611b428 None admin] RBAC: Authorizing `identity:get_domain(domain_id=eae8ad2d6bfd4a269e61700268708488)` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 15:49:22.767592 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e6a13d9-683c-428a-8141-8c6d8611b428 None admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 15:49:22.769138 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 599/1176] 10.222.0.15 () {56 vars in 1183 bytes} [Mon Aug 22 15:49:22 2022] GET /identity/v3/domains/eae8ad2d6bfd4a269e61700268708488 => generated 267 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 15:49:22.775616 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37519489-29c9-4d2f-954d-8f63517ff176 None None] Authenticating user token {{(pid=81767) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 15:49:22.869815 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG dbcounter [-] [81767] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81767) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 15:49:22.882022 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37519489-29c9-4d2f-954d-8f63517ff176 None None] Validating token access rules against request {{(pid=81767) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 15:49:22.922877 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b68529f7-b07a-45ca-8c64-2e956453dcb9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81767) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 15:49:22.923597 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b68529f7-b07a-45ca-8c64-2e956453dcb9 None admin] REQUEST_METHOD: `POST` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:49:22.923881 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b68529f7-b07a-45ca-8c64-2e956453dcb9 None admin] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:49:22.924152 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b68529f7-b07a-45ca-8c64-2e956453dcb9 None admin] PATH_INFO: `/v3/users` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:49:22.924803 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b68529f7-b07a-45ca-8c64-2e956453dcb9 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 15:49:22.926259 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b68529f7-b07a-45ca-8c64-2e956453dcb9 None admin] RBAC: Authorization granted {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 15:49:23.078153 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 578/1177] 10.222.0.15 () {60 vars in 1132 bytes} [Mon Aug 22 15:49:22 2022] POST /identity/v3/users => generated 311 bytes in 303 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 22 15:49:25.093662 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c10f33f5-459d-4ca7-b98c-86ced814e985 None None] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:49:25.094237 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c10f33f5-459d-4ca7-b98c-86ced814e985 None None] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:49:25.094237 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c10f33f5-459d-4ca7-b98c-86ced814e985 None None] PATH_INFO: `/` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:49:25.095695 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 600/1178] 10.222.0.15 () {54 vars in 927 bytes} [Mon Aug 22 15:49:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 15:49:25.102543 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-312d72d7-40e1-4144-a97a-21afae7c2541 None None] REQUEST_METHOD: `POST` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:49:25.103101 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-312d72d7-40e1-4144-a97a-21afae7c2541 None None] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:49:25.103101 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-312d72d7-40e1-4144-a97a-21afae7c2541 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:49:25.123203 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.auth.core [None req-312d72d7-40e1-4144-a97a-21afae7c2541 None None] MFA Rules not processed for user `6b2199ae0fb0480aa917ada391f5ec54`. Rule list: `[]` (Enabled: `True`). {{(pid=81767) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 15:49:25.124454 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.fernet_utils [None req-312d72d7-40e1-4144-a97a-21afae7c2541 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81767) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 15:49:25.208096 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 579/1179] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Aug 22 15:49:25 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 22 15:49:25.493747 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c10f33f5-459d-4ca7-b98c-86ced814e985 None None] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 15:49:25.613728 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=100 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 15:49:25.634369 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c10f33f5-459d-4ca7-b98c-86ced814e985 None None] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 15:49:25.680776 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31f7357a-1ae0-4c02-a5ad-964020819937 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 15:49:25.681427 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31f7357a-1ae0-4c02-a5ad-964020819937 None admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:49:25.681700 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31f7357a-1ae0-4c02-a5ad-964020819937 None admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:49:25.681966 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31f7357a-1ae0-4c02-a5ad-964020819937 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:49:25.696275 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31f7357a-1ae0-4c02-a5ad-964020819937 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 22 15:49:25.701068 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application [None req-31f7357a-1ae0-4c02-a5ad-964020819937 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 22 15:49:25.701068 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 15:49:25.701068 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 15:49:25.701068 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 15:49:25.701068 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 15:49:25.701068 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 15:49:25.701068 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 15:49:25.701068 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 15:49:25.701068 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 15:49:25.701068 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 15:49:25.701068 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 15:49:25.701068 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 15:49:25.701068 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 22 15:49:25.701068 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 22 15:49:25.701068 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 22 15:49:25.701068 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 22 15:49:25.703667 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 22 15:49:25.703667 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 22 15:49:25.703667 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 22 15:49:25.703667 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 22 15:49:25.703667 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 15:49:25.703667 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 15:49:25.703667 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 15:49:25.703667 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 15:49:25.703667 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 15:49:25.703667 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 15:49:25.703667 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 15:49:25.703667 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application with Lock( Aug 22 15:49:25.703667 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 15:49:25.703667 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self._enter() Aug 22 15:49:25.703667 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 15:49:25.703667 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 15:49:25.704518 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 15:49:25.704518 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self.creator() Aug 22 15:49:25.704518 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 15:49:25.704518 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application created_value = creator( Aug 22 15:49:25.704518 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 22 15:49:25.704518 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 22 15:49:25.704518 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 22 15:49:25.704518 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 22 15:49:25.704518 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 22 15:49:25.704518 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 22 15:49:25.704518 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 22 15:49:25.704518 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application Aug 22 15:49:25.704518 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 601/1180] 10.222.0.15 () {56 vars in 1096 bytes} [Mon Aug 22 15:49:25 2022] GET /identity/v3/roles/admin => generated 83 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 15:49:25.708770 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-312d72d7-40e1-4144-a97a-21afae7c2541 None None] Authenticating user token {{(pid=81767) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 15:49:25.794200 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG dbcounter [-] [81767] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81767) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 15:49:25.823929 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-312d72d7-40e1-4144-a97a-21afae7c2541 None None] Validating token access rules against request {{(pid=81767) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 15:49:25.861154 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9e97912-c609-4d06-86ec-75a97a8ceca6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81767) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 15:49:25.861878 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9e97912-c609-4d06-86ec-75a97a8ceca6 None admin] REQUEST_METHOD: `GET` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:49:25.862157 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9e97912-c609-4d06-86ec-75a97a8ceca6 None admin] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:49:25.862430 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9e97912-c609-4d06-86ec-75a97a8ceca6 None admin] PATH_INFO: `/v3/roles` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:49:25.863149 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9e97912-c609-4d06-86ec-75a97a8ceca6 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81767) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 15:49:25.863483 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9e97912-c609-4d06-86ec-75a97a8ceca6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 15:49:25.864967 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9e97912-c609-4d06-86ec-75a97a8ceca6 None admin] RBAC: Authorization granted {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 15:49:25.874282 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 580/1181] 10.222.0.15 () {56 vars in 1099 bytes} [Mon Aug 22 15:49:25 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 15:49:25.882015 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31f7357a-1ae0-4c02-a5ad-964020819937 None admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 15:49:25.997198 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31f7357a-1ae0-4c02-a5ad-964020819937 None admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 15:49:26.034671 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90ceee34-6522-4ba6-9737-540d85fe796e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 15:49:26.035333 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90ceee34-6522-4ba6-9737-540d85fe796e None admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:49:26.035593 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90ceee34-6522-4ba6-9737-540d85fe796e None admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:49:26.035868 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90ceee34-6522-4ba6-9737-540d85fe796e None admin] PATH_INFO: `/v3/users/6ce1cf40994f4ee1bf38e8c85ccba866` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:49:26.049244 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=100 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 15:49:26.052334 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90ceee34-6522-4ba6-9737-540d85fe796e None admin] RBAC: Authorizing `identity:get_user(user_id=6ce1cf40994f4ee1bf38e8c85ccba866)` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 15:49:26.059008 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90ceee34-6522-4ba6-9737-540d85fe796e None admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 15:49:26.059008 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 602/1182] 10.222.0.15 () {56 vars in 1177 bytes} [Mon Aug 22 15:49:25 2022] GET /identity/v3/users/6ce1cf40994f4ee1bf38e8c85ccba866 => generated 311 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 15:49:26.062072 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9e97912-c609-4d06-86ec-75a97a8ceca6 None admin] Authenticating user token {{(pid=81767) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 15:49:26.187060 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9e97912-c609-4d06-86ec-75a97a8ceca6 None admin] Validating token access rules against request {{(pid=81767) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 15:49:26.223053 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27166b86-7337-4402-827d-a712d12e3bf9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81767) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 15:49:26.223690 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27166b86-7337-4402-827d-a712d12e3bf9 None admin] REQUEST_METHOD: `GET` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:49:26.223963 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27166b86-7337-4402-827d-a712d12e3bf9 None admin] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:49:26.224225 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27166b86-7337-4402-827d-a712d12e3bf9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:49:26.228359 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG dbcounter [-] [81767] Writing DB stats keystone:SELECT=100 {{(pid=81767) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 15:49:26.300863 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27166b86-7337-4402-827d-a712d12e3bf9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 15:49:26.302226 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27166b86-7337-4402-827d-a712d12e3bf9 None admin] RBAC: Authorization granted {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 15:49:26.409569 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 581/1183] 10.222.0.15 () {58 vars in 1282 bytes} [Mon Aug 22 15:49:26 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 348 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 22 15:49:26.416569 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90ceee34-6522-4ba6-9737-540d85fe796e None admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 15:49:26.532583 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90ceee34-6522-4ba6-9737-540d85fe796e None admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 15:49:26.571254 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8e2648d-c8f6-4297-a4f0-1f284fe5ffce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 15:49:26.572678 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8e2648d-c8f6-4297-a4f0-1f284fe5ffce None admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:49:26.572678 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8e2648d-c8f6-4297-a4f0-1f284fe5ffce None admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:49:26.572678 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8e2648d-c8f6-4297-a4f0-1f284fe5ffce None admin] PATH_INFO: `/v3/projects/c9e610ce717d4d4fa0ff74633c0b5724` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:49:26.573648 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8e2648d-c8f6-4297-a4f0-1f284fe5ffce None admin] RBAC: Authorizing `identity:get_project(project_id=c9e610ce717d4d4fa0ff74633c0b5724)` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 15:49:26.575147 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8e2648d-c8f6-4297-a4f0-1f284fe5ffce None admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 15:49:26.577016 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 603/1184] 10.222.0.15 () {56 vars in 1186 bytes} [Mon Aug 22 15:49:26 2022] GET /identity/v3/projects/c9e610ce717d4d4fa0ff74633c0b5724 => generated 361 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 15:49:26.584530 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27166b86-7337-4402-827d-a712d12e3bf9 None admin] Authenticating user token {{(pid=81767) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 15:49:26.622726 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG dbcounter [-] [81767] Writing DB stats keystone:SELECT=100 {{(pid=81767) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 15:49:26.731830 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27166b86-7337-4402-827d-a712d12e3bf9 None admin] Validating token access rules against request {{(pid=81767) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 15:49:26.786540 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51364886-a859-4a8f-bbb5-deea9ff302fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81767) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 15:49:26.787498 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51364886-a859-4a8f-bbb5-deea9ff302fb None admin] REQUEST_METHOD: `GET` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:49:26.788478 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51364886-a859-4a8f-bbb5-deea9ff302fb None admin] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:49:26.788929 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51364886-a859-4a8f-bbb5-deea9ff302fb None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:49:26.790155 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51364886-a859-4a8f-bbb5-deea9ff302fb None admin] RBAC: Adding query filter params (role.id=9d9848cfd22e4b089db443c350e56670, scope.project.id=c9e610ce717d4d4fa0ff74633c0b5724, user.id=6ce1cf40994f4ee1bf38e8c85ccba866) {{(pid=81767) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 15:49:26.790628 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51364886-a859-4a8f-bbb5-deea9ff302fb None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 15:49:26.792848 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51364886-a859-4a8f-bbb5-deea9ff302fb None admin] RBAC: Authorization granted {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 15:49:26.801961 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 582/1185] 10.222.0.15 () {56 vars in 1374 bytes} [Mon Aug 22 15:49:26 2022] GET /identity/v3/role_assignments?user.id=6ce1cf40994f4ee1bf38e8c85ccba866&scope.project.id=c9e610ce717d4d4fa0ff74633c0b5724&role.id=9d9848cfd22e4b089db443c350e56670 => generated 259 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 22 15:49:28.780403 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f0ff61e-4c15-483d-97b7-77fb975b7fcc None None] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:49:28.781205 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f0ff61e-4c15-483d-97b7-77fb975b7fcc None None] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:49:28.781205 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f0ff61e-4c15-483d-97b7-77fb975b7fcc None None] PATH_INFO: `/` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:49:28.782219 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 604/1186] 10.222.0.15 () {54 vars in 927 bytes} [Mon Aug 22 15:49:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 15:49:28.789286 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34390c36-c5b9-4bc4-87f5-f9307a11e302 None None] REQUEST_METHOD: `POST` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:49:28.789838 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34390c36-c5b9-4bc4-87f5-f9307a11e302 None None] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:49:28.789838 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34390c36-c5b9-4bc4-87f5-f9307a11e302 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:49:28.809424 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.auth.core [None req-34390c36-c5b9-4bc4-87f5-f9307a11e302 None None] MFA Rules not processed for user `6b2199ae0fb0480aa917ada391f5ec54`. Rule list: `[]` (Enabled: `True`). {{(pid=81767) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 15:49:28.810652 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.fernet_utils [None req-34390c36-c5b9-4bc4-87f5-f9307a11e302 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81767) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 15:49:28.901728 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 583/1187] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Aug 22 15:49:28 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 22 15:49:29.329066 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f0ff61e-4c15-483d-97b7-77fb975b7fcc None None] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 15:49:29.447905 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=100 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 15:49:29.490195 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f0ff61e-4c15-483d-97b7-77fb975b7fcc None None] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 15:49:29.529342 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87ab6144-b74d-4506-92d6-26111e94adb5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 15:49:29.530152 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87ab6144-b74d-4506-92d6-26111e94adb5 None admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:49:29.530484 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87ab6144-b74d-4506-92d6-26111e94adb5 None admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:49:29.530799 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87ab6144-b74d-4506-92d6-26111e94adb5 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:49:29.542833 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87ab6144-b74d-4506-92d6-26111e94adb5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 22 15:49:29.547766 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application [None req-87ab6144-b74d-4506-92d6-26111e94adb5 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 22 15:49:29.547766 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 15:49:29.547766 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 15:49:29.547766 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 15:49:29.547766 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 15:49:29.547766 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 15:49:29.547766 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 15:49:29.547766 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 15:49:29.547766 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 15:49:29.547766 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 15:49:29.547766 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 15:49:29.547766 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 15:49:29.547766 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 22 15:49:29.547766 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 22 15:49:29.547766 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 22 15:49:29.547766 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 22 15:49:29.549905 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 22 15:49:29.549905 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 22 15:49:29.549905 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 22 15:49:29.549905 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 22 15:49:29.549905 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 15:49:29.549905 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 15:49:29.549905 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 15:49:29.549905 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 15:49:29.549905 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 15:49:29.549905 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 15:49:29.549905 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 15:49:29.549905 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application with Lock( Aug 22 15:49:29.549905 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 15:49:29.549905 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self._enter() Aug 22 15:49:29.549905 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 15:49:29.549905 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 15:49:29.551927 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 15:49:29.551927 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self.creator() Aug 22 15:49:29.551927 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 15:49:29.551927 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application created_value = creator( Aug 22 15:49:29.551927 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 22 15:49:29.551927 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 22 15:49:29.551927 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 22 15:49:29.551927 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 22 15:49:29.551927 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 22 15:49:29.551927 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 22 15:49:29.551927 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 22 15:49:29.551927 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application Aug 22 15:49:29.551927 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 605/1188] 10.222.0.15 () {56 vars in 1096 bytes} [Mon Aug 22 15:49:29 2022] GET /identity/v3/roles/admin => generated 83 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 15:49:29.557000 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34390c36-c5b9-4bc4-87f5-f9307a11e302 None None] Authenticating user token {{(pid=81767) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 15:49:29.596309 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG dbcounter [-] [81767] Writing DB stats keystone:SELECT=100 {{(pid=81767) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 15:49:29.682445 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34390c36-c5b9-4bc4-87f5-f9307a11e302 None None] Validating token access rules against request {{(pid=81767) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 15:49:29.732892 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb21fa0d-9d5e-403f-aed7-7060297fc52f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81767) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 15:49:29.733798 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb21fa0d-9d5e-403f-aed7-7060297fc52f None admin] REQUEST_METHOD: `GET` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:49:29.734262 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb21fa0d-9d5e-403f-aed7-7060297fc52f None admin] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:49:29.734718 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb21fa0d-9d5e-403f-aed7-7060297fc52f None admin] PATH_INFO: `/v3/roles` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:49:29.735828 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb21fa0d-9d5e-403f-aed7-7060297fc52f None admin] RBAC: Adding query filter params (name=admin) {{(pid=81767) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 15:49:29.736327 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb21fa0d-9d5e-403f-aed7-7060297fc52f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 15:49:29.738515 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb21fa0d-9d5e-403f-aed7-7060297fc52f None admin] RBAC: Authorization granted {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 15:49:29.754096 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 584/1189] 10.222.0.15 () {56 vars in 1099 bytes} [Mon Aug 22 15:49:29 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 22 15:49:29.760909 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87ab6144-b74d-4506-92d6-26111e94adb5 None admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 15:49:29.893591 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87ab6144-b74d-4506-92d6-26111e94adb5 None admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 15:49:29.925804 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=100 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 15:49:29.935167 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7848dea7-0e1b-4fcd-8e33-d4105ffdba9a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 15:49:29.935886 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7848dea7-0e1b-4fcd-8e33-d4105ffdba9a None admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:49:29.936207 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7848dea7-0e1b-4fcd-8e33-d4105ffdba9a None admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:49:29.936514 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7848dea7-0e1b-4fcd-8e33-d4105ffdba9a None admin] PATH_INFO: `/v3/users/6ce1cf40994f4ee1bf38e8c85ccba866` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:49:29.937704 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7848dea7-0e1b-4fcd-8e33-d4105ffdba9a None admin] RBAC: Authorizing `identity:get_user(user_id=6ce1cf40994f4ee1bf38e8c85ccba866)` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 15:49:29.939160 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7848dea7-0e1b-4fcd-8e33-d4105ffdba9a None admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 15:49:29.940810 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 606/1190] 10.222.0.15 () {56 vars in 1177 bytes} [Mon Aug 22 15:49:29 2022] GET /identity/v3/users/6ce1cf40994f4ee1bf38e8c85ccba866 => generated 311 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 15:49:29.950005 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb21fa0d-9d5e-403f-aed7-7060297fc52f None admin] Authenticating user token {{(pid=81767) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 15:49:30.061137 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG dbcounter [-] [81767] Writing DB stats keystone:SELECT=100 {{(pid=81767) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 15:49:30.071205 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb21fa0d-9d5e-403f-aed7-7060297fc52f None admin] Validating token access rules against request {{(pid=81767) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 15:49:30.128492 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a9cde2a-2838-4fc1-926e-2633cb3cdd47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81767) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 15:49:30.129269 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a9cde2a-2838-4fc1-926e-2633cb3cdd47 None admin] REQUEST_METHOD: `GET` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:49:30.129629 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a9cde2a-2838-4fc1-926e-2633cb3cdd47 None admin] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:49:30.129983 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a9cde2a-2838-4fc1-926e-2633cb3cdd47 None admin] PATH_INFO: `/v3/projects/c9e610ce717d4d4fa0ff74633c0b5724` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:49:30.131404 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a9cde2a-2838-4fc1-926e-2633cb3cdd47 None admin] RBAC: Authorizing `identity:get_project(project_id=c9e610ce717d4d4fa0ff74633c0b5724)` {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 15:49:30.132970 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a9cde2a-2838-4fc1-926e-2633cb3cdd47 None admin] RBAC: Authorization granted {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 15:49:30.134796 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 585/1191] 10.222.0.15 () {56 vars in 1186 bytes} [Mon Aug 22 15:49:29 2022] GET /identity/v3/projects/c9e610ce717d4d4fa0ff74633c0b5724 => generated 361 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 15:49:30.141299 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7848dea7-0e1b-4fcd-8e33-d4105ffdba9a None admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 15:49:30.263907 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7848dea7-0e1b-4fcd-8e33-d4105ffdba9a None admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 15:49:30.303207 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16157e6e-3744-45e2-a947-e9e534655674 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 15:49:30.304437 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16157e6e-3744-45e2-a947-e9e534655674 None admin] REQUEST_METHOD: `PUT` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:49:30.304756 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16157e6e-3744-45e2-a947-e9e534655674 None admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:49:30.305072 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16157e6e-3744-45e2-a947-e9e534655674 None admin] PATH_INFO: `/v3/projects/c9e610ce717d4d4fa0ff74633c0b5724/users/6ce1cf40994f4ee1bf38e8c85ccba866/roles/9d9848cfd22e4b089db443c350e56670` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:49:30.307514 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16157e6e-3744-45e2-a947-e9e534655674 None admin] RBAC: Authorizing `identity:create_grant(project_id=c9e610ce717d4d4fa0ff74633c0b5724, user_id=6ce1cf40994f4ee1bf38e8c85ccba866, role_id=9d9848cfd22e4b089db443c350e56670)` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 15:49:30.309290 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16157e6e-3744-45e2-a947-e9e534655674 None admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 15:49:30.436270 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 607/1192] 10.222.0.15 () {58 vars in 1439 bytes} [Mon Aug 22 15:49:30 2022] PUT /identity/v3/projects/c9e610ce717d4d4fa0ff74633c0b5724/users/6ce1cf40994f4ee1bf38e8c85ccba866/roles/9d9848cfd22e4b089db443c350e56670 => generated 0 bytes in 295 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 22 15:49:32.494402 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e62bfc6-93ac-408c-abe7-3855cd29aa48 None None] REQUEST_METHOD: `GET` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:49:32.495013 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e62bfc6-93ac-408c-abe7-3855cd29aa48 None None] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:49:32.495013 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e62bfc6-93ac-408c-abe7-3855cd29aa48 None None] PATH_INFO: `/` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:49:32.496316 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 586/1193] 10.222.0.15 () {54 vars in 927 bytes} [Mon Aug 22 15:49:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 15:49:32.501978 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9c324a5-193f-4566-97f8-5332ca4fb31a None None] REQUEST_METHOD: `POST` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:49:32.502498 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9c324a5-193f-4566-97f8-5332ca4fb31a None None] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:49:32.502498 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9c324a5-193f-4566-97f8-5332ca4fb31a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:49:32.521580 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.auth.core [None req-e9c324a5-193f-4566-97f8-5332ca4fb31a None None] MFA Rules not processed for user `6b2199ae0fb0480aa917ada391f5ec54`. Rule list: `[]` (Enabled: `True`). {{(pid=81768) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 15:49:32.523383 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-e9c324a5-193f-4566-97f8-5332ca4fb31a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 15:49:32.576588 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 15:49:32.594583 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 608/1194] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Aug 22 15:49:32 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 22 15:49:32.866546 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e62bfc6-93ac-408c-abe7-3855cd29aa48 None None] Authenticating user token {{(pid=81767) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 15:49:32.973798 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e62bfc6-93ac-408c-abe7-3855cd29aa48 None None] Validating token access rules against request {{(pid=81767) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 15:49:33.010175 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-434c8b4f-e467-4757-99aa-17d30e87b186 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81767) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 15:49:33.010847 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-434c8b4f-e467-4757-99aa-17d30e87b186 None admin] REQUEST_METHOD: `GET` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:49:33.011125 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-434c8b4f-e467-4757-99aa-17d30e87b186 None admin] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:49:33.011427 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-434c8b4f-e467-4757-99aa-17d30e87b186 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:49:33.022743 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-434c8b4f-e467-4757-99aa-17d30e87b186 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 22 15:49:33.027537 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application [None req-434c8b4f-e467-4757-99aa-17d30e87b186 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 22 15:49:33.027537 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 15:49:33.027537 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 15:49:33.027537 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 15:49:33.027537 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 15:49:33.027537 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 15:49:33.027537 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 15:49:33.027537 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 15:49:33.027537 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 15:49:33.027537 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 15:49:33.027537 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 15:49:33.027537 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 15:49:33.027537 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 22 15:49:33.027537 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 22 15:49:33.027537 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 22 15:49:33.027537 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 22 15:49:33.028541 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 22 15:49:33.028541 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 22 15:49:33.028541 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 22 15:49:33.028541 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 22 15:49:33.028541 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 15:49:33.028541 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 15:49:33.028541 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 15:49:33.028541 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 15:49:33.028541 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 15:49:33.028541 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 15:49:33.028541 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 15:49:33.028541 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application with Lock( Aug 22 15:49:33.028541 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 15:49:33.028541 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return self._enter() Aug 22 15:49:33.028541 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 15:49:33.028541 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 15:49:33.029655 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 15:49:33.029655 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return self.creator() Aug 22 15:49:33.029655 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 15:49:33.029655 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application created_value = creator( Aug 22 15:49:33.029655 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 22 15:49:33.029655 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 22 15:49:33.029655 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 22 15:49:33.029655 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 22 15:49:33.029655 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 22 15:49:33.029655 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 22 15:49:33.029655 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 22 15:49:33.029655 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application Aug 22 15:49:33.029655 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 587/1195] 10.222.0.15 () {56 vars in 1096 bytes} [Mon Aug 22 15:49:32 2022] GET /identity/v3/roles/admin => generated 83 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 15:49:33.035248 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9c324a5-193f-4566-97f8-5332ca4fb31a None None] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 15:49:33.138210 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9c324a5-193f-4566-97f8-5332ca4fb31a None None] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 15:49:33.173915 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51748519-7e4e-4b8f-9230-67d3ff3dee96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 15:49:33.174521 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51748519-7e4e-4b8f-9230-67d3ff3dee96 None admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:49:33.174791 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51748519-7e4e-4b8f-9230-67d3ff3dee96 None admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:49:33.175057 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51748519-7e4e-4b8f-9230-67d3ff3dee96 None admin] PATH_INFO: `/v3/roles` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:49:33.175745 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51748519-7e4e-4b8f-9230-67d3ff3dee96 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81768) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 15:49:33.176044 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51748519-7e4e-4b8f-9230-67d3ff3dee96 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 15:49:33.177414 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51748519-7e4e-4b8f-9230-67d3ff3dee96 None admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 15:49:33.186447 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 609/1196] 10.222.0.15 () {56 vars in 1099 bytes} [Mon Aug 22 15:49:33 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 15:49:33.192553 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-434c8b4f-e467-4757-99aa-17d30e87b186 None admin] Authenticating user token {{(pid=81767) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 15:49:33.215919 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG dbcounter [-] [81767] Writing DB stats keystone:SELECT=100 {{(pid=81767) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 15:49:33.296795 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-434c8b4f-e467-4757-99aa-17d30e87b186 None admin] Validating token access rules against request {{(pid=81767) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 15:49:33.336597 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3099634-10ae-4b31-a41f-df3d3d7d6e0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81767) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 15:49:33.337241 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3099634-10ae-4b31-a41f-df3d3d7d6e0c None admin] REQUEST_METHOD: `GET` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:49:33.337509 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3099634-10ae-4b31-a41f-df3d3d7d6e0c None admin] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:49:33.337770 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3099634-10ae-4b31-a41f-df3d3d7d6e0c None admin] PATH_INFO: `/v3/users/6ce1cf40994f4ee1bf38e8c85ccba866` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:49:33.338994 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3099634-10ae-4b31-a41f-df3d3d7d6e0c None admin] RBAC: Authorizing `identity:get_user(user_id=6ce1cf40994f4ee1bf38e8c85ccba866)` {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 15:49:33.340516 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3099634-10ae-4b31-a41f-df3d3d7d6e0c None admin] RBAC: Authorization granted {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 15:49:33.342158 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 588/1197] 10.222.0.15 () {56 vars in 1177 bytes} [Mon Aug 22 15:49:33 2022] GET /identity/v3/users/6ce1cf40994f4ee1bf38e8c85ccba866 => generated 311 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 15:49:33.348672 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51748519-7e4e-4b8f-9230-67d3ff3dee96 None admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 15:49:33.403610 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=100 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 15:49:33.458766 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51748519-7e4e-4b8f-9230-67d3ff3dee96 None admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 15:49:33.493976 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-282d39ac-235a-433e-a23a-27d9975122dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 15:49:33.494586 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-282d39ac-235a-433e-a23a-27d9975122dd None admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:49:33.494863 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-282d39ac-235a-433e-a23a-27d9975122dd None admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:49:33.495131 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-282d39ac-235a-433e-a23a-27d9975122dd None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:49:33.566863 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-282d39ac-235a-433e-a23a-27d9975122dd None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 15:49:33.568368 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-282d39ac-235a-433e-a23a-27d9975122dd None admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 15:49:33.627754 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=100 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 15:49:33.684318 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 610/1198] 10.222.0.15 () {58 vars in 1282 bytes} [Mon Aug 22 15:49:33 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 336 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 22 15:49:33.690424 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3099634-10ae-4b31-a41f-df3d3d7d6e0c None admin] Authenticating user token {{(pid=81767) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 15:49:33.782428 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG dbcounter [-] [81767] Writing DB stats keystone:SELECT=100 {{(pid=81767) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 15:49:33.794718 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3099634-10ae-4b31-a41f-df3d3d7d6e0c None admin] Validating token access rules against request {{(pid=81767) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 15:49:33.830696 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae8c6707-6ca4-408a-8778-d672673af23b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81767) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 15:49:33.831339 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae8c6707-6ca4-408a-8778-d672673af23b None admin] REQUEST_METHOD: `GET` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:49:33.831622 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae8c6707-6ca4-408a-8778-d672673af23b None admin] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:49:33.831888 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae8c6707-6ca4-408a-8778-d672673af23b None admin] PATH_INFO: `/v3/projects/c9e610ce717d4d4fa0ff74633c0b5724` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:49:33.833012 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae8c6707-6ca4-408a-8778-d672673af23b None admin] RBAC: Authorizing `identity:get_project(project_id=c9e610ce717d4d4fa0ff74633c0b5724)` {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 15:49:33.834425 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae8c6707-6ca4-408a-8778-d672673af23b None admin] RBAC: Authorization granted {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 15:49:33.836094 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 589/1199] 10.222.0.15 () {56 vars in 1186 bytes} [Mon Aug 22 15:49:33 2022] GET /identity/v3/projects/c9e610ce717d4d4fa0ff74633c0b5724 => generated 361 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 15:49:33.842926 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-282d39ac-235a-433e-a23a-27d9975122dd None admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 15:49:34.018582 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-282d39ac-235a-433e-a23a-27d9975122dd None admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 15:49:34.077099 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6570e99-f622-4e56-bf88-640cb4c062fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 15:49:34.077695 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6570e99-f622-4e56-bf88-640cb4c062fe None admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:49:34.077972 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6570e99-f622-4e56-bf88-640cb4c062fe None admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:49:34.078241 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6570e99-f622-4e56-bf88-640cb4c062fe None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:49:34.078964 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6570e99-f622-4e56-bf88-640cb4c062fe None admin] RBAC: Adding query filter params (role.id=9d9848cfd22e4b089db443c350e56670, scope.project.id=c9e610ce717d4d4fa0ff74633c0b5724, user.id=6ce1cf40994f4ee1bf38e8c85ccba866) {{(pid=81768) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 15:49:34.079321 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6570e99-f622-4e56-bf88-640cb4c062fe None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 15:49:34.080760 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6570e99-f622-4e56-bf88-640cb4c062fe None admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 15:49:34.087794 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 611/1200] 10.222.0.15 () {56 vars in 1374 bytes} [Mon Aug 22 15:49:33 2022] GET /identity/v3/role_assignments?user.id=6ce1cf40994f4ee1bf38e8c85ccba866&scope.project.id=c9e610ce717d4d4fa0ff74633c0b5724&role.id=9d9848cfd22e4b089db443c350e56670 => generated 608 bytes in 246 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 15:49:43.826368 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG dbcounter [-] [81767] Writing DB stats keystone:SELECT=22 {{(pid=81767) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 15:49:44.084372 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=94 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 15:51:07.732978 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f4b5e6c-29a3-4d4b-9101-85344566cfae None None] REQUEST_METHOD: `GET` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:51:07.733559 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f4b5e6c-29a3-4d4b-9101-85344566cfae None None] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:51:07.733559 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f4b5e6c-29a3-4d4b-9101-85344566cfae None None] PATH_INFO: `/` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:51:07.734792 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 590/1201] 10.222.0.15 () {54 vars in 927 bytes} [Mon Aug 22 15:51:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 22 15:51:07.741398 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b13b2223-6b7e-48d6-94bd-02c6208287d1 None None] REQUEST_METHOD: `POST` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:51:07.747200 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b13b2223-6b7e-48d6-94bd-02c6208287d1 None None] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:51:07.747200 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b13b2223-6b7e-48d6-94bd-02c6208287d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:51:07.761173 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.auth.core [None req-b13b2223-6b7e-48d6-94bd-02c6208287d1 None None] MFA Rules not processed for user `6b2199ae0fb0480aa917ada391f5ec54`. Rule list: `[]` (Enabled: `True`). {{(pid=81768) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 15:51:07.762374 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-b13b2223-6b7e-48d6-94bd-02c6208287d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 15:51:07.831927 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 612/1202] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Aug 22 15:51:07 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 22 15:51:08.105370 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f4b5e6c-29a3-4d4b-9101-85344566cfae None None] Authenticating user token {{(pid=81767) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 15:51:08.209137 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f4b5e6c-29a3-4d4b-9101-85344566cfae None None] Validating token access rules against request {{(pid=81767) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 15:51:08.243036 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eca1d696-c5e7-4b53-aa19-3882ca5c23d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81767) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 15:51:08.243692 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eca1d696-c5e7-4b53-aa19-3882ca5c23d7 None admin] REQUEST_METHOD: `GET` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:51:08.243970 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eca1d696-c5e7-4b53-aa19-3882ca5c23d7 None admin] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:51:08.244244 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eca1d696-c5e7-4b53-aa19-3882ca5c23d7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:51:08.254847 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eca1d696-c5e7-4b53-aa19-3882ca5c23d7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 15:51:08.256297 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eca1d696-c5e7-4b53-aa19-3882ca5c23d7 None admin] RBAC: Authorization granted {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 15:51:08.270617 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application [None req-eca1d696-c5e7-4b53-aa19-3882ca5c23d7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 22 15:51:08.270617 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 15:51:08.270617 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 15:51:08.270617 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 15:51:08.270617 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 15:51:08.270617 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 15:51:08.270617 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 15:51:08.270617 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 15:51:08.270617 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 15:51:08.270617 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 15:51:08.270617 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 15:51:08.270617 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 15:51:08.270617 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 22 15:51:08.270617 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 22 15:51:08.270617 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 22 15:51:08.270617 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 22 15:51:08.272341 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 15:51:08.272341 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 15:51:08.272341 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 15:51:08.272341 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 15:51:08.272341 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 15:51:08.272341 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 15:51:08.272341 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 15:51:08.272341 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application with Lock( Aug 22 15:51:08.272341 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 15:51:08.272341 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return self._enter() Aug 22 15:51:08.272341 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 15:51:08.272341 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 15:51:08.272341 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 15:51:08.272341 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return self.creator() Aug 22 15:51:08.272341 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 15:51:08.272341 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application created_value = creator( Aug 22 15:51:08.272341 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 22 15:51:08.273307 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 22 15:51:08.273307 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 22 15:51:08.273307 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application Aug 22 15:51:08.273307 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 591/1203] 10.222.0.15 () {56 vars in 1108 bytes} [Mon Aug 22 15:51:08 2022] GET /identity/v3/domains/Default => generated 87 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 15:51:08.278293 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b13b2223-6b7e-48d6-94bd-02c6208287d1 None None] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 15:51:08.377186 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b13b2223-6b7e-48d6-94bd-02c6208287d1 None None] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 15:51:08.408388 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=100 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 15:51:08.414863 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bde56520-9664-40e4-8036-d412b4f337b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 15:51:08.415502 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde56520-9664-40e4-8036-d412b4f337b0 None admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:51:08.415783 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde56520-9664-40e4-8036-d412b4f337b0 None admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:51:08.416052 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde56520-9664-40e4-8036-d412b4f337b0 None admin] PATH_INFO: `/v3/domains` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:51:08.416697 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bde56520-9664-40e4-8036-d412b4f337b0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81768) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 15:51:08.417100 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bde56520-9664-40e4-8036-d412b4f337b0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 15:51:08.418395 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bde56520-9664-40e4-8036-d412b4f337b0 None admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 15:51:08.430769 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 613/1204] 10.222.0.15 () {56 vars in 1109 bytes} [Mon Aug 22 15:51:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 15:51:08.437113 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eca1d696-c5e7-4b53-aa19-3882ca5c23d7 None admin] Authenticating user token {{(pid=81767) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 15:51:08.490485 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG dbcounter [-] [81767] Writing DB stats keystone:SELECT=100 {{(pid=81767) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 15:51:08.536889 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eca1d696-c5e7-4b53-aa19-3882ca5c23d7 None admin] Validating token access rules against request {{(pid=81767) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 15:51:08.570983 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69e397fd-58de-44fb-becb-559329f40bea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81767) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 15:51:08.571614 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69e397fd-58de-44fb-becb-559329f40bea None admin] REQUEST_METHOD: `POST` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:51:08.571888 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69e397fd-58de-44fb-becb-559329f40bea None admin] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:51:08.572151 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69e397fd-58de-44fb-becb-559329f40bea None admin] PATH_INFO: `/v3/users` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:51:08.572781 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69e397fd-58de-44fb-becb-559329f40bea None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 15:51:08.574187 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69e397fd-58de-44fb-becb-559329f40bea None admin] RBAC: Authorization granted {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 15:51:08.708378 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 592/1205] 10.222.0.15 () {60 vars in 1132 bytes} [Mon Aug 22 15:51:08 2022] POST /identity/v3/users => generated 251 bytes in 272 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 22 15:51:10.764749 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9addaa1f-a62a-4b2b-86df-7da1410ffb2b None None] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:51:10.765265 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9addaa1f-a62a-4b2b-86df-7da1410ffb2b None None] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:51:10.765265 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9addaa1f-a62a-4b2b-86df-7da1410ffb2b None None] PATH_INFO: `/` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:51:10.766543 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 614/1206] 10.222.0.15 () {54 vars in 927 bytes} [Mon Aug 22 15:51:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 15:51:10.773023 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bddca8ad-c073-43e5-93c7-e9b1f750d8f7 None None] REQUEST_METHOD: `POST` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:51:10.773559 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bddca8ad-c073-43e5-93c7-e9b1f750d8f7 None None] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:51:10.773559 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bddca8ad-c073-43e5-93c7-e9b1f750d8f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:51:10.792959 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.auth.core [None req-bddca8ad-c073-43e5-93c7-e9b1f750d8f7 None None] MFA Rules not processed for user `6b2199ae0fb0480aa917ada391f5ec54`. Rule list: `[]` (Enabled: `True`). {{(pid=81767) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 15:51:10.794175 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.fernet_utils [None req-bddca8ad-c073-43e5-93c7-e9b1f750d8f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81767) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 15:51:10.867468 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 593/1207] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Aug 22 15:51:10 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 22 15:51:11.150420 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9addaa1f-a62a-4b2b-86df-7da1410ffb2b None None] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 15:51:11.255478 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9addaa1f-a62a-4b2b-86df-7da1410ffb2b None None] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 15:51:11.289031 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-389189db-4c97-42a0-a9e4-db6b8acfa3df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 15:51:11.289605 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-389189db-4c97-42a0-a9e4-db6b8acfa3df None admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:51:11.289873 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-389189db-4c97-42a0-a9e4-db6b8acfa3df None admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:51:11.290138 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-389189db-4c97-42a0-a9e4-db6b8acfa3df None admin] PATH_INFO: `/v3/roles/service` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:51:11.300882 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-389189db-4c97-42a0-a9e4-db6b8acfa3df None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 22 15:51:11.305488 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application [None req-389189db-4c97-42a0-a9e4-db6b8acfa3df None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 22 15:51:11.305488 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 15:51:11.305488 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 15:51:11.305488 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 15:51:11.305488 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 15:51:11.305488 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 15:51:11.305488 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 15:51:11.305488 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 15:51:11.305488 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 15:51:11.305488 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 15:51:11.305488 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 15:51:11.305488 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 15:51:11.305488 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 22 15:51:11.305488 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 22 15:51:11.305488 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 22 15:51:11.305488 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 22 15:51:11.306704 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 22 15:51:11.306704 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 22 15:51:11.306704 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 22 15:51:11.306704 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 22 15:51:11.306704 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 15:51:11.306704 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 15:51:11.306704 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 15:51:11.306704 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 15:51:11.306704 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 15:51:11.306704 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 15:51:11.306704 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 15:51:11.306704 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application with Lock( Aug 22 15:51:11.306704 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 15:51:11.306704 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self._enter() Aug 22 15:51:11.306704 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 15:51:11.306704 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 15:51:11.307654 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 15:51:11.307654 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self.creator() Aug 22 15:51:11.307654 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 15:51:11.307654 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application created_value = creator( Aug 22 15:51:11.307654 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 22 15:51:11.307654 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 22 15:51:11.307654 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 22 15:51:11.307654 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 22 15:51:11.307654 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 22 15:51:11.307654 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 22 15:51:11.307654 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 22 15:51:11.307654 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application Aug 22 15:51:11.307654 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 615/1208] 10.222.0.15 () {56 vars in 1102 bytes} [Mon Aug 22 15:51:11 2022] GET /identity/v3/roles/service => generated 85 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 15:51:11.313120 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bddca8ad-c073-43e5-93c7-e9b1f750d8f7 None None] Authenticating user token {{(pid=81767) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 15:51:11.353863 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG dbcounter [-] [81767] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81767) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 15:51:11.415033 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bddca8ad-c073-43e5-93c7-e9b1f750d8f7 None None] Validating token access rules against request {{(pid=81767) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 15:51:11.451250 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc571b66-1c5d-4a41-bfe6-ecec106a9c2c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81767) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 15:51:11.451880 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc571b66-1c5d-4a41-bfe6-ecec106a9c2c None admin] REQUEST_METHOD: `GET` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:51:11.452156 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc571b66-1c5d-4a41-bfe6-ecec106a9c2c None admin] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:51:11.452427 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc571b66-1c5d-4a41-bfe6-ecec106a9c2c None admin] PATH_INFO: `/v3/roles` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:51:11.453087 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc571b66-1c5d-4a41-bfe6-ecec106a9c2c None admin] RBAC: Adding query filter params (name=service) {{(pid=81767) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 15:51:11.453386 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc571b66-1c5d-4a41-bfe6-ecec106a9c2c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 15:51:11.454769 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc571b66-1c5d-4a41-bfe6-ecec106a9c2c None admin] RBAC: Authorization granted {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 15:51:11.462769 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 594/1209] 10.222.0.15 () {56 vars in 1103 bytes} [Mon Aug 22 15:51:11 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 15:51:11.468477 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-389189db-4c97-42a0-a9e4-db6b8acfa3df None admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 15:51:11.520735 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=100 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 15:51:11.571152 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-389189db-4c97-42a0-a9e4-db6b8acfa3df None admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 15:51:11.606162 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9403fdc2-5247-4e67-bac4-b9efb9cf163e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 15:51:11.606815 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9403fdc2-5247-4e67-bac4-b9efb9cf163e None admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:51:11.607264 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9403fdc2-5247-4e67-bac4-b9efb9cf163e None admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:51:11.607567 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9403fdc2-5247-4e67-bac4-b9efb9cf163e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:51:11.618761 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9403fdc2-5247-4e67-bac4-b9efb9cf163e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 15:51:11.620270 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9403fdc2-5247-4e67-bac4-b9efb9cf163e None admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 15:51:11.634822 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application [None req-9403fdc2-5247-4e67-bac4-b9efb9cf163e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 22 15:51:11.634822 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 15:51:11.634822 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 15:51:11.634822 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 15:51:11.634822 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 15:51:11.634822 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 15:51:11.634822 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 15:51:11.634822 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 15:51:11.634822 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 15:51:11.634822 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 15:51:11.634822 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 15:51:11.634822 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 15:51:11.634822 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 22 15:51:11.634822 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 22 15:51:11.634822 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 22 15:51:11.634822 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 22 15:51:11.636247 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 15:51:11.636247 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 15:51:11.636247 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 15:51:11.636247 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 15:51:11.636247 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 15:51:11.636247 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 15:51:11.636247 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 15:51:11.636247 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application with Lock( Aug 22 15:51:11.636247 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 15:51:11.636247 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self._enter() Aug 22 15:51:11.636247 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 15:51:11.636247 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 15:51:11.636247 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 15:51:11.636247 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self.creator() Aug 22 15:51:11.636247 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 15:51:11.636247 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application created_value = creator( Aug 22 15:51:11.636247 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 22 15:51:11.637287 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 22 15:51:11.637287 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 22 15:51:11.637287 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application Aug 22 15:51:11.637287 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 616/1210] 10.222.0.15 () {56 vars in 1108 bytes} [Mon Aug 22 15:51:11 2022] GET /identity/v3/domains/Default => generated 87 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 15:51:11.641984 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc571b66-1c5d-4a41-bfe6-ecec106a9c2c None admin] Authenticating user token {{(pid=81767) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 15:51:11.743631 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc571b66-1c5d-4a41-bfe6-ecec106a9c2c None admin] Validating token access rules against request {{(pid=81767) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 15:51:11.777611 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9818be5f-99d9-4804-886c-6b6165a49813 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81767) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 15:51:11.778381 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9818be5f-99d9-4804-886c-6b6165a49813 None admin] REQUEST_METHOD: `GET` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:51:11.778562 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9818be5f-99d9-4804-886c-6b6165a49813 None admin] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:51:11.778725 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9818be5f-99d9-4804-886c-6b6165a49813 None admin] PATH_INFO: `/v3/domains` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:51:11.779389 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9818be5f-99d9-4804-886c-6b6165a49813 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81767) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 15:51:11.779679 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9818be5f-99d9-4804-886c-6b6165a49813 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 15:51:11.781080 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9818be5f-99d9-4804-886c-6b6165a49813 None admin] RBAC: Authorization granted {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 15:51:11.793441 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 595/1211] 10.222.0.15 () {56 vars in 1109 bytes} [Mon Aug 22 15:51:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 15:51:11.799216 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9403fdc2-5247-4e67-bac4-b9efb9cf163e None admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 15:51:11.902074 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9403fdc2-5247-4e67-bac4-b9efb9cf163e None admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 15:51:11.939470 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b27cc02-5d25-455a-92d1-ac1832291b77 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 15:51:11.940157 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b27cc02-5d25-455a-92d1-ac1832291b77 None admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:51:11.940437 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b27cc02-5d25-455a-92d1-ac1832291b77 None admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:51:11.940706 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b27cc02-5d25-455a-92d1-ac1832291b77 None admin] PATH_INFO: `/v3/users/placement` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:51:11.949062 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b27cc02-5d25-455a-92d1-ac1832291b77 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 15:51:11.950692 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b27cc02-5d25-455a-92d1-ac1832291b77 None admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 15:51:11.955243 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=100 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 15:51:11.964417 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application [None req-9b27cc02-5d25-455a-92d1-ac1832291b77 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 22 15:51:11.964417 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 15:51:11.964417 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 15:51:11.964417 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 15:51:11.964417 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 15:51:11.964417 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 15:51:11.964417 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 15:51:11.964417 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 15:51:11.964417 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 15:51:11.964417 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 15:51:11.964417 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 15:51:11.964417 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 15:51:11.964417 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 22 15:51:11.964417 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 22 15:51:11.964417 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 22 15:51:11.964417 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 22 15:51:11.965505 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 15:51:11.965505 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 15:51:11.965505 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 22 15:51:11.965505 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 22 15:51:11.965505 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 22 15:51:11.965505 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 22 15:51:11.965505 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 15:51:11.965505 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 15:51:11.965505 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 15:51:11.965505 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 15:51:11.965505 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 15:51:11.965505 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application with Lock( Aug 22 15:51:11.965505 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 15:51:11.965505 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self._enter() Aug 22 15:51:11.965505 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 15:51:11.965505 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 15:51:11.965505 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 15:51:11.965505 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self.creator() Aug 22 15:51:11.966893 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 15:51:11.966893 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application created_value = creator( Aug 22 15:51:11.966893 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 22 15:51:11.966893 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 22 15:51:11.966893 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 22 15:51:11.966893 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 22 15:51:11.966893 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 22 15:51:11.966893 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 22 15:51:11.966893 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 22 15:51:11.966893 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application Aug 22 15:51:11.966893 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 617/1212] 10.222.0.15 () {56 vars in 1108 bytes} [Mon Aug 22 15:51:11 2022] GET /identity/v3/users/placement => generated 87 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 15:51:11.971902 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9818be5f-99d9-4804-886c-6b6165a49813 None admin] Authenticating user token {{(pid=81767) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 15:51:12.065877 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG dbcounter [-] [81767] Writing DB stats keystone:SELECT=100 {{(pid=81767) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 15:51:12.078038 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9818be5f-99d9-4804-886c-6b6165a49813 None admin] Validating token access rules against request {{(pid=81767) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 15:51:12.114811 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-083108e6-98f7-43df-ad97-ea2551215d5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81767) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 15:51:12.115451 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-083108e6-98f7-43df-ad97-ea2551215d5f None admin] REQUEST_METHOD: `GET` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:51:12.115727 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-083108e6-98f7-43df-ad97-ea2551215d5f None admin] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:51:12.115997 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-083108e6-98f7-43df-ad97-ea2551215d5f None admin] PATH_INFO: `/v3/users` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:51:12.116720 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-083108e6-98f7-43df-ad97-ea2551215d5f None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81767) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 15:51:12.117018 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-083108e6-98f7-43df-ad97-ea2551215d5f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 15:51:12.118450 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-083108e6-98f7-43df-ad97-ea2551215d5f None admin] RBAC: Authorization granted {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 15:51:12.135887 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 596/1213] 10.222.0.15 () {56 vars in 1143 bytes} [Mon Aug 22 15:51:11 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 15:51:12.142135 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b27cc02-5d25-455a-92d1-ac1832291b77 None admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 15:51:12.250613 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b27cc02-5d25-455a-92d1-ac1832291b77 None admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 15:51:12.286799 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d04febfb-456f-48db-a48e-07bfbd45d818 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 15:51:12.287428 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d04febfb-456f-48db-a48e-07bfbd45d818 None admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:51:12.287704 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d04febfb-456f-48db-a48e-07bfbd45d818 None admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:51:12.288025 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d04febfb-456f-48db-a48e-07bfbd45d818 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:51:12.333454 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=100 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 15:51:12.359818 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d04febfb-456f-48db-a48e-07bfbd45d818 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 15:51:12.361284 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d04febfb-456f-48db-a48e-07bfbd45d818 None admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 15:51:12.466692 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 618/1214] 10.222.0.15 () {58 vars in 1282 bytes} [Mon Aug 22 15:51:12 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 325 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 22 15:51:12.472672 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-083108e6-98f7-43df-ad97-ea2551215d5f None admin] Authenticating user token {{(pid=81767) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 15:51:12.551102 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG dbcounter [-] [81767] Writing DB stats keystone:SELECT=100 {{(pid=81767) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 15:51:12.577847 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-083108e6-98f7-43df-ad97-ea2551215d5f None admin] Validating token access rules against request {{(pid=81767) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 15:51:12.614548 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2518fa38-9421-4a56-9084-26f4fe6806d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81767) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 15:51:12.615152 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2518fa38-9421-4a56-9084-26f4fe6806d4 None admin] REQUEST_METHOD: `GET` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:51:12.615504 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2518fa38-9421-4a56-9084-26f4fe6806d4 None admin] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:51:12.615806 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2518fa38-9421-4a56-9084-26f4fe6806d4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:51:12.626008 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2518fa38-9421-4a56-9084-26f4fe6806d4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 15:51:12.627522 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2518fa38-9421-4a56-9084-26f4fe6806d4 None admin] RBAC: Authorization granted {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 15:51:12.643089 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application [None req-2518fa38-9421-4a56-9084-26f4fe6806d4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 22 15:51:12.643089 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 15:51:12.643089 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 15:51:12.643089 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 15:51:12.643089 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 15:51:12.643089 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 15:51:12.643089 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 15:51:12.643089 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 15:51:12.643089 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 15:51:12.643089 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 15:51:12.643089 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 15:51:12.643089 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 15:51:12.643089 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 22 15:51:12.643089 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 22 15:51:12.643089 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 22 15:51:12.643089 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 22 15:51:12.644344 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 15:51:12.644344 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 15:51:12.644344 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 15:51:12.644344 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 15:51:12.644344 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 15:51:12.644344 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 15:51:12.644344 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 15:51:12.644344 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application with Lock( Aug 22 15:51:12.644344 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 15:51:12.644344 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return self._enter() Aug 22 15:51:12.644344 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 15:51:12.644344 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 15:51:12.644344 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 15:51:12.644344 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return self.creator() Aug 22 15:51:12.644344 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 15:51:12.644344 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application created_value = creator( Aug 22 15:51:12.644344 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 22 15:51:12.645799 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 22 15:51:12.645799 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 22 15:51:12.645799 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application Aug 22 15:51:12.645799 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 597/1215] 10.222.0.15 () {56 vars in 1108 bytes} [Mon Aug 22 15:51:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 15:51:12.650751 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d04febfb-456f-48db-a48e-07bfbd45d818 None admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 15:51:12.747539 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d04febfb-456f-48db-a48e-07bfbd45d818 None admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 15:51:12.769491 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=100 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 15:51:12.787474 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32a78f38-4890-43c1-87de-19a63059a551 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 15:51:12.788144 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32a78f38-4890-43c1-87de-19a63059a551 None admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:51:12.788457 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32a78f38-4890-43c1-87de-19a63059a551 None admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:51:12.788768 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32a78f38-4890-43c1-87de-19a63059a551 None admin] PATH_INFO: `/v3/domains` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:51:12.789600 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32a78f38-4890-43c1-87de-19a63059a551 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81768) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 15:51:12.789922 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32a78f38-4890-43c1-87de-19a63059a551 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 15:51:12.791403 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32a78f38-4890-43c1-87de-19a63059a551 None admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 15:51:12.804437 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 619/1216] 10.222.0.15 () {56 vars in 1109 bytes} [Mon Aug 22 15:51:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 15:51:12.810951 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2518fa38-9421-4a56-9084-26f4fe6806d4 None admin] Authenticating user token {{(pid=81767) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 15:51:12.914215 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2518fa38-9421-4a56-9084-26f4fe6806d4 None admin] Validating token access rules against request {{(pid=81767) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 15:51:12.951255 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91958735-c536-4eaa-b7e3-ff881a1a99e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81767) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 15:51:12.951913 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91958735-c536-4eaa-b7e3-ff881a1a99e1 None admin] REQUEST_METHOD: `GET` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:51:12.952198 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91958735-c536-4eaa-b7e3-ff881a1a99e1 None admin] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:51:12.952474 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91958735-c536-4eaa-b7e3-ff881a1a99e1 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:51:12.959331 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91958735-c536-4eaa-b7e3-ff881a1a99e1 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 15:51:12.960790 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91958735-c536-4eaa-b7e3-ff881a1a99e1 None admin] RBAC: Authorization granted {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 15:51:12.965339 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG dbcounter [-] [81767] Writing DB stats keystone:SELECT=100 {{(pid=81767) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 15:51:12.975671 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application [None req-91958735-c536-4eaa-b7e3-ff881a1a99e1 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 22 15:51:12.975671 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 15:51:12.975671 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 15:51:12.975671 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 15:51:12.975671 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 15:51:12.975671 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 15:51:12.975671 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 15:51:12.975671 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 15:51:12.975671 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 15:51:12.975671 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 15:51:12.975671 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 15:51:12.975671 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 15:51:12.975671 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 22 15:51:12.975671 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 22 15:51:12.975671 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 22 15:51:12.982930 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 22 15:51:12.982930 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 15:51:12.982930 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 15:51:12.982930 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 15:51:12.982930 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 15:51:12.982930 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 15:51:12.982930 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 15:51:12.982930 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 15:51:12.982930 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application with Lock( Aug 22 15:51:12.982930 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 15:51:12.982930 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return self._enter() Aug 22 15:51:12.982930 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 15:51:12.982930 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 15:51:12.982930 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 15:51:12.982930 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return self.creator() Aug 22 15:51:12.982930 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 15:51:12.982930 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application created_value = creator( Aug 22 15:51:12.983945 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 22 15:51:12.983945 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 22 15:51:12.983945 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 22 15:51:12.983945 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 22 15:51:12.983945 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 22 15:51:12.983945 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 22 15:51:12.983945 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 22 15:51:12.983945 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application Aug 22 15:51:12.983945 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 598/1217] 10.222.0.15 () {56 vars in 1111 bytes} [Mon Aug 22 15:51:12 2022] GET /identity/v3/projects/service => generated 88 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 15:51:12.984481 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32a78f38-4890-43c1-87de-19a63059a551 None admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 15:51:13.092569 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32a78f38-4890-43c1-87de-19a63059a551 None admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 15:51:13.126766 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-210227da-4114-46f2-ba2f-04fdbdd7d4cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 15:51:13.127385 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-210227da-4114-46f2-ba2f-04fdbdd7d4cf None admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:51:13.127661 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-210227da-4114-46f2-ba2f-04fdbdd7d4cf None admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:51:13.127927 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-210227da-4114-46f2-ba2f-04fdbdd7d4cf None admin] PATH_INFO: `/v3/projects` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:51:13.128613 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-210227da-4114-46f2-ba2f-04fdbdd7d4cf None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81768) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 15:51:13.128910 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-210227da-4114-46f2-ba2f-04fdbdd7d4cf None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 15:51:13.130890 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-210227da-4114-46f2-ba2f-04fdbdd7d4cf None admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 15:51:13.142274 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=100 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 15:51:13.146152 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 620/1218] 10.222.0.15 () {56 vars in 1148 bytes} [Mon Aug 22 15:51:12 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 15:51:13.152696 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91958735-c536-4eaa-b7e3-ff881a1a99e1 None admin] Authenticating user token {{(pid=81767) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 15:51:13.267823 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91958735-c536-4eaa-b7e3-ff881a1a99e1 None admin] Validating token access rules against request {{(pid=81767) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 15:51:13.304629 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecbf8753-db78-45d3-80ef-1990188f2f94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81767) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 15:51:13.305234 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecbf8753-db78-45d3-80ef-1990188f2f94 None admin] REQUEST_METHOD: `GET` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:51:13.305511 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecbf8753-db78-45d3-80ef-1990188f2f94 None admin] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:51:13.305839 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecbf8753-db78-45d3-80ef-1990188f2f94 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:51:13.306513 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecbf8753-db78-45d3-80ef-1990188f2f94 None admin] RBAC: Adding query filter params (role.id=a5fbba1dc615490a96cde27e3a102eb6, scope.project.id=c4ee0413f9ef4d4fb1f95a660babe090, user.id=08159bf8edda4246bb463f7b11b0894b) {{(pid=81767) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 15:51:13.306817 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecbf8753-db78-45d3-80ef-1990188f2f94 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 15:51:13.308281 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecbf8753-db78-45d3-80ef-1990188f2f94 None admin] RBAC: Authorization granted {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 15:51:13.314039 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 599/1219] 10.222.0.15 () {56 vars in 1374 bytes} [Mon Aug 22 15:51:13 2022] GET /identity/v3/role_assignments?user.id=08159bf8edda4246bb463f7b11b0894b&scope.project.id=c4ee0413f9ef4d4fb1f95a660babe090&role.id=a5fbba1dc615490a96cde27e3a102eb6 => generated 259 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 15:51:13.605629 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG dbcounter [-] [81767] Writing DB stats keystone:SELECT=100 {{(pid=81767) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 15:51:15.228699 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5cb3d7a-fe23-4541-b668-34dde28e94a7 None None] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:51:15.229269 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5cb3d7a-fe23-4541-b668-34dde28e94a7 None None] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:51:15.229269 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5cb3d7a-fe23-4541-b668-34dde28e94a7 None None] PATH_INFO: `/` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:51:15.230603 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 621/1220] 10.222.0.15 () {54 vars in 927 bytes} [Mon Aug 22 15:51:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 15:51:15.236509 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1da2f65-4158-46f1-88b5-09700f8e53e9 None None] REQUEST_METHOD: `POST` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:51:15.236990 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1da2f65-4158-46f1-88b5-09700f8e53e9 None None] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:51:15.237098 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1da2f65-4158-46f1-88b5-09700f8e53e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:51:15.256593 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.auth.core [None req-f1da2f65-4158-46f1-88b5-09700f8e53e9 None None] MFA Rules not processed for user `6b2199ae0fb0480aa917ada391f5ec54`. Rule list: `[]` (Enabled: `True`). {{(pid=81767) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 15:51:15.257924 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.fernet_utils [None req-f1da2f65-4158-46f1-88b5-09700f8e53e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81767) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 15:51:15.331962 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 600/1221] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Aug 22 15:51:15 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 22 15:51:15.615604 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5cb3d7a-fe23-4541-b668-34dde28e94a7 None None] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 15:51:15.728547 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5cb3d7a-fe23-4541-b668-34dde28e94a7 None None] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 15:51:15.767295 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad162d0b-5c91-4c87-8465-ff4eccd2c4d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 15:51:15.768072 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad162d0b-5c91-4c87-8465-ff4eccd2c4d4 None admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:51:15.770275 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad162d0b-5c91-4c87-8465-ff4eccd2c4d4 None admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:51:15.770275 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad162d0b-5c91-4c87-8465-ff4eccd2c4d4 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:51:15.780962 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad162d0b-5c91-4c87-8465-ff4eccd2c4d4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 22 15:51:15.786224 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application [None req-ad162d0b-5c91-4c87-8465-ff4eccd2c4d4 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 22 15:51:15.786224 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 15:51:15.786224 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 15:51:15.786224 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 15:51:15.786224 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 15:51:15.786224 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 15:51:15.786224 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 15:51:15.786224 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 15:51:15.786224 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 15:51:15.786224 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 15:51:15.786224 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 15:51:15.786224 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 15:51:15.786224 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 22 15:51:15.786224 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 22 15:51:15.786224 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 22 15:51:15.786224 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 22 15:51:15.791830 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 22 15:51:15.791830 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 22 15:51:15.791830 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 22 15:51:15.791830 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 22 15:51:15.791830 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 15:51:15.791830 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 15:51:15.791830 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 15:51:15.791830 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 15:51:15.791830 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 15:51:15.791830 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 15:51:15.791830 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 15:51:15.791830 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application with Lock( Aug 22 15:51:15.791830 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 15:51:15.791830 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self._enter() Aug 22 15:51:15.791830 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 15:51:15.791830 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 15:51:15.792961 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 15:51:15.792961 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self.creator() Aug 22 15:51:15.792961 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 15:51:15.792961 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application created_value = creator( Aug 22 15:51:15.792961 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 22 15:51:15.792961 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 22 15:51:15.792961 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 22 15:51:15.792961 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 22 15:51:15.792961 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 22 15:51:15.792961 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 22 15:51:15.792961 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 22 15:51:15.792961 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application Aug 22 15:51:15.792961 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 622/1222] 10.222.0.15 () {56 vars in 1102 bytes} [Mon Aug 22 15:51:15 2022] GET /identity/v3/roles/service => generated 85 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 15:51:15.794075 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1da2f65-4158-46f1-88b5-09700f8e53e9 None None] Authenticating user token {{(pid=81767) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 15:51:15.901418 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1da2f65-4158-46f1-88b5-09700f8e53e9 None None] Validating token access rules against request {{(pid=81767) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 15:51:15.929565 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG dbcounter [-] [81767] Writing DB stats keystone:SELECT=100 {{(pid=81767) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 15:51:15.942629 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a6c0460-0a90-4713-bd1b-ea738d4b5a67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81767) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 15:51:15.943300 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a6c0460-0a90-4713-bd1b-ea738d4b5a67 None admin] REQUEST_METHOD: `GET` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:51:15.943592 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a6c0460-0a90-4713-bd1b-ea738d4b5a67 None admin] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:51:15.943863 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a6c0460-0a90-4713-bd1b-ea738d4b5a67 None admin] PATH_INFO: `/v3/roles` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:51:15.944546 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a6c0460-0a90-4713-bd1b-ea738d4b5a67 None admin] RBAC: Adding query filter params (name=service) {{(pid=81767) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 15:51:15.944858 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a6c0460-0a90-4713-bd1b-ea738d4b5a67 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 15:51:15.946352 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a6c0460-0a90-4713-bd1b-ea738d4b5a67 None admin] RBAC: Authorization granted {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 15:51:15.956437 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 601/1223] 10.222.0.15 () {56 vars in 1103 bytes} [Mon Aug 22 15:51:15 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 15:51:15.962465 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad162d0b-5c91-4c87-8465-ff4eccd2c4d4 None admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 15:51:16.013689 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=100 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 15:51:16.073538 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad162d0b-5c91-4c87-8465-ff4eccd2c4d4 None admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 15:51:16.110330 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae818a59-81f3-4982-839f-2f531d5ae3a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 15:51:16.110986 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae818a59-81f3-4982-839f-2f531d5ae3a5 None admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:51:16.111261 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae818a59-81f3-4982-839f-2f531d5ae3a5 None admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:51:16.111592 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae818a59-81f3-4982-839f-2f531d5ae3a5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:51:16.123751 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae818a59-81f3-4982-839f-2f531d5ae3a5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 15:51:16.130283 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae818a59-81f3-4982-839f-2f531d5ae3a5 None admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 15:51:16.140425 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application [None req-ae818a59-81f3-4982-839f-2f531d5ae3a5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 22 15:51:16.140425 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 15:51:16.140425 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 15:51:16.140425 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 15:51:16.140425 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 15:51:16.140425 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 15:51:16.140425 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 15:51:16.140425 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 15:51:16.140425 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 15:51:16.140425 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 15:51:16.140425 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 15:51:16.140425 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 15:51:16.140425 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 22 15:51:16.140425 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 22 15:51:16.140425 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 22 15:51:16.140425 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 22 15:51:16.141657 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 15:51:16.141657 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 15:51:16.141657 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 15:51:16.141657 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 15:51:16.141657 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 15:51:16.141657 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 15:51:16.141657 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 15:51:16.141657 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application with Lock( Aug 22 15:51:16.141657 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 15:51:16.141657 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self._enter() Aug 22 15:51:16.141657 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 15:51:16.141657 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 15:51:16.141657 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 15:51:16.141657 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self.creator() Aug 22 15:51:16.141657 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 15:51:16.141657 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application created_value = creator( Aug 22 15:51:16.141657 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 22 15:51:16.142770 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 22 15:51:16.142770 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 22 15:51:16.142770 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application Aug 22 15:51:16.142770 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 623/1224] 10.222.0.15 () {56 vars in 1108 bytes} [Mon Aug 22 15:51:15 2022] GET /identity/v3/domains/Default => generated 87 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 15:51:16.147933 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a6c0460-0a90-4713-bd1b-ea738d4b5a67 None admin] Authenticating user token {{(pid=81767) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 15:51:16.256605 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a6c0460-0a90-4713-bd1b-ea738d4b5a67 None admin] Validating token access rules against request {{(pid=81767) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 15:51:16.291480 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-216382ff-988a-49e3-9d22-e6e564ce568e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81767) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 15:51:16.292081 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-216382ff-988a-49e3-9d22-e6e564ce568e None admin] REQUEST_METHOD: `GET` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:51:16.292356 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-216382ff-988a-49e3-9d22-e6e564ce568e None admin] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:51:16.292625 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-216382ff-988a-49e3-9d22-e6e564ce568e None admin] PATH_INFO: `/v3/domains` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:51:16.293261 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-216382ff-988a-49e3-9d22-e6e564ce568e None admin] RBAC: Adding query filter params (name=Default) {{(pid=81767) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 15:51:16.293559 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-216382ff-988a-49e3-9d22-e6e564ce568e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 15:51:16.294903 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-216382ff-988a-49e3-9d22-e6e564ce568e None admin] RBAC: Authorization granted {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 15:51:16.307452 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 602/1225] 10.222.0.15 () {56 vars in 1109 bytes} [Mon Aug 22 15:51:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 22 15:51:16.314463 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae818a59-81f3-4982-839f-2f531d5ae3a5 None admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 15:51:16.418659 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae818a59-81f3-4982-839f-2f531d5ae3a5 None admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 15:51:16.454159 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5e55dc9-3a88-4a03-95f6-6c334c893262 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 15:51:16.454786 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5e55dc9-3a88-4a03-95f6-6c334c893262 None admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:51:16.455065 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5e55dc9-3a88-4a03-95f6-6c334c893262 None admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:51:16.455365 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5e55dc9-3a88-4a03-95f6-6c334c893262 None admin] PATH_INFO: `/v3/users/placement` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:51:16.463207 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5e55dc9-3a88-4a03-95f6-6c334c893262 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 15:51:16.464781 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5e55dc9-3a88-4a03-95f6-6c334c893262 None admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 15:51:16.476721 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application [None req-e5e55dc9-3a88-4a03-95f6-6c334c893262 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 22 15:51:16.476721 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 15:51:16.476721 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 15:51:16.476721 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 15:51:16.476721 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 15:51:16.476721 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 15:51:16.476721 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 15:51:16.476721 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 15:51:16.476721 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 15:51:16.476721 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 15:51:16.476721 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 15:51:16.476721 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 15:51:16.476721 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 22 15:51:16.476721 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 22 15:51:16.476721 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 22 15:51:16.476721 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 22 15:51:16.491824 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 15:51:16.491824 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 15:51:16.491824 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 22 15:51:16.491824 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 22 15:51:16.491824 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 22 15:51:16.491824 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 22 15:51:16.491824 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 15:51:16.491824 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 15:51:16.491824 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 15:51:16.491824 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 15:51:16.491824 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 15:51:16.491824 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application with Lock( Aug 22 15:51:16.491824 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 15:51:16.491824 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self._enter() Aug 22 15:51:16.491824 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 15:51:16.491824 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 15:51:16.491824 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 15:51:16.491824 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self.creator() Aug 22 15:51:16.493408 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 15:51:16.493408 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application created_value = creator( Aug 22 15:51:16.493408 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 22 15:51:16.493408 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 22 15:51:16.493408 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 22 15:51:16.493408 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 22 15:51:16.493408 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 22 15:51:16.493408 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 22 15:51:16.493408 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 22 15:51:16.493408 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application Aug 22 15:51:16.493408 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 624/1226] 10.222.0.15 () {56 vars in 1108 bytes} [Mon Aug 22 15:51:16 2022] GET /identity/v3/users/placement => generated 87 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 15:51:16.494236 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-216382ff-988a-49e3-9d22-e6e564ce568e None admin] Authenticating user token {{(pid=81767) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 15:51:16.542788 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG dbcounter [-] [81767] Writing DB stats keystone:SELECT=100 {{(pid=81767) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 15:51:16.603736 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-216382ff-988a-49e3-9d22-e6e564ce568e None admin] Validating token access rules against request {{(pid=81767) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 15:51:16.639140 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f825b0c7-e48d-4fd5-8db0-df6e326c1b15 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81767) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 15:51:16.639775 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f825b0c7-e48d-4fd5-8db0-df6e326c1b15 None admin] REQUEST_METHOD: `GET` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:51:16.640056 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f825b0c7-e48d-4fd5-8db0-df6e326c1b15 None admin] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:51:16.640325 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f825b0c7-e48d-4fd5-8db0-df6e326c1b15 None admin] PATH_INFO: `/v3/users` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:51:16.641058 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f825b0c7-e48d-4fd5-8db0-df6e326c1b15 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81767) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 15:51:16.641358 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f825b0c7-e48d-4fd5-8db0-df6e326c1b15 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 15:51:16.642771 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f825b0c7-e48d-4fd5-8db0-df6e326c1b15 None admin] RBAC: Authorization granted {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 15:51:16.661640 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 603/1227] 10.222.0.15 () {56 vars in 1143 bytes} [Mon Aug 22 15:51:16 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 15:51:16.667778 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5e55dc9-3a88-4a03-95f6-6c334c893262 None admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 15:51:16.700498 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=100 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 15:51:16.785870 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5e55dc9-3a88-4a03-95f6-6c334c893262 None admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 15:51:16.819482 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86fd70cf-799d-46ad-8840-66c47f08822a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 15:51:16.820026 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86fd70cf-799d-46ad-8840-66c47f08822a None admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:51:16.820306 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86fd70cf-799d-46ad-8840-66c47f08822a None admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:51:16.820579 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86fd70cf-799d-46ad-8840-66c47f08822a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:51:16.831083 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86fd70cf-799d-46ad-8840-66c47f08822a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 15:51:16.833101 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86fd70cf-799d-46ad-8840-66c47f08822a None admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 15:51:16.847124 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application [None req-86fd70cf-799d-46ad-8840-66c47f08822a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 22 15:51:16.847124 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 15:51:16.847124 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 15:51:16.847124 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 15:51:16.847124 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 15:51:16.847124 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 15:51:16.847124 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 15:51:16.847124 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 15:51:16.847124 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 15:51:16.847124 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 15:51:16.847124 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 15:51:16.847124 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 15:51:16.847124 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 22 15:51:16.847124 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 22 15:51:16.847124 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 22 15:51:16.847124 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 22 15:51:16.848974 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 15:51:16.848974 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 15:51:16.848974 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 15:51:16.848974 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 15:51:16.848974 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 15:51:16.848974 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 15:51:16.848974 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 15:51:16.848974 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application with Lock( Aug 22 15:51:16.848974 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 15:51:16.848974 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self._enter() Aug 22 15:51:16.848974 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 15:51:16.848974 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 15:51:16.848974 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 15:51:16.848974 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self.creator() Aug 22 15:51:16.848974 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 15:51:16.848974 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application created_value = creator( Aug 22 15:51:16.848974 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 22 15:51:16.850281 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 22 15:51:16.850281 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 22 15:51:16.850281 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application Aug 22 15:51:16.850281 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 625/1228] 10.222.0.15 () {56 vars in 1108 bytes} [Mon Aug 22 15:51:16 2022] GET /identity/v3/domains/Default => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 15:51:16.855099 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f825b0c7-e48d-4fd5-8db0-df6e326c1b15 None admin] Authenticating user token {{(pid=81767) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 15:51:16.959007 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f825b0c7-e48d-4fd5-8db0-df6e326c1b15 None admin] Validating token access rules against request {{(pid=81767) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 15:51:16.962159 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG dbcounter [-] [81767] Writing DB stats keystone:SELECT=100 {{(pid=81767) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 15:51:16.995233 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bf059ea-562e-46a5-9b2e-7a25556c3e39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81767) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 15:51:16.995860 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bf059ea-562e-46a5-9b2e-7a25556c3e39 None admin] REQUEST_METHOD: `GET` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:51:16.996137 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bf059ea-562e-46a5-9b2e-7a25556c3e39 None admin] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:51:16.996405 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bf059ea-562e-46a5-9b2e-7a25556c3e39 None admin] PATH_INFO: `/v3/domains` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:51:16.997037 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bf059ea-562e-46a5-9b2e-7a25556c3e39 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81767) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 15:51:16.997333 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bf059ea-562e-46a5-9b2e-7a25556c3e39 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 15:51:16.998677 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bf059ea-562e-46a5-9b2e-7a25556c3e39 None admin] RBAC: Authorization granted {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 15:51:17.010137 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 604/1229] 10.222.0.15 () {56 vars in 1109 bytes} [Mon Aug 22 15:51:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 15:51:17.016023 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86fd70cf-799d-46ad-8840-66c47f08822a None admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 15:51:17.119833 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86fd70cf-799d-46ad-8840-66c47f08822a None admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 15:51:17.153992 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67f66570-9fbd-4497-afef-95bc3ec3ece2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 15:51:17.154608 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f66570-9fbd-4497-afef-95bc3ec3ece2 None admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:51:17.154882 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f66570-9fbd-4497-afef-95bc3ec3ece2 None admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:51:17.157386 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f66570-9fbd-4497-afef-95bc3ec3ece2 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:51:17.161722 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67f66570-9fbd-4497-afef-95bc3ec3ece2 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 15:51:17.163155 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67f66570-9fbd-4497-afef-95bc3ec3ece2 None admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 15:51:17.173970 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application [None req-67f66570-9fbd-4497-afef-95bc3ec3ece2 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 22 15:51:17.173970 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 15:51:17.173970 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 15:51:17.173970 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 15:51:17.173970 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 15:51:17.173970 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 15:51:17.173970 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 15:51:17.173970 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 15:51:17.173970 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 15:51:17.173970 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 15:51:17.173970 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 15:51:17.173970 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 15:51:17.173970 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 22 15:51:17.173970 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 22 15:51:17.173970 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 22 15:51:17.175048 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 22 15:51:17.175048 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 15:51:17.175048 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 15:51:17.175048 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 15:51:17.175048 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 15:51:17.175048 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 15:51:17.175048 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 15:51:17.175048 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 15:51:17.175048 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application with Lock( Aug 22 15:51:17.175048 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 15:51:17.175048 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self._enter() Aug 22 15:51:17.175048 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 15:51:17.175048 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 15:51:17.175048 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 15:51:17.175048 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self.creator() Aug 22 15:51:17.175048 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 15:51:17.175048 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application created_value = creator( Aug 22 15:51:17.176251 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 22 15:51:17.176251 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 22 15:51:17.176251 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 22 15:51:17.176251 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 22 15:51:17.176251 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 22 15:51:17.176251 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 22 15:51:17.176251 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 22 15:51:17.176251 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application Aug 22 15:51:17.176251 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 626/1230] 10.222.0.15 () {56 vars in 1111 bytes} [Mon Aug 22 15:51:17 2022] GET /identity/v3/projects/service => generated 88 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 22 15:51:17.181004 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bf059ea-562e-46a5-9b2e-7a25556c3e39 None admin] Authenticating user token {{(pid=81767) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 15:51:17.205436 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=100 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 15:51:17.285138 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bf059ea-562e-46a5-9b2e-7a25556c3e39 None admin] Validating token access rules against request {{(pid=81767) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 15:51:17.320285 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dc19e12-beaa-46ee-93e9-c8f91cba8510 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81767) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 15:51:17.320886 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dc19e12-beaa-46ee-93e9-c8f91cba8510 None admin] REQUEST_METHOD: `GET` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:51:17.321188 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dc19e12-beaa-46ee-93e9-c8f91cba8510 None admin] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:51:17.321453 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dc19e12-beaa-46ee-93e9-c8f91cba8510 None admin] PATH_INFO: `/v3/projects` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:51:17.322133 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dc19e12-beaa-46ee-93e9-c8f91cba8510 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81767) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 15:51:17.322430 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dc19e12-beaa-46ee-93e9-c8f91cba8510 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 15:51:17.323871 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dc19e12-beaa-46ee-93e9-c8f91cba8510 None admin] RBAC: Authorization granted {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 15:51:17.336366 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 605/1231] 10.222.0.15 () {56 vars in 1148 bytes} [Mon Aug 22 15:51:17 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 15:51:17.342416 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67f66570-9fbd-4497-afef-95bc3ec3ece2 None admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 15:51:17.447174 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67f66570-9fbd-4497-afef-95bc3ec3ece2 None admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 15:51:17.484414 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6781b683-881e-4462-92bb-a4b98ca3a263 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 15:51:17.485160 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6781b683-881e-4462-92bb-a4b98ca3a263 None admin] REQUEST_METHOD: `PUT` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:51:17.485442 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6781b683-881e-4462-92bb-a4b98ca3a263 None admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:51:17.485714 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6781b683-881e-4462-92bb-a4b98ca3a263 None admin] PATH_INFO: `/v3/projects/c4ee0413f9ef4d4fb1f95a660babe090/users/08159bf8edda4246bb463f7b11b0894b/roles/a5fbba1dc615490a96cde27e3a102eb6` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:51:17.501601 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6781b683-881e-4462-92bb-a4b98ca3a263 None admin] RBAC: Authorizing `identity:create_grant(project_id=c4ee0413f9ef4d4fb1f95a660babe090, user_id=08159bf8edda4246bb463f7b11b0894b, role_id=a5fbba1dc615490a96cde27e3a102eb6)` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 15:51:17.503206 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6781b683-881e-4462-92bb-a4b98ca3a263 None admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 15:51:17.928017 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=100 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 15:51:17.933644 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 627/1232] 10.222.0.15 () {58 vars in 1439 bytes} [Mon Aug 22 15:51:17 2022] PUT /identity/v3/projects/c4ee0413f9ef4d4fb1f95a660babe090/users/08159bf8edda4246bb463f7b11b0894b/roles/a5fbba1dc615490a96cde27e3a102eb6 => generated 0 bytes in 592 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 15:51:19.850636 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40b31763-e17a-42cd-a19a-6f4df8587876 None None] REQUEST_METHOD: `GET` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:51:19.851216 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40b31763-e17a-42cd-a19a-6f4df8587876 None None] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:51:19.851216 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40b31763-e17a-42cd-a19a-6f4df8587876 None None] PATH_INFO: `/` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:51:19.852561 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 606/1233] 10.222.0.15 () {54 vars in 927 bytes} [Mon Aug 22 15:51:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 15:51:19.858473 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7902f78c-282b-43f3-8ef3-138ccf602c37 None None] REQUEST_METHOD: `POST` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:51:19.858945 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7902f78c-282b-43f3-8ef3-138ccf602c37 None None] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:51:19.860362 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7902f78c-282b-43f3-8ef3-138ccf602c37 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:51:19.876926 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.auth.core [None req-7902f78c-282b-43f3-8ef3-138ccf602c37 None None] MFA Rules not processed for user `6b2199ae0fb0480aa917ada391f5ec54`. Rule list: `[]` (Enabled: `True`). {{(pid=81768) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 15:51:19.878122 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-7902f78c-282b-43f3-8ef3-138ccf602c37 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 15:51:19.946688 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 628/1234] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Aug 22 15:51:19 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 22 15:51:20.214139 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40b31763-e17a-42cd-a19a-6f4df8587876 None None] Authenticating user token {{(pid=81767) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 15:51:20.241444 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG dbcounter [-] [81767] Writing DB stats keystone:SELECT=100 {{(pid=81767) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 15:51:20.321952 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40b31763-e17a-42cd-a19a-6f4df8587876 None None] Validating token access rules against request {{(pid=81767) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 15:51:20.356562 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-032972af-e37a-4371-8dea-68ad3e1b17b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81767) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 15:51:20.357190 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-032972af-e37a-4371-8dea-68ad3e1b17b5 None admin] REQUEST_METHOD: `GET` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:51:20.357479 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-032972af-e37a-4371-8dea-68ad3e1b17b5 None admin] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:51:20.357737 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-032972af-e37a-4371-8dea-68ad3e1b17b5 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:51:20.368500 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-032972af-e37a-4371-8dea-68ad3e1b17b5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 22 15:51:20.373259 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application [None req-032972af-e37a-4371-8dea-68ad3e1b17b5 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 22 15:51:20.373259 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 15:51:20.373259 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 15:51:20.373259 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 15:51:20.373259 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 15:51:20.373259 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 15:51:20.373259 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 15:51:20.373259 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 15:51:20.373259 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 15:51:20.373259 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 15:51:20.373259 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 15:51:20.373259 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 15:51:20.373259 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 22 15:51:20.373259 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 22 15:51:20.373259 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 22 15:51:20.373259 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 22 15:51:20.380245 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 22 15:51:20.380245 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 22 15:51:20.380245 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 22 15:51:20.380245 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 22 15:51:20.380245 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 15:51:20.380245 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 15:51:20.380245 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 15:51:20.380245 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 15:51:20.380245 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 15:51:20.380245 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 15:51:20.380245 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 15:51:20.380245 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application with Lock( Aug 22 15:51:20.380245 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 15:51:20.380245 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return self._enter() Aug 22 15:51:20.380245 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 15:51:20.380245 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 15:51:20.381273 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 15:51:20.381273 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return self.creator() Aug 22 15:51:20.381273 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 15:51:20.381273 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application created_value = creator( Aug 22 15:51:20.381273 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 22 15:51:20.381273 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 22 15:51:20.381273 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 22 15:51:20.381273 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 22 15:51:20.381273 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 22 15:51:20.381273 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 22 15:51:20.381273 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 22 15:51:20.381273 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application Aug 22 15:51:20.381273 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 607/1235] 10.222.0.15 () {56 vars in 1102 bytes} [Mon Aug 22 15:51:20 2022] GET /identity/v3/roles/service => generated 85 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 15:51:20.382278 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7902f78c-282b-43f3-8ef3-138ccf602c37 None None] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 15:51:20.427539 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 15:51:20.480474 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7902f78c-282b-43f3-8ef3-138ccf602c37 None None] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 15:51:20.514608 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-327e1f42-3ec6-4319-b90d-5f12f8e23c1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 15:51:20.517920 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-327e1f42-3ec6-4319-b90d-5f12f8e23c1a None admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:51:20.517920 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-327e1f42-3ec6-4319-b90d-5f12f8e23c1a None admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:51:20.517920 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-327e1f42-3ec6-4319-b90d-5f12f8e23c1a None admin] PATH_INFO: `/v3/roles` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:51:20.517920 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-327e1f42-3ec6-4319-b90d-5f12f8e23c1a None admin] RBAC: Adding query filter params (name=service) {{(pid=81768) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 15:51:20.517920 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-327e1f42-3ec6-4319-b90d-5f12f8e23c1a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 15:51:20.518425 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-327e1f42-3ec6-4319-b90d-5f12f8e23c1a None admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 15:51:20.525931 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 629/1236] 10.222.0.15 () {56 vars in 1103 bytes} [Mon Aug 22 15:51:20 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 22 15:51:20.531904 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-032972af-e37a-4371-8dea-68ad3e1b17b5 None admin] Authenticating user token {{(pid=81767) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 15:51:20.619635 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG dbcounter [-] [81767] Writing DB stats keystone:SELECT=100 {{(pid=81767) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 15:51:20.635541 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-032972af-e37a-4371-8dea-68ad3e1b17b5 None admin] Validating token access rules against request {{(pid=81767) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 15:51:20.668663 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1d99d50-9019-4393-bd4c-a80e1141983b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81767) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 15:51:20.669284 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1d99d50-9019-4393-bd4c-a80e1141983b None admin] REQUEST_METHOD: `GET` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:51:20.669592 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1d99d50-9019-4393-bd4c-a80e1141983b None admin] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:51:20.669862 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1d99d50-9019-4393-bd4c-a80e1141983b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:51:20.681144 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1d99d50-9019-4393-bd4c-a80e1141983b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 15:51:20.682609 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1d99d50-9019-4393-bd4c-a80e1141983b None admin] RBAC: Authorization granted {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 15:51:20.699322 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application [None req-c1d99d50-9019-4393-bd4c-a80e1141983b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 22 15:51:20.699322 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 15:51:20.699322 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 15:51:20.699322 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 15:51:20.699322 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 15:51:20.699322 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 15:51:20.699322 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 15:51:20.699322 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 15:51:20.699322 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 15:51:20.699322 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 15:51:20.699322 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 15:51:20.699322 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 15:51:20.699322 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 22 15:51:20.699322 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 22 15:51:20.699322 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 22 15:51:20.699322 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 22 15:51:20.701410 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 15:51:20.701410 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 15:51:20.701410 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 15:51:20.701410 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 15:51:20.701410 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 15:51:20.701410 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 15:51:20.701410 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 15:51:20.701410 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application with Lock( Aug 22 15:51:20.701410 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 15:51:20.701410 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return self._enter() Aug 22 15:51:20.701410 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 15:51:20.701410 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 15:51:20.701410 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 15:51:20.701410 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return self.creator() Aug 22 15:51:20.701410 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 15:51:20.701410 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application created_value = creator( Aug 22 15:51:20.701410 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 22 15:51:20.702361 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 22 15:51:20.702361 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 22 15:51:20.702361 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application Aug 22 15:51:20.702361 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 608/1237] 10.222.0.15 () {56 vars in 1108 bytes} [Mon Aug 22 15:51:20 2022] GET /identity/v3/domains/Default => generated 87 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 22 15:51:20.707354 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-327e1f42-3ec6-4319-b90d-5f12f8e23c1a None admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 15:51:20.809640 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-327e1f42-3ec6-4319-b90d-5f12f8e23c1a None admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 15:51:20.818955 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=100 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 15:51:20.844257 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16b553e7-5316-4db3-9d34-bec2b577a198 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 15:51:20.844871 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16b553e7-5316-4db3-9d34-bec2b577a198 None admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:51:20.845149 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16b553e7-5316-4db3-9d34-bec2b577a198 None admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:51:20.845419 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16b553e7-5316-4db3-9d34-bec2b577a198 None admin] PATH_INFO: `/v3/domains` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:51:20.846064 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16b553e7-5316-4db3-9d34-bec2b577a198 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81768) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 15:51:20.846363 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16b553e7-5316-4db3-9d34-bec2b577a198 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 15:51:20.848318 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16b553e7-5316-4db3-9d34-bec2b577a198 None admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 15:51:20.860029 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 630/1238] 10.222.0.15 () {56 vars in 1109 bytes} [Mon Aug 22 15:51:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 15:51:20.866822 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1d99d50-9019-4393-bd4c-a80e1141983b None admin] Authenticating user token {{(pid=81767) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 15:51:20.969666 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1d99d50-9019-4393-bd4c-a80e1141983b None admin] Validating token access rules against request {{(pid=81767) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 15:51:21.004785 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19790191-122d-4dc2-903e-288b1f3c2c5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81767) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 15:51:21.005418 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19790191-122d-4dc2-903e-288b1f3c2c5f None admin] REQUEST_METHOD: `GET` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:51:21.005698 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19790191-122d-4dc2-903e-288b1f3c2c5f None admin] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:51:21.005971 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19790191-122d-4dc2-903e-288b1f3c2c5f None admin] PATH_INFO: `/v3/users/placement` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:51:21.008884 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG dbcounter [-] [81767] Writing DB stats keystone:SELECT=100 {{(pid=81767) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 15:51:21.014204 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19790191-122d-4dc2-903e-288b1f3c2c5f None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 15:51:21.015880 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19790191-122d-4dc2-903e-288b1f3c2c5f None admin] RBAC: Authorization granted {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 15:51:21.027636 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application [None req-19790191-122d-4dc2-903e-288b1f3c2c5f None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 22 15:51:21.027636 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 15:51:21.027636 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 15:51:21.027636 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 15:51:21.027636 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 15:51:21.027636 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 15:51:21.027636 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 15:51:21.027636 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 15:51:21.027636 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 15:51:21.027636 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 15:51:21.027636 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 15:51:21.027636 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 15:51:21.027636 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 22 15:51:21.027636 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 22 15:51:21.027636 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 22 15:51:21.027636 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 22 15:51:21.028847 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 15:51:21.028847 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 15:51:21.028847 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 22 15:51:21.028847 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 22 15:51:21.028847 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 22 15:51:21.028847 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 22 15:51:21.028847 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 15:51:21.028847 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 15:51:21.028847 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 15:51:21.028847 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 15:51:21.028847 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 15:51:21.028847 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application with Lock( Aug 22 15:51:21.028847 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 15:51:21.028847 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return self._enter() Aug 22 15:51:21.028847 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 15:51:21.028847 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 15:51:21.028847 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 15:51:21.028847 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return self.creator() Aug 22 15:51:21.029788 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 15:51:21.029788 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application created_value = creator( Aug 22 15:51:21.029788 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 22 15:51:21.029788 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 22 15:51:21.029788 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 22 15:51:21.029788 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 22 15:51:21.029788 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 22 15:51:21.029788 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 22 15:51:21.029788 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 22 15:51:21.029788 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application Aug 22 15:51:21.029788 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 609/1239] 10.222.0.15 () {56 vars in 1108 bytes} [Mon Aug 22 15:51:20 2022] GET /identity/v3/users/placement => generated 87 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 15:51:21.035894 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16b553e7-5316-4db3-9d34-bec2b577a198 None admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 15:51:21.140022 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16b553e7-5316-4db3-9d34-bec2b577a198 None admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 15:51:21.174605 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79db3c14-16d7-407b-98a1-1d1fa037cc20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 15:51:21.175202 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79db3c14-16d7-407b-98a1-1d1fa037cc20 None admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:51:21.175514 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79db3c14-16d7-407b-98a1-1d1fa037cc20 None admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:51:21.175802 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79db3c14-16d7-407b-98a1-1d1fa037cc20 None admin] PATH_INFO: `/v3/users` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:51:21.176518 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79db3c14-16d7-407b-98a1-1d1fa037cc20 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81768) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 15:51:21.176819 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79db3c14-16d7-407b-98a1-1d1fa037cc20 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 15:51:21.178230 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79db3c14-16d7-407b-98a1-1d1fa037cc20 None admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 15:51:21.196915 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 631/1240] 10.222.0.15 () {56 vars in 1143 bytes} [Mon Aug 22 15:51:21 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 15:51:21.202866 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19790191-122d-4dc2-903e-288b1f3c2c5f None admin] Authenticating user token {{(pid=81767) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 15:51:21.307337 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19790191-122d-4dc2-903e-288b1f3c2c5f None admin] Validating token access rules against request {{(pid=81767) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 15:51:21.345064 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17058afb-3595-468c-b948-431a7386a838 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81767) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 15:51:21.345672 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17058afb-3595-468c-b948-431a7386a838 None admin] REQUEST_METHOD: `GET` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:51:21.345949 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17058afb-3595-468c-b948-431a7386a838 None admin] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:51:21.346219 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17058afb-3595-468c-b948-431a7386a838 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:51:21.409858 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG dbcounter [-] [81767] Writing DB stats keystone:SELECT=100 {{(pid=81767) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 15:51:21.416791 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17058afb-3595-468c-b948-431a7386a838 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 15:51:21.418228 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17058afb-3595-468c-b948-431a7386a838 None admin] RBAC: Authorization granted {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 15:51:21.514156 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 610/1241] 10.222.0.15 () {58 vars in 1282 bytes} [Mon Aug 22 15:51:21 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 22 15:51:21.520682 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79db3c14-16d7-407b-98a1-1d1fa037cc20 None admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 15:51:21.548305 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=100 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 15:51:21.630275 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79db3c14-16d7-407b-98a1-1d1fa037cc20 None admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 15:51:21.667973 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6af3b177-6651-46b4-8e61-9e7faf7b0e0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 15:51:21.668533 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6af3b177-6651-46b4-8e61-9e7faf7b0e0c None admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:51:21.668796 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6af3b177-6651-46b4-8e61-9e7faf7b0e0c None admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:51:21.669166 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6af3b177-6651-46b4-8e61-9e7faf7b0e0c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:51:21.680182 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6af3b177-6651-46b4-8e61-9e7faf7b0e0c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 15:51:21.681660 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6af3b177-6651-46b4-8e61-9e7faf7b0e0c None admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 15:51:21.698734 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application [None req-6af3b177-6651-46b4-8e61-9e7faf7b0e0c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 22 15:51:21.698734 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 15:51:21.698734 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 15:51:21.698734 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 15:51:21.698734 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 15:51:21.698734 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 15:51:21.698734 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 15:51:21.698734 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 15:51:21.698734 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 15:51:21.698734 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 15:51:21.698734 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 15:51:21.698734 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 15:51:21.698734 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 22 15:51:21.698734 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 22 15:51:21.698734 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 22 15:51:21.698734 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 22 15:51:21.700224 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 15:51:21.700224 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 15:51:21.700224 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 15:51:21.700224 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 15:51:21.700224 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 15:51:21.700224 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 15:51:21.700224 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 15:51:21.700224 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application with Lock( Aug 22 15:51:21.700224 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 15:51:21.700224 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self._enter() Aug 22 15:51:21.700224 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 15:51:21.700224 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 15:51:21.700224 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 15:51:21.700224 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self.creator() Aug 22 15:51:21.700224 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 15:51:21.700224 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application created_value = creator( Aug 22 15:51:21.700224 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 22 15:51:21.701553 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 22 15:51:21.701553 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 22 15:51:21.701553 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application Aug 22 15:51:21.703486 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 632/1242] 10.222.0.15 () {56 vars in 1108 bytes} [Mon Aug 22 15:51:21 2022] GET /identity/v3/domains/Default => generated 87 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 15:51:21.705806 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17058afb-3595-468c-b948-431a7386a838 None admin] Authenticating user token {{(pid=81767) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 15:51:21.806803 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG dbcounter [-] [81767] Writing DB stats keystone:SELECT=100 {{(pid=81767) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 15:51:21.812814 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17058afb-3595-468c-b948-431a7386a838 None admin] Validating token access rules against request {{(pid=81767) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 15:51:21.847695 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89fdf9d4-15f0-4a19-a6bc-3bf1b94056b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81767) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 15:51:21.848297 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89fdf9d4-15f0-4a19-a6bc-3bf1b94056b4 None admin] REQUEST_METHOD: `GET` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:51:21.848574 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89fdf9d4-15f0-4a19-a6bc-3bf1b94056b4 None admin] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:51:21.848842 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89fdf9d4-15f0-4a19-a6bc-3bf1b94056b4 None admin] PATH_INFO: `/v3/domains` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:51:21.849487 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89fdf9d4-15f0-4a19-a6bc-3bf1b94056b4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81767) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 15:51:21.849781 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89fdf9d4-15f0-4a19-a6bc-3bf1b94056b4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 15:51:21.851127 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89fdf9d4-15f0-4a19-a6bc-3bf1b94056b4 None admin] RBAC: Authorization granted {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 15:51:21.862235 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 611/1243] 10.222.0.15 () {56 vars in 1109 bytes} [Mon Aug 22 15:51:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 15:51:21.868125 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6af3b177-6651-46b4-8e61-9e7faf7b0e0c None admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 15:51:21.955759 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=100 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 15:51:21.978688 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6af3b177-6651-46b4-8e61-9e7faf7b0e0c None admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 15:51:22.014378 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85242d7f-d85f-4fb3-80ae-1e1d5c99889d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 15:51:22.015063 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85242d7f-d85f-4fb3-80ae-1e1d5c99889d None admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:51:22.015370 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85242d7f-d85f-4fb3-80ae-1e1d5c99889d None admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:51:22.015643 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85242d7f-d85f-4fb3-80ae-1e1d5c99889d None admin] PATH_INFO: `/v3/projects/service` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:51:22.022615 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85242d7f-d85f-4fb3-80ae-1e1d5c99889d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 15:51:22.024184 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85242d7f-d85f-4fb3-80ae-1e1d5c99889d None admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 15:51:22.035781 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application [None req-85242d7f-d85f-4fb3-80ae-1e1d5c99889d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 22 15:51:22.035781 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 15:51:22.035781 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 15:51:22.035781 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 15:51:22.035781 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 15:51:22.035781 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 15:51:22.035781 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 15:51:22.035781 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 15:51:22.035781 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 15:51:22.035781 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 15:51:22.035781 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 15:51:22.035781 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 15:51:22.035781 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 22 15:51:22.035781 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 22 15:51:22.035781 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 22 15:51:22.036890 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 22 15:51:22.036890 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 15:51:22.036890 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 15:51:22.036890 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 15:51:22.036890 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 15:51:22.036890 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 15:51:22.036890 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 15:51:22.036890 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 15:51:22.036890 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application with Lock( Aug 22 15:51:22.036890 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 15:51:22.036890 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self._enter() Aug 22 15:51:22.036890 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 15:51:22.036890 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 15:51:22.036890 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 15:51:22.036890 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self.creator() Aug 22 15:51:22.036890 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 15:51:22.036890 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application created_value = creator( Aug 22 15:51:22.037907 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 22 15:51:22.037907 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 22 15:51:22.037907 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 22 15:51:22.037907 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 22 15:51:22.037907 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 22 15:51:22.037907 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 22 15:51:22.037907 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 22 15:51:22.037907 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application Aug 22 15:51:22.037907 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 633/1244] 10.222.0.15 () {56 vars in 1111 bytes} [Mon Aug 22 15:51:21 2022] GET /identity/v3/projects/service => generated 88 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 15:51:22.042973 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89fdf9d4-15f0-4a19-a6bc-3bf1b94056b4 None admin] Authenticating user token {{(pid=81767) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 15:51:22.152292 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89fdf9d4-15f0-4a19-a6bc-3bf1b94056b4 None admin] Validating token access rules against request {{(pid=81767) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 15:51:22.187464 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a02d967-3527-4f28-a7d2-9a81b1de1330 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81767) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 15:51:22.196641 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a02d967-3527-4f28-a7d2-9a81b1de1330 None admin] REQUEST_METHOD: `GET` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:51:22.196641 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a02d967-3527-4f28-a7d2-9a81b1de1330 None admin] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:51:22.196641 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a02d967-3527-4f28-a7d2-9a81b1de1330 None admin] PATH_INFO: `/v3/projects` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:51:22.196641 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a02d967-3527-4f28-a7d2-9a81b1de1330 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81767) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 15:51:22.196641 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a02d967-3527-4f28-a7d2-9a81b1de1330 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 15:51:22.196641 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a02d967-3527-4f28-a7d2-9a81b1de1330 None admin] RBAC: Authorization granted {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 15:51:22.204078 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 612/1245] 10.222.0.15 () {56 vars in 1148 bytes} [Mon Aug 22 15:51:22 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 15:51:22.210720 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85242d7f-d85f-4fb3-80ae-1e1d5c99889d None admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 15:51:22.318988 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85242d7f-d85f-4fb3-80ae-1e1d5c99889d None admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 15:51:22.353162 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6cdaa7c-1e90-44a1-a848-4743d4e87ab8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 15:51:22.353768 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6cdaa7c-1e90-44a1-a848-4743d4e87ab8 None admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:51:22.354169 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6cdaa7c-1e90-44a1-a848-4743d4e87ab8 None admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:51:22.354310 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6cdaa7c-1e90-44a1-a848-4743d4e87ab8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:51:22.355044 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6cdaa7c-1e90-44a1-a848-4743d4e87ab8 None admin] RBAC: Adding query filter params (role.id=a5fbba1dc615490a96cde27e3a102eb6, scope.project.id=c4ee0413f9ef4d4fb1f95a660babe090, user.id=08159bf8edda4246bb463f7b11b0894b) {{(pid=81768) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 15:51:22.355390 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6cdaa7c-1e90-44a1-a848-4743d4e87ab8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 15:51:22.357056 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6cdaa7c-1e90-44a1-a848-4743d4e87ab8 None admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 15:51:22.362975 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 634/1246] 10.222.0.15 () {56 vars in 1374 bytes} [Mon Aug 22 15:51:22 2022] GET /identity/v3/role_assignments?user.id=08159bf8edda4246bb463f7b11b0894b&scope.project.id=c4ee0413f9ef4d4fb1f95a660babe090&role.id=a5fbba1dc615490a96cde27e3a102eb6 => generated 608 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 15:51:24.387783 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-255bc596-51ab-434a-a365-8f55c4c46d29 None None] REQUEST_METHOD: `GET` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:51:24.388300 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-255bc596-51ab-434a-a365-8f55c4c46d29 None None] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:51:24.388300 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-255bc596-51ab-434a-a365-8f55c4c46d29 None None] PATH_INFO: `/` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:51:24.389604 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 613/1247] 10.222.0.15 () {54 vars in 927 bytes} [Mon Aug 22 15:51:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 15:51:24.395062 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acac698b-bc61-42ba-9f0c-1651320adf29 None None] REQUEST_METHOD: `POST` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:51:24.395541 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acac698b-bc61-42ba-9f0c-1651320adf29 None None] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:51:24.395631 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acac698b-bc61-42ba-9f0c-1651320adf29 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:51:24.401866 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=100 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 15:51:24.413901 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.auth.core [None req-acac698b-bc61-42ba-9f0c-1651320adf29 None None] MFA Rules not processed for user `6b2199ae0fb0480aa917ada391f5ec54`. Rule list: `[]` (Enabled: `True`). {{(pid=81768) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 15:51:24.415080 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-acac698b-bc61-42ba-9f0c-1651320adf29 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 15:51:24.484982 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 635/1248] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Aug 22 15:51:24 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 22 15:51:24.759462 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-255bc596-51ab-434a-a365-8f55c4c46d29 None None] Authenticating user token {{(pid=81767) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 15:51:24.777959 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG dbcounter [-] [81767] Writing DB stats keystone:SELECT=100 {{(pid=81767) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 15:51:24.864866 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-255bc596-51ab-434a-a365-8f55c4c46d29 None None] Validating token access rules against request {{(pid=81767) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 15:51:24.905494 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1785149-7024-4b93-b26e-969da1a45fb6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81767) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 15:51:24.906141 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1785149-7024-4b93-b26e-969da1a45fb6 None admin] REQUEST_METHOD: `GET` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:51:24.906444 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1785149-7024-4b93-b26e-969da1a45fb6 None admin] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:51:24.906709 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1785149-7024-4b93-b26e-969da1a45fb6 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:51:24.919593 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1785149-7024-4b93-b26e-969da1a45fb6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 22 15:51:24.924370 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application [None req-f1785149-7024-4b93-b26e-969da1a45fb6 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 22 15:51:24.924370 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 15:51:24.924370 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 15:51:24.924370 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 15:51:24.924370 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 15:51:24.924370 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 15:51:24.924370 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 15:51:24.924370 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 15:51:24.924370 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 15:51:24.924370 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 15:51:24.924370 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 15:51:24.924370 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 15:51:24.924370 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 22 15:51:24.924370 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 22 15:51:24.924370 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 22 15:51:24.924370 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 22 15:51:24.925459 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 22 15:51:24.925459 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 22 15:51:24.925459 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 22 15:51:24.925459 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 22 15:51:24.925459 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 15:51:24.925459 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 15:51:24.925459 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 15:51:24.925459 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 15:51:24.925459 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 15:51:24.925459 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 15:51:24.925459 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 15:51:24.925459 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application with Lock( Aug 22 15:51:24.925459 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 15:51:24.925459 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return self._enter() Aug 22 15:51:24.925459 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 15:51:24.925459 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 15:51:24.926418 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 15:51:24.926418 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return self.creator() Aug 22 15:51:24.926418 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 15:51:24.926418 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application created_value = creator( Aug 22 15:51:24.926418 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 22 15:51:24.926418 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 22 15:51:24.926418 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 22 15:51:24.926418 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 22 15:51:24.926418 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 22 15:51:24.926418 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 22 15:51:24.926418 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 22 15:51:24.926418 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application Aug 22 15:51:24.926418 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 614/1249] 10.222.0.15 () {56 vars in 1096 bytes} [Mon Aug 22 15:51:24 2022] GET /identity/v3/roles/admin => generated 83 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 15:51:24.931911 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acac698b-bc61-42ba-9f0c-1651320adf29 None None] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 15:51:25.038828 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acac698b-bc61-42ba-9f0c-1651320adf29 None None] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 15:51:25.072680 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=100 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 15:51:25.077853 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acb3878d-1d65-4ca5-a1be-5ca5a106b7b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 15:51:25.078493 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acb3878d-1d65-4ca5-a1be-5ca5a106b7b3 None admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:51:25.078732 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acb3878d-1d65-4ca5-a1be-5ca5a106b7b3 None admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:51:25.079000 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acb3878d-1d65-4ca5-a1be-5ca5a106b7b3 None admin] PATH_INFO: `/v3/roles` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:51:25.079696 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acb3878d-1d65-4ca5-a1be-5ca5a106b7b3 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81768) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 15:51:25.079994 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acb3878d-1d65-4ca5-a1be-5ca5a106b7b3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 15:51:25.081460 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acb3878d-1d65-4ca5-a1be-5ca5a106b7b3 None admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 15:51:25.090434 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 636/1250] 10.222.0.15 () {56 vars in 1099 bytes} [Mon Aug 22 15:51:24 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 15:51:25.096160 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1785149-7024-4b93-b26e-969da1a45fb6 None admin] Authenticating user token {{(pid=81767) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 15:51:25.177000 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG dbcounter [-] [81767] Writing DB stats keystone:SELECT=100 {{(pid=81767) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 15:51:25.200735 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1785149-7024-4b93-b26e-969da1a45fb6 None admin] Validating token access rules against request {{(pid=81767) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 15:51:25.237711 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c8162ae-27d8-4ba1-ae8f-e5dfa92ddcc0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81767) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 15:51:25.238469 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c8162ae-27d8-4ba1-ae8f-e5dfa92ddcc0 None admin] REQUEST_METHOD: `GET` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:51:25.238750 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c8162ae-27d8-4ba1-ae8f-e5dfa92ddcc0 None admin] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:51:25.239022 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c8162ae-27d8-4ba1-ae8f-e5dfa92ddcc0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:51:25.249622 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c8162ae-27d8-4ba1-ae8f-e5dfa92ddcc0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 15:51:25.251071 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c8162ae-27d8-4ba1-ae8f-e5dfa92ddcc0 None admin] RBAC: Authorization granted {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 15:51:25.266039 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application [None req-6c8162ae-27d8-4ba1-ae8f-e5dfa92ddcc0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 22 15:51:25.266039 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 15:51:25.266039 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 15:51:25.266039 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 15:51:25.266039 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 15:51:25.266039 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 15:51:25.266039 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 15:51:25.266039 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 15:51:25.266039 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 15:51:25.266039 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 15:51:25.266039 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 15:51:25.266039 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 15:51:25.266039 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 22 15:51:25.266039 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 22 15:51:25.266039 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 22 15:51:25.266039 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 22 15:51:25.274344 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 15:51:25.274344 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 15:51:25.274344 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 15:51:25.274344 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 15:51:25.274344 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 15:51:25.274344 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 15:51:25.274344 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 15:51:25.274344 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application with Lock( Aug 22 15:51:25.274344 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 15:51:25.274344 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return self._enter() Aug 22 15:51:25.274344 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 15:51:25.274344 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 15:51:25.274344 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 15:51:25.274344 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return self.creator() Aug 22 15:51:25.274344 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 15:51:25.274344 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application created_value = creator( Aug 22 15:51:25.274344 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 22 15:51:25.275477 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 22 15:51:25.275477 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 22 15:51:25.275477 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application Aug 22 15:51:25.275477 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 615/1251] 10.222.0.15 () {56 vars in 1108 bytes} [Mon Aug 22 15:51:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 15:51:25.275785 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acb3878d-1d65-4ca5-a1be-5ca5a106b7b3 None admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 15:51:25.380412 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acb3878d-1d65-4ca5-a1be-5ca5a106b7b3 None admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 15:51:25.421355 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23e0de86-c5cf-487c-8b37-50e12aae3e96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 15:51:25.422112 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23e0de86-c5cf-487c-8b37-50e12aae3e96 None admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:51:25.422391 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23e0de86-c5cf-487c-8b37-50e12aae3e96 None admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:51:25.422661 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23e0de86-c5cf-487c-8b37-50e12aae3e96 None admin] PATH_INFO: `/v3/domains` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:51:25.423322 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23e0de86-c5cf-487c-8b37-50e12aae3e96 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81768) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 15:51:25.423625 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23e0de86-c5cf-487c-8b37-50e12aae3e96 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 15:51:25.424985 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23e0de86-c5cf-487c-8b37-50e12aae3e96 None admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 15:51:25.436888 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 637/1252] 10.222.0.15 () {56 vars in 1109 bytes} [Mon Aug 22 15:51:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 15:51:25.443085 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c8162ae-27d8-4ba1-ae8f-e5dfa92ddcc0 None admin] Authenticating user token {{(pid=81767) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 15:51:25.543547 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c8162ae-27d8-4ba1-ae8f-e5dfa92ddcc0 None admin] Validating token access rules against request {{(pid=81767) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 15:51:25.569743 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG dbcounter [-] [81767] Writing DB stats keystone:SELECT=100 {{(pid=81767) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 15:51:25.577843 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-900e59c5-6675-4686-ac15-07a812c71ccf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81767) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 15:51:25.578481 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-900e59c5-6675-4686-ac15-07a812c71ccf None admin] REQUEST_METHOD: `GET` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:51:25.578757 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-900e59c5-6675-4686-ac15-07a812c71ccf None admin] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:51:25.579029 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-900e59c5-6675-4686-ac15-07a812c71ccf None admin] PATH_INFO: `/v3/users/placement` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:51:25.587361 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-900e59c5-6675-4686-ac15-07a812c71ccf None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 15:51:25.588832 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-900e59c5-6675-4686-ac15-07a812c71ccf None admin] RBAC: Authorization granted {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 15:51:25.599841 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application [None req-900e59c5-6675-4686-ac15-07a812c71ccf None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 22 15:51:25.599841 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 15:51:25.599841 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 15:51:25.599841 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 15:51:25.599841 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 15:51:25.599841 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 15:51:25.599841 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 15:51:25.599841 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 15:51:25.599841 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 15:51:25.599841 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 15:51:25.599841 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 15:51:25.599841 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 15:51:25.599841 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 22 15:51:25.599841 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 22 15:51:25.599841 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 22 15:51:25.599841 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 22 15:51:25.605547 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 15:51:25.605547 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 15:51:25.605547 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 22 15:51:25.605547 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 22 15:51:25.605547 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 22 15:51:25.605547 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 22 15:51:25.605547 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 15:51:25.605547 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 15:51:25.605547 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 15:51:25.605547 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 15:51:25.605547 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 15:51:25.605547 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application with Lock( Aug 22 15:51:25.605547 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 15:51:25.605547 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return self._enter() Aug 22 15:51:25.605547 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 15:51:25.605547 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 15:51:25.605547 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 15:51:25.605547 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return self.creator() Aug 22 15:51:25.606697 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 15:51:25.606697 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application created_value = creator( Aug 22 15:51:25.606697 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 22 15:51:25.606697 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 22 15:51:25.606697 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 22 15:51:25.606697 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 22 15:51:25.606697 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 22 15:51:25.606697 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 22 15:51:25.606697 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 22 15:51:25.606697 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application Aug 22 15:51:25.606697 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 616/1253] 10.222.0.15 () {56 vars in 1108 bytes} [Mon Aug 22 15:51:25 2022] GET /identity/v3/users/placement => generated 87 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 15:51:25.608805 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23e0de86-c5cf-487c-8b37-50e12aae3e96 None admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 15:51:25.665356 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=100 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 15:51:25.710661 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23e0de86-c5cf-487c-8b37-50e12aae3e96 None admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 15:51:25.749724 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cc46841-fd3f-43bc-a541-3b4464060492 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 15:51:25.750327 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cc46841-fd3f-43bc-a541-3b4464060492 None admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:51:25.750603 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cc46841-fd3f-43bc-a541-3b4464060492 None admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:51:25.750863 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cc46841-fd3f-43bc-a541-3b4464060492 None admin] PATH_INFO: `/v3/users` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:51:25.751618 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cc46841-fd3f-43bc-a541-3b4464060492 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81768) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 15:51:25.751914 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cc46841-fd3f-43bc-a541-3b4464060492 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 15:51:25.753322 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cc46841-fd3f-43bc-a541-3b4464060492 None admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 15:51:25.770680 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 638/1254] 10.222.0.15 () {56 vars in 1143 bytes} [Mon Aug 22 15:51:25 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 15:51:25.776694 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-900e59c5-6675-4686-ac15-07a812c71ccf None admin] Authenticating user token {{(pid=81767) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 15:51:25.881237 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-900e59c5-6675-4686-ac15-07a812c71ccf None admin] Validating token access rules against request {{(pid=81767) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 15:51:25.915081 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c5272a0-6b3b-44e5-8187-f100c4176464 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81767) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 15:51:25.915767 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c5272a0-6b3b-44e5-8187-f100c4176464 None admin] REQUEST_METHOD: `GET` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:51:25.916104 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c5272a0-6b3b-44e5-8187-f100c4176464 None admin] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:51:25.916385 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c5272a0-6b3b-44e5-8187-f100c4176464 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:51:25.974401 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG dbcounter [-] [81767] Writing DB stats keystone:SELECT=100 {{(pid=81767) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 15:51:25.986544 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c5272a0-6b3b-44e5-8187-f100c4176464 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 15:51:25.988011 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c5272a0-6b3b-44e5-8187-f100c4176464 None admin] RBAC: Authorization granted {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 15:51:26.092153 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 617/1255] 10.222.0.15 () {58 vars in 1282 bytes} [Mon Aug 22 15:51:25 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 316 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 22 15:51:26.103268 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cc46841-fd3f-43bc-a541-3b4464060492 None admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 15:51:26.216311 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cc46841-fd3f-43bc-a541-3b4464060492 None admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 15:51:26.231739 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=100 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 15:51:26.255619 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-611f357b-1f63-45e2-b354-e8f07cd62ced None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 15:51:26.256375 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-611f357b-1f63-45e2-b354-e8f07cd62ced None admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:51:26.256735 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-611f357b-1f63-45e2-b354-e8f07cd62ced None admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:51:26.257125 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-611f357b-1f63-45e2-b354-e8f07cd62ced None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:51:26.268667 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-611f357b-1f63-45e2-b354-e8f07cd62ced None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 15:51:26.270192 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-611f357b-1f63-45e2-b354-e8f07cd62ced None admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 15:51:26.286473 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application [None req-611f357b-1f63-45e2-b354-e8f07cd62ced None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 22 15:51:26.286473 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 15:51:26.286473 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 15:51:26.286473 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 15:51:26.286473 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 15:51:26.286473 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 15:51:26.286473 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 15:51:26.286473 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 15:51:26.286473 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 15:51:26.286473 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 15:51:26.286473 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 15:51:26.286473 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 15:51:26.286473 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 22 15:51:26.286473 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 22 15:51:26.286473 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 22 15:51:26.286473 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 22 15:51:26.288432 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 15:51:26.288432 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 15:51:26.288432 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 15:51:26.288432 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 15:51:26.288432 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 15:51:26.288432 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 15:51:26.288432 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 15:51:26.288432 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application with Lock( Aug 22 15:51:26.288432 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 15:51:26.288432 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self._enter() Aug 22 15:51:26.288432 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 15:51:26.288432 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 15:51:26.288432 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 15:51:26.288432 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self.creator() Aug 22 15:51:26.288432 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 15:51:26.288432 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application created_value = creator( Aug 22 15:51:26.288432 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 22 15:51:26.290252 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 22 15:51:26.290252 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 22 15:51:26.290252 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application Aug 22 15:51:26.290252 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 639/1256] 10.222.0.15 () {56 vars in 1108 bytes} [Mon Aug 22 15:51:26 2022] GET /identity/v3/domains/Default => generated 87 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 22 15:51:26.297519 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c5272a0-6b3b-44e5-8187-f100c4176464 None admin] Authenticating user token {{(pid=81767) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 15:51:26.398680 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG dbcounter [-] [81767] Writing DB stats keystone:SELECT=100 {{(pid=81767) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 15:51:26.411017 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c5272a0-6b3b-44e5-8187-f100c4176464 None admin] Validating token access rules against request {{(pid=81767) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 15:51:26.445889 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea390906-f9b4-4e6e-9d87-9ae71e4dca50 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81767) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 15:51:26.446610 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea390906-f9b4-4e6e-9d87-9ae71e4dca50 None admin] REQUEST_METHOD: `GET` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:51:26.446752 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea390906-f9b4-4e6e-9d87-9ae71e4dca50 None admin] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:51:26.447014 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea390906-f9b4-4e6e-9d87-9ae71e4dca50 None admin] PATH_INFO: `/v3/domains` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:51:26.447680 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea390906-f9b4-4e6e-9d87-9ae71e4dca50 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81767) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 15:51:26.447974 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea390906-f9b4-4e6e-9d87-9ae71e4dca50 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 15:51:26.449632 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea390906-f9b4-4e6e-9d87-9ae71e4dca50 None admin] RBAC: Authorization granted {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 15:51:26.461929 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 618/1257] 10.222.0.15 () {56 vars in 1109 bytes} [Mon Aug 22 15:51:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 15:51:26.467855 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-611f357b-1f63-45e2-b354-e8f07cd62ced None admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 15:51:26.568769 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-611f357b-1f63-45e2-b354-e8f07cd62ced None admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 15:51:26.603620 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7b5cd66-ef5c-454b-bdd5-7359c6ce5a87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 15:51:26.604240 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7b5cd66-ef5c-454b-bdd5-7359c6ce5a87 None admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:51:26.604516 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7b5cd66-ef5c-454b-bdd5-7359c6ce5a87 None admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:51:26.604781 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7b5cd66-ef5c-454b-bdd5-7359c6ce5a87 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:51:26.611321 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7b5cd66-ef5c-454b-bdd5-7359c6ce5a87 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 15:51:26.612770 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7b5cd66-ef5c-454b-bdd5-7359c6ce5a87 None admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 15:51:26.624884 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application [None req-e7b5cd66-ef5c-454b-bdd5-7359c6ce5a87 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 22 15:51:26.624884 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 15:51:26.624884 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 15:51:26.624884 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 15:51:26.624884 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 15:51:26.624884 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 15:51:26.624884 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 15:51:26.624884 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 15:51:26.624884 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 15:51:26.624884 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 15:51:26.624884 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 15:51:26.624884 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 15:51:26.624884 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 22 15:51:26.624884 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 22 15:51:26.624884 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 22 15:51:26.626276 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 22 15:51:26.626276 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 15:51:26.626276 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 15:51:26.626276 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 15:51:26.626276 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 15:51:26.626276 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 15:51:26.626276 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 15:51:26.626276 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 15:51:26.626276 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application with Lock( Aug 22 15:51:26.626276 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 15:51:26.626276 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self._enter() Aug 22 15:51:26.626276 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 15:51:26.626276 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 15:51:26.626276 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 15:51:26.626276 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self.creator() Aug 22 15:51:26.626276 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 15:51:26.626276 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application created_value = creator( Aug 22 15:51:26.627411 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 22 15:51:26.627411 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 22 15:51:26.627411 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 22 15:51:26.627411 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 22 15:51:26.627411 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 22 15:51:26.627411 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 22 15:51:26.627411 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 22 15:51:26.627411 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application Aug 22 15:51:26.627411 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 640/1258] 10.222.0.15 () {56 vars in 1111 bytes} [Mon Aug 22 15:51:26 2022] GET /identity/v3/projects/service => generated 88 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 15:51:26.632091 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea390906-f9b4-4e6e-9d87-9ae71e4dca50 None admin] Authenticating user token {{(pid=81767) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 15:51:26.737651 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea390906-f9b4-4e6e-9d87-9ae71e4dca50 None admin] Validating token access rules against request {{(pid=81767) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 15:51:26.772458 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf8d72ac-152e-4ffd-afb6-51e6bea93a22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81767) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 15:51:26.773056 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf8d72ac-152e-4ffd-afb6-51e6bea93a22 None admin] REQUEST_METHOD: `GET` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:51:26.773328 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf8d72ac-152e-4ffd-afb6-51e6bea93a22 None admin] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:51:26.773594 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf8d72ac-152e-4ffd-afb6-51e6bea93a22 None admin] PATH_INFO: `/v3/projects` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:51:26.774276 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf8d72ac-152e-4ffd-afb6-51e6bea93a22 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81767) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 15:51:26.774578 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf8d72ac-152e-4ffd-afb6-51e6bea93a22 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 15:51:26.776011 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf8d72ac-152e-4ffd-afb6-51e6bea93a22 None admin] RBAC: Authorization granted {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 15:51:26.788190 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 619/1259] 10.222.0.15 () {56 vars in 1148 bytes} [Mon Aug 22 15:51:26 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 15:51:26.794416 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7b5cd66-ef5c-454b-bdd5-7359c6ce5a87 None admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 15:51:26.822325 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=100 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 15:51:26.904234 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7b5cd66-ef5c-454b-bdd5-7359c6ce5a87 None admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 15:51:26.939851 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e88598b-9305-4d3c-81b4-64c3bb290a8c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 15:51:26.940450 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e88598b-9305-4d3c-81b4-64c3bb290a8c None admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:51:26.940729 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e88598b-9305-4d3c-81b4-64c3bb290a8c None admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:51:26.940999 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e88598b-9305-4d3c-81b4-64c3bb290a8c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:51:26.942215 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e88598b-9305-4d3c-81b4-64c3bb290a8c None admin] RBAC: Adding query filter params (role.id=9d9848cfd22e4b089db443c350e56670, scope.project.id=c4ee0413f9ef4d4fb1f95a660babe090, user.id=08159bf8edda4246bb463f7b11b0894b) {{(pid=81768) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 15:51:26.942518 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e88598b-9305-4d3c-81b4-64c3bb290a8c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 15:51:26.944144 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e88598b-9305-4d3c-81b4-64c3bb290a8c None admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 15:51:26.949753 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 641/1260] 10.222.0.15 () {56 vars in 1374 bytes} [Mon Aug 22 15:51:26 2022] GET /identity/v3/role_assignments?user.id=08159bf8edda4246bb463f7b11b0894b&scope.project.id=c4ee0413f9ef4d4fb1f95a660babe090&role.id=9d9848cfd22e4b089db443c350e56670 => generated 259 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 15:51:28.863259 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e4bb6dc-b0bd-465e-ab26-f6e28a88de61 None None] REQUEST_METHOD: `GET` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:51:28.863833 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e4bb6dc-b0bd-465e-ab26-f6e28a88de61 None None] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:51:28.863833 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e4bb6dc-b0bd-465e-ab26-f6e28a88de61 None None] PATH_INFO: `/` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:51:28.865180 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 620/1261] 10.222.0.15 () {54 vars in 927 bytes} [Mon Aug 22 15:51:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 15:51:28.870628 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b10f8727-0b5a-4c1a-bbbb-491b9078f288 None None] REQUEST_METHOD: `POST` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:51:28.871070 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b10f8727-0b5a-4c1a-bbbb-491b9078f288 None None] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:51:28.871164 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b10f8727-0b5a-4c1a-bbbb-491b9078f288 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:51:28.890145 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.auth.core [None req-b10f8727-0b5a-4c1a-bbbb-491b9078f288 None None] MFA Rules not processed for user `6b2199ae0fb0480aa917ada391f5ec54`. Rule list: `[]` (Enabled: `True`). {{(pid=81768) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 15:51:28.891405 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-b10f8727-0b5a-4c1a-bbbb-491b9078f288 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 15:51:28.964902 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 642/1262] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Aug 22 15:51:28 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 22 15:51:29.245088 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e4bb6dc-b0bd-465e-ab26-f6e28a88de61 None None] Authenticating user token {{(pid=81767) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 15:51:29.257033 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG dbcounter [-] [81767] Writing DB stats keystone:SELECT=100 {{(pid=81767) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 15:51:29.354603 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e4bb6dc-b0bd-465e-ab26-f6e28a88de61 None None] Validating token access rules against request {{(pid=81767) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 15:51:29.393763 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-648afc1e-bc5d-46d0-8114-423754575746 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81767) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 15:51:29.395022 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-648afc1e-bc5d-46d0-8114-423754575746 None admin] REQUEST_METHOD: `GET` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:51:29.395338 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-648afc1e-bc5d-46d0-8114-423754575746 None admin] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:51:29.395598 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-648afc1e-bc5d-46d0-8114-423754575746 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:51:29.406224 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-648afc1e-bc5d-46d0-8114-423754575746 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 22 15:51:29.411176 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application [None req-648afc1e-bc5d-46d0-8114-423754575746 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 22 15:51:29.411176 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 15:51:29.411176 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 15:51:29.411176 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 15:51:29.411176 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 15:51:29.411176 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 15:51:29.411176 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 15:51:29.411176 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 15:51:29.411176 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 15:51:29.411176 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 15:51:29.411176 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 15:51:29.411176 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 15:51:29.411176 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 22 15:51:29.411176 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 22 15:51:29.411176 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 22 15:51:29.411176 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 22 15:51:29.412272 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 22 15:51:29.412272 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 22 15:51:29.412272 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 22 15:51:29.412272 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 22 15:51:29.412272 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 15:51:29.412272 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 15:51:29.412272 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 15:51:29.412272 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 15:51:29.412272 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 15:51:29.412272 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 15:51:29.412272 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 15:51:29.412272 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application with Lock( Aug 22 15:51:29.412272 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 15:51:29.412272 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return self._enter() Aug 22 15:51:29.412272 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 15:51:29.412272 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 15:51:29.413474 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 15:51:29.413474 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return self.creator() Aug 22 15:51:29.413474 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 15:51:29.413474 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application created_value = creator( Aug 22 15:51:29.413474 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 22 15:51:29.413474 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 22 15:51:29.413474 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 22 15:51:29.413474 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 22 15:51:29.413474 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 22 15:51:29.413474 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 22 15:51:29.413474 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 22 15:51:29.413474 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application Aug 22 15:51:29.413474 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 621/1263] 10.222.0.15 () {56 vars in 1096 bytes} [Mon Aug 22 15:51:29 2022] GET /identity/v3/roles/admin => generated 83 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 15:51:29.418463 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b10f8727-0b5a-4c1a-bbbb-491b9078f288 None None] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 15:51:29.437700 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=100 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 15:51:29.524160 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b10f8727-0b5a-4c1a-bbbb-491b9078f288 None None] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 15:51:29.559048 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52e9257f-2c8c-49af-82d5-052beac0b4b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 15:51:29.559695 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52e9257f-2c8c-49af-82d5-052beac0b4b2 None admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:51:29.559973 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52e9257f-2c8c-49af-82d5-052beac0b4b2 None admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:51:29.560242 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52e9257f-2c8c-49af-82d5-052beac0b4b2 None admin] PATH_INFO: `/v3/roles` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:51:29.560907 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52e9257f-2c8c-49af-82d5-052beac0b4b2 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81768) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 15:51:29.561205 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52e9257f-2c8c-49af-82d5-052beac0b4b2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 15:51:29.562838 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52e9257f-2c8c-49af-82d5-052beac0b4b2 None admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 15:51:29.571594 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 643/1264] 10.222.0.15 () {56 vars in 1099 bytes} [Mon Aug 22 15:51:29 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 15:51:29.577378 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-648afc1e-bc5d-46d0-8114-423754575746 None admin] Authenticating user token {{(pid=81767) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 15:51:29.647174 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG dbcounter [-] [81767] Writing DB stats keystone:SELECT=100 {{(pid=81767) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 15:51:29.680229 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-648afc1e-bc5d-46d0-8114-423754575746 None admin] Validating token access rules against request {{(pid=81767) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 15:51:29.715806 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-727852e9-3d35-45ef-a1ee-70c6d59f5c55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81767) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 15:51:29.716702 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-727852e9-3d35-45ef-a1ee-70c6d59f5c55 None admin] REQUEST_METHOD: `GET` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:51:29.717148 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-727852e9-3d35-45ef-a1ee-70c6d59f5c55 None admin] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:51:29.718840 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-727852e9-3d35-45ef-a1ee-70c6d59f5c55 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:51:29.729764 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-727852e9-3d35-45ef-a1ee-70c6d59f5c55 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 15:51:29.731212 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-727852e9-3d35-45ef-a1ee-70c6d59f5c55 None admin] RBAC: Authorization granted {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 15:51:29.745701 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application [None req-727852e9-3d35-45ef-a1ee-70c6d59f5c55 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 22 15:51:29.745701 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 15:51:29.745701 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 15:51:29.745701 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 15:51:29.745701 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 15:51:29.745701 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 15:51:29.745701 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 15:51:29.745701 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 15:51:29.745701 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 15:51:29.745701 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 15:51:29.745701 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 15:51:29.745701 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 15:51:29.745701 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 22 15:51:29.745701 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 22 15:51:29.745701 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 22 15:51:29.745701 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 22 15:51:29.747010 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 15:51:29.747010 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 15:51:29.747010 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 15:51:29.747010 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 15:51:29.747010 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 15:51:29.747010 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 15:51:29.747010 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 15:51:29.747010 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application with Lock( Aug 22 15:51:29.747010 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 15:51:29.747010 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return self._enter() Aug 22 15:51:29.747010 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 15:51:29.747010 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 15:51:29.747010 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 15:51:29.747010 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return self.creator() Aug 22 15:51:29.747010 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 15:51:29.747010 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application created_value = creator( Aug 22 15:51:29.747010 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 22 15:51:29.748247 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 22 15:51:29.748247 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 22 15:51:29.748247 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application Aug 22 15:51:29.751510 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 622/1265] 10.222.0.15 () {56 vars in 1108 bytes} [Mon Aug 22 15:51:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 15:51:29.752883 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52e9257f-2c8c-49af-82d5-052beac0b4b2 None admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 15:51:29.838457 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=100 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 15:51:29.857644 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52e9257f-2c8c-49af-82d5-052beac0b4b2 None admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 15:51:29.890836 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4ecf8a8-288b-49d9-ab79-9bbf4cf253ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 15:51:29.891572 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4ecf8a8-288b-49d9-ab79-9bbf4cf253ad None admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:51:29.891734 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4ecf8a8-288b-49d9-ab79-9bbf4cf253ad None admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:51:29.892003 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4ecf8a8-288b-49d9-ab79-9bbf4cf253ad None admin] PATH_INFO: `/v3/domains` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:51:29.892644 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4ecf8a8-288b-49d9-ab79-9bbf4cf253ad None admin] RBAC: Adding query filter params (name=Default) {{(pid=81768) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 15:51:29.892948 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4ecf8a8-288b-49d9-ab79-9bbf4cf253ad None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 15:51:29.894325 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4ecf8a8-288b-49d9-ab79-9bbf4cf253ad None admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 15:51:29.907123 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 644/1266] 10.222.0.15 () {56 vars in 1109 bytes} [Mon Aug 22 15:51:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 15:51:29.913223 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-727852e9-3d35-45ef-a1ee-70c6d59f5c55 None admin] Authenticating user token {{(pid=81767) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 15:51:30.022317 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-727852e9-3d35-45ef-a1ee-70c6d59f5c55 None admin] Validating token access rules against request {{(pid=81767) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 15:51:30.043237 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG dbcounter [-] [81767] Writing DB stats keystone:SELECT=100 {{(pid=81767) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 15:51:30.057567 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54e8b84c-bd13-465c-b03a-55737d023983 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81767) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 15:51:30.058194 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54e8b84c-bd13-465c-b03a-55737d023983 None admin] REQUEST_METHOD: `GET` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:51:30.058470 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54e8b84c-bd13-465c-b03a-55737d023983 None admin] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:51:30.058739 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54e8b84c-bd13-465c-b03a-55737d023983 None admin] PATH_INFO: `/v3/users/placement` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:51:30.067464 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54e8b84c-bd13-465c-b03a-55737d023983 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 15:51:30.068850 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54e8b84c-bd13-465c-b03a-55737d023983 None admin] RBAC: Authorization granted {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 15:51:30.080359 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application [None req-54e8b84c-bd13-465c-b03a-55737d023983 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 22 15:51:30.080359 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 15:51:30.080359 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 15:51:30.080359 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 15:51:30.080359 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 15:51:30.080359 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 15:51:30.080359 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 15:51:30.080359 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 15:51:30.080359 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 15:51:30.080359 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 15:51:30.080359 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 15:51:30.080359 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 15:51:30.080359 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 22 15:51:30.080359 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 22 15:51:30.080359 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 22 15:51:30.080359 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 22 15:51:30.094552 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 15:51:30.094552 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 15:51:30.094552 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 22 15:51:30.094552 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 22 15:51:30.094552 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 22 15:51:30.094552 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 22 15:51:30.094552 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 15:51:30.094552 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 15:51:30.094552 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 15:51:30.094552 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 15:51:30.094552 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 15:51:30.094552 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application with Lock( Aug 22 15:51:30.094552 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 15:51:30.094552 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return self._enter() Aug 22 15:51:30.094552 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 15:51:30.094552 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 15:51:30.094552 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 15:51:30.094552 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return self.creator() Aug 22 15:51:30.095910 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 15:51:30.095910 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application created_value = creator( Aug 22 15:51:30.095910 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 22 15:51:30.095910 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 22 15:51:30.095910 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 22 15:51:30.095910 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 22 15:51:30.095910 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 22 15:51:30.095910 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 22 15:51:30.095910 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 22 15:51:30.095910 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application Aug 22 15:51:30.095910 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 623/1267] 10.222.0.15 () {56 vars in 1108 bytes} [Mon Aug 22 15:51:29 2022] GET /identity/v3/users/placement => generated 87 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 15:51:30.096717 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4ecf8a8-288b-49d9-ab79-9bbf4cf253ad None admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 15:51:30.195814 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4ecf8a8-288b-49d9-ab79-9bbf4cf253ad None admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 15:51:30.230336 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ff1e30c-8c97-4437-b3b5-91dc407031e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 15:51:30.230943 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff1e30c-8c97-4437-b3b5-91dc407031e7 None admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:51:30.231222 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff1e30c-8c97-4437-b3b5-91dc407031e7 None admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:51:30.231548 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff1e30c-8c97-4437-b3b5-91dc407031e7 None admin] PATH_INFO: `/v3/users` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:51:30.232277 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ff1e30c-8c97-4437-b3b5-91dc407031e7 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81768) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 15:51:30.232580 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ff1e30c-8c97-4437-b3b5-91dc407031e7 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 15:51:30.234166 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ff1e30c-8c97-4437-b3b5-91dc407031e7 None admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 15:51:30.243661 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=100 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 15:51:30.253310 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 645/1268] 10.222.0.15 () {56 vars in 1143 bytes} [Mon Aug 22 15:51:30 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 15:51:30.259899 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54e8b84c-bd13-465c-b03a-55737d023983 None admin] Authenticating user token {{(pid=81767) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 15:51:30.373631 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54e8b84c-bd13-465c-b03a-55737d023983 None admin] Validating token access rules against request {{(pid=81767) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 15:51:30.397054 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d868e95f-7ab4-4e50-b9d8-3a27513cefc2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81767) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 15:51:30.397675 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d868e95f-7ab4-4e50-b9d8-3a27513cefc2 None admin] REQUEST_METHOD: `GET` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:51:30.397951 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d868e95f-7ab4-4e50-b9d8-3a27513cefc2 None admin] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:51:30.398208 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d868e95f-7ab4-4e50-b9d8-3a27513cefc2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:51:30.408692 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d868e95f-7ab4-4e50-b9d8-3a27513cefc2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 15:51:30.410137 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d868e95f-7ab4-4e50-b9d8-3a27513cefc2 None admin] RBAC: Authorization granted {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 15:51:30.424153 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application [None req-d868e95f-7ab4-4e50-b9d8-3a27513cefc2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 22 15:51:30.424153 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 15:51:30.424153 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 15:51:30.424153 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 15:51:30.424153 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 15:51:30.424153 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 15:51:30.424153 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 15:51:30.424153 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 15:51:30.424153 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 15:51:30.424153 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 15:51:30.424153 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 15:51:30.424153 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 15:51:30.424153 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 22 15:51:30.424153 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 22 15:51:30.424153 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 22 15:51:30.424153 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 22 15:51:30.425278 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 15:51:30.425278 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 15:51:30.425278 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 15:51:30.425278 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 15:51:30.425278 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 15:51:30.425278 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 15:51:30.425278 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 15:51:30.425278 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application with Lock( Aug 22 15:51:30.425278 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 15:51:30.425278 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return self._enter() Aug 22 15:51:30.425278 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 15:51:30.425278 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 15:51:30.425278 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 15:51:30.425278 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return self.creator() Aug 22 15:51:30.425278 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 15:51:30.425278 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application created_value = creator( Aug 22 15:51:30.425278 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 22 15:51:30.426980 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 22 15:51:30.426980 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 22 15:51:30.426980 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application Aug 22 15:51:30.426980 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 624/1269] 10.222.0.15 () {56 vars in 1108 bytes} [Mon Aug 22 15:51:30 2022] GET /identity/v3/domains/Default => generated 87 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 15:51:30.431727 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ff1e30c-8c97-4437-b3b5-91dc407031e7 None admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 15:51:30.535920 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ff1e30c-8c97-4437-b3b5-91dc407031e7 None admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 15:51:30.570615 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82f010ab-7427-4b80-9be3-6c87d64fa343 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 15:51:30.571223 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82f010ab-7427-4b80-9be3-6c87d64fa343 None admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:51:30.571537 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82f010ab-7427-4b80-9be3-6c87d64fa343 None admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:51:30.571806 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82f010ab-7427-4b80-9be3-6c87d64fa343 None admin] PATH_INFO: `/v3/domains` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:51:30.572438 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82f010ab-7427-4b80-9be3-6c87d64fa343 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81768) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 15:51:30.572743 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82f010ab-7427-4b80-9be3-6c87d64fa343 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 15:51:30.574133 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82f010ab-7427-4b80-9be3-6c87d64fa343 None admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 15:51:30.586482 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 646/1270] 10.222.0.15 () {56 vars in 1109 bytes} [Mon Aug 22 15:51:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 15:51:30.592353 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d868e95f-7ab4-4e50-b9d8-3a27513cefc2 None admin] Authenticating user token {{(pid=81767) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 15:51:30.624467 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG dbcounter [-] [81767] Writing DB stats keystone:SELECT=100 {{(pid=81767) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 15:51:30.693644 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d868e95f-7ab4-4e50-b9d8-3a27513cefc2 None admin] Validating token access rules against request {{(pid=81767) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 15:51:30.729317 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bf95f53-3c33-468f-9314-1606424c2c62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81767) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 15:51:30.729945 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bf95f53-3c33-468f-9314-1606424c2c62 None admin] REQUEST_METHOD: `GET` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:51:30.730215 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bf95f53-3c33-468f-9314-1606424c2c62 None admin] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:51:30.730498 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bf95f53-3c33-468f-9314-1606424c2c62 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:51:30.738957 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bf95f53-3c33-468f-9314-1606424c2c62 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 15:51:30.740410 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bf95f53-3c33-468f-9314-1606424c2c62 None admin] RBAC: Authorization granted {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 15:51:30.751367 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application [None req-1bf95f53-3c33-468f-9314-1606424c2c62 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 22 15:51:30.751367 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 15:51:30.751367 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 15:51:30.751367 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 15:51:30.751367 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 15:51:30.751367 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 15:51:30.751367 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 15:51:30.751367 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 15:51:30.751367 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 15:51:30.751367 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 15:51:30.751367 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 15:51:30.751367 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 15:51:30.751367 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 22 15:51:30.751367 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 22 15:51:30.751367 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 22 15:51:30.752486 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 22 15:51:30.752486 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 15:51:30.752486 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 15:51:30.752486 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 15:51:30.752486 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 15:51:30.752486 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 15:51:30.752486 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 15:51:30.752486 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 15:51:30.752486 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application with Lock( Aug 22 15:51:30.752486 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 15:51:30.752486 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return self._enter() Aug 22 15:51:30.752486 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 15:51:30.752486 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 15:51:30.752486 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 15:51:30.752486 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return self.creator() Aug 22 15:51:30.752486 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 15:51:30.752486 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application created_value = creator( Aug 22 15:51:30.753518 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 22 15:51:30.753518 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 22 15:51:30.753518 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 22 15:51:30.753518 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 22 15:51:30.753518 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 22 15:51:30.753518 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 22 15:51:30.753518 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 22 15:51:30.753518 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application Aug 22 15:51:30.753518 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 625/1271] 10.222.0.15 () {56 vars in 1111 bytes} [Mon Aug 22 15:51:30 2022] GET /identity/v3/projects/service => generated 88 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 22 15:51:30.760212 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82f010ab-7427-4b80-9be3-6c87d64fa343 None admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 15:51:30.821305 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=100 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 15:51:30.863772 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82f010ab-7427-4b80-9be3-6c87d64fa343 None admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 15:51:30.898170 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-145fc538-ab27-41be-b1c7-824abab5dafe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 15:51:30.898798 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-145fc538-ab27-41be-b1c7-824abab5dafe None admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:51:30.898956 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-145fc538-ab27-41be-b1c7-824abab5dafe None admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:51:30.899224 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-145fc538-ab27-41be-b1c7-824abab5dafe None admin] PATH_INFO: `/v3/projects` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:51:30.899935 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-145fc538-ab27-41be-b1c7-824abab5dafe None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81768) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 15:51:30.900234 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-145fc538-ab27-41be-b1c7-824abab5dafe None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 15:51:30.901687 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-145fc538-ab27-41be-b1c7-824abab5dafe None admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 15:51:30.914152 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 647/1272] 10.222.0.15 () {56 vars in 1148 bytes} [Mon Aug 22 15:51:30 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 15:51:30.920200 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bf95f53-3c33-468f-9314-1606424c2c62 None admin] Authenticating user token {{(pid=81767) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 15:51:31.021334 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG dbcounter [-] [81767] Writing DB stats keystone:SELECT=100 {{(pid=81767) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 15:51:31.030149 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bf95f53-3c33-468f-9314-1606424c2c62 None admin] Validating token access rules against request {{(pid=81767) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 15:51:31.065648 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1c54de1-ecb2-440d-91b9-c4e9986216ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81767) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 15:51:31.067037 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1c54de1-ecb2-440d-91b9-c4e9986216ef None admin] REQUEST_METHOD: `PUT` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:51:31.068152 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1c54de1-ecb2-440d-91b9-c4e9986216ef None admin] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:51:31.068152 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1c54de1-ecb2-440d-91b9-c4e9986216ef None admin] PATH_INFO: `/v3/projects/c4ee0413f9ef4d4fb1f95a660babe090/users/08159bf8edda4246bb463f7b11b0894b/roles/9d9848cfd22e4b089db443c350e56670` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:51:31.069559 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1c54de1-ecb2-440d-91b9-c4e9986216ef None admin] RBAC: Authorizing `identity:create_grant(project_id=c4ee0413f9ef4d4fb1f95a660babe090, user_id=08159bf8edda4246bb463f7b11b0894b, role_id=9d9848cfd22e4b089db443c350e56670)` {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 15:51:31.071056 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1c54de1-ecb2-440d-91b9-c4e9986216ef None admin] RBAC: Authorization granted {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 15:51:31.314274 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 626/1273] 10.222.0.15 () {58 vars in 1439 bytes} [Mon Aug 22 15:51:30 2022] PUT /identity/v3/projects/c4ee0413f9ef4d4fb1f95a660babe090/users/08159bf8edda4246bb463f7b11b0894b/roles/9d9848cfd22e4b089db443c350e56670 => generated 0 bytes in 395 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 15:51:33.226891 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a67265b5-ceda-4a7b-bbdd-f4d9c3fb713f None None] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:51:33.227448 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a67265b5-ceda-4a7b-bbdd-f4d9c3fb713f None None] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:51:33.227565 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a67265b5-ceda-4a7b-bbdd-f4d9c3fb713f None None] PATH_INFO: `/` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:51:33.228803 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 648/1274] 10.222.0.15 () {54 vars in 927 bytes} [Mon Aug 22 15:51:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 15:51:33.234221 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a2c3317-2bec-4967-ad34-87350f46a371 None None] REQUEST_METHOD: `POST` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:51:33.234651 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a2c3317-2bec-4967-ad34-87350f46a371 None None] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:51:33.234764 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a2c3317-2bec-4967-ad34-87350f46a371 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:51:33.254502 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.auth.core [None req-7a2c3317-2bec-4967-ad34-87350f46a371 None None] MFA Rules not processed for user `6b2199ae0fb0480aa917ada391f5ec54`. Rule list: `[]` (Enabled: `True`). {{(pid=81767) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 15:51:33.255733 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.fernet_utils [None req-7a2c3317-2bec-4967-ad34-87350f46a371 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81767) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 15:51:33.323510 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 627/1275] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Aug 22 15:51:33 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 22 15:51:33.589992 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a67265b5-ceda-4a7b-bbdd-f4d9c3fb713f None None] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 15:51:33.694556 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a67265b5-ceda-4a7b-bbdd-f4d9c3fb713f None None] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 15:51:33.712255 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=100 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 15:51:33.731394 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-260cd027-f54b-4437-9b08-27f8f5257342 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 15:51:33.732022 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-260cd027-f54b-4437-9b08-27f8f5257342 None admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:51:33.732295 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-260cd027-f54b-4437-9b08-27f8f5257342 None admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:51:33.732561 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-260cd027-f54b-4437-9b08-27f8f5257342 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:51:33.743528 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-260cd027-f54b-4437-9b08-27f8f5257342 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 22 15:51:33.748160 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application [None req-260cd027-f54b-4437-9b08-27f8f5257342 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 22 15:51:33.748160 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 15:51:33.748160 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 15:51:33.748160 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 15:51:33.748160 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 15:51:33.748160 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 15:51:33.748160 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 15:51:33.748160 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 15:51:33.748160 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 15:51:33.748160 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 15:51:33.748160 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 15:51:33.748160 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 15:51:33.748160 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 22 15:51:33.748160 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 22 15:51:33.748160 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 22 15:51:33.748160 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 22 15:51:33.749199 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 22 15:51:33.749199 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 22 15:51:33.749199 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 22 15:51:33.749199 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 22 15:51:33.749199 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 15:51:33.749199 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 15:51:33.749199 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 15:51:33.749199 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 15:51:33.749199 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 15:51:33.749199 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 15:51:33.749199 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 15:51:33.749199 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application with Lock( Aug 22 15:51:33.749199 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 15:51:33.749199 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self._enter() Aug 22 15:51:33.749199 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 15:51:33.749199 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 15:51:33.750446 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 15:51:33.750446 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self.creator() Aug 22 15:51:33.750446 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 15:51:33.750446 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application created_value = creator( Aug 22 15:51:33.750446 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 22 15:51:33.750446 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 22 15:51:33.750446 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 22 15:51:33.750446 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 22 15:51:33.750446 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 22 15:51:33.750446 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 22 15:51:33.750446 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 22 15:51:33.750446 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application Aug 22 15:51:33.750446 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 649/1276] 10.222.0.15 () {56 vars in 1096 bytes} [Mon Aug 22 15:51:33 2022] GET /identity/v3/roles/admin => generated 83 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 15:51:33.755332 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a2c3317-2bec-4967-ad34-87350f46a371 None None] Authenticating user token {{(pid=81767) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 15:51:33.836676 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG dbcounter [-] [81767] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81767) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 15:51:33.854735 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a2c3317-2bec-4967-ad34-87350f46a371 None None] Validating token access rules against request {{(pid=81767) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 15:51:33.888342 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46ad4598-b3dc-42c1-9ac0-43cee521737f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81767) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 15:51:33.888955 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46ad4598-b3dc-42c1-9ac0-43cee521737f None admin] REQUEST_METHOD: `GET` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:51:33.889223 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46ad4598-b3dc-42c1-9ac0-43cee521737f None admin] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:51:33.889488 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46ad4598-b3dc-42c1-9ac0-43cee521737f None admin] PATH_INFO: `/v3/roles` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:51:33.890146 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46ad4598-b3dc-42c1-9ac0-43cee521737f None admin] RBAC: Adding query filter params (name=admin) {{(pid=81767) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 15:51:33.890461 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46ad4598-b3dc-42c1-9ac0-43cee521737f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 15:51:33.891834 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46ad4598-b3dc-42c1-9ac0-43cee521737f None admin] RBAC: Authorization granted {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 15:51:33.901238 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 628/1277] 10.222.0.15 () {56 vars in 1099 bytes} [Mon Aug 22 15:51:33 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 15:51:33.907089 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-260cd027-f54b-4437-9b08-27f8f5257342 None admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 15:51:34.014899 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-260cd027-f54b-4437-9b08-27f8f5257342 None admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 15:51:34.050136 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ae08805-85ff-469c-83bb-ad37eae012c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 15:51:34.050759 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae08805-85ff-469c-83bb-ad37eae012c4 None admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:51:34.051034 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae08805-85ff-469c-83bb-ad37eae012c4 None admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:51:34.051326 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae08805-85ff-469c-83bb-ad37eae012c4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:51:34.061606 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ae08805-85ff-469c-83bb-ad37eae012c4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 15:51:34.063331 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ae08805-85ff-469c-83bb-ad37eae012c4 None admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 15:51:34.080507 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application [None req-7ae08805-85ff-469c-83bb-ad37eae012c4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 22 15:51:34.080507 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 15:51:34.080507 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 15:51:34.080507 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 15:51:34.080507 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 15:51:34.080507 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 15:51:34.080507 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 15:51:34.080507 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 15:51:34.080507 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 15:51:34.080507 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 15:51:34.080507 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 15:51:34.080507 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 15:51:34.080507 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 22 15:51:34.080507 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 22 15:51:34.080507 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 22 15:51:34.080507 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 22 15:51:34.083135 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 15:51:34.083135 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 15:51:34.083135 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 15:51:34.083135 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 15:51:34.083135 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 15:51:34.083135 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 15:51:34.083135 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 15:51:34.083135 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application with Lock( Aug 22 15:51:34.083135 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 15:51:34.083135 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self._enter() Aug 22 15:51:34.083135 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 15:51:34.083135 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 15:51:34.083135 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 15:51:34.083135 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self.creator() Aug 22 15:51:34.083135 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 15:51:34.083135 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application created_value = creator( Aug 22 15:51:34.083135 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 22 15:51:34.085394 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 22 15:51:34.085394 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 22 15:51:34.085394 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application Aug 22 15:51:34.085394 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 650/1278] 10.222.0.15 () {56 vars in 1108 bytes} [Mon Aug 22 15:51:33 2022] GET /identity/v3/domains/Default => generated 87 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 15:51:34.089946 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46ad4598-b3dc-42c1-9ac0-43cee521737f None admin] Authenticating user token {{(pid=81767) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 15:51:34.245457 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46ad4598-b3dc-42c1-9ac0-43cee521737f None admin] Validating token access rules against request {{(pid=81767) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 15:51:34.245457 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d54ab177-d5a5-4af7-b3d8-98a68d413df7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81767) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 15:51:34.245457 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d54ab177-d5a5-4af7-b3d8-98a68d413df7 None admin] REQUEST_METHOD: `GET` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:51:34.245457 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d54ab177-d5a5-4af7-b3d8-98a68d413df7 None admin] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:51:34.246446 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d54ab177-d5a5-4af7-b3d8-98a68d413df7 None admin] PATH_INFO: `/v3/domains` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:51:34.246446 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d54ab177-d5a5-4af7-b3d8-98a68d413df7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81767) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 15:51:34.246446 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d54ab177-d5a5-4af7-b3d8-98a68d413df7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 15:51:34.246446 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d54ab177-d5a5-4af7-b3d8-98a68d413df7 None admin] RBAC: Authorization granted {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 15:51:34.257884 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 629/1279] 10.222.0.15 () {56 vars in 1109 bytes} [Mon Aug 22 15:51:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 15:51:34.263683 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ae08805-85ff-469c-83bb-ad37eae012c4 None admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 15:51:34.295952 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=100 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 15:51:34.366445 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ae08805-85ff-469c-83bb-ad37eae012c4 None admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 15:51:34.400424 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdae0a79-f03b-426a-9788-3a6211263184 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 15:51:34.401045 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdae0a79-f03b-426a-9788-3a6211263184 None admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:51:34.401323 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdae0a79-f03b-426a-9788-3a6211263184 None admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:51:34.401597 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdae0a79-f03b-426a-9788-3a6211263184 None admin] PATH_INFO: `/v3/users/placement` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:51:34.408968 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdae0a79-f03b-426a-9788-3a6211263184 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 15:51:34.410963 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdae0a79-f03b-426a-9788-3a6211263184 None admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 15:51:34.422522 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application [None req-fdae0a79-f03b-426a-9788-3a6211263184 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 22 15:51:34.422522 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 15:51:34.422522 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 15:51:34.422522 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 15:51:34.422522 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 15:51:34.422522 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 15:51:34.422522 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 15:51:34.422522 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 15:51:34.422522 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 15:51:34.422522 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 15:51:34.422522 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 15:51:34.422522 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 15:51:34.422522 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 22 15:51:34.422522 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 22 15:51:34.422522 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 22 15:51:34.422522 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 22 15:51:34.424177 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 15:51:34.424177 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 15:51:34.424177 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 22 15:51:34.424177 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 22 15:51:34.424177 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 22 15:51:34.424177 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 22 15:51:34.424177 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 15:51:34.424177 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 15:51:34.424177 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 15:51:34.424177 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 15:51:34.424177 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 15:51:34.424177 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application with Lock( Aug 22 15:51:34.424177 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 15:51:34.424177 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self._enter() Aug 22 15:51:34.424177 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 15:51:34.424177 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 15:51:34.424177 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 15:51:34.424177 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self.creator() Aug 22 15:51:34.426324 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 15:51:34.426324 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application created_value = creator( Aug 22 15:51:34.426324 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 22 15:51:34.426324 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 22 15:51:34.426324 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 22 15:51:34.426324 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 22 15:51:34.426324 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 22 15:51:34.426324 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 22 15:51:34.426324 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 22 15:51:34.426324 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application Aug 22 15:51:34.426324 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 651/1280] 10.222.0.15 () {56 vars in 1108 bytes} [Mon Aug 22 15:51:34 2022] GET /identity/v3/users/placement => generated 87 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 15:51:34.430897 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d54ab177-d5a5-4af7-b3d8-98a68d413df7 None admin] Authenticating user token {{(pid=81767) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 15:51:34.434717 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG dbcounter [-] [81767] Writing DB stats keystone:SELECT=100 {{(pid=81767) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 15:51:34.529298 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d54ab177-d5a5-4af7-b3d8-98a68d413df7 None admin] Validating token access rules against request {{(pid=81767) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 15:51:34.564028 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69520d9e-fd7f-4c6e-a65a-af8510301f5d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81767) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 15:51:34.564630 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69520d9e-fd7f-4c6e-a65a-af8510301f5d None admin] REQUEST_METHOD: `GET` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:51:34.564906 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69520d9e-fd7f-4c6e-a65a-af8510301f5d None admin] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:51:34.565175 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69520d9e-fd7f-4c6e-a65a-af8510301f5d None admin] PATH_INFO: `/v3/users` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:51:34.565898 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69520d9e-fd7f-4c6e-a65a-af8510301f5d None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81767) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 15:51:34.566199 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69520d9e-fd7f-4c6e-a65a-af8510301f5d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 15:51:34.567704 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69520d9e-fd7f-4c6e-a65a-af8510301f5d None admin] RBAC: Authorization granted {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 15:51:34.586381 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 630/1281] 10.222.0.15 () {56 vars in 1143 bytes} [Mon Aug 22 15:51:34 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 15:51:34.592683 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdae0a79-f03b-426a-9788-3a6211263184 None admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 15:51:34.687145 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=100 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 15:51:34.697033 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdae0a79-f03b-426a-9788-3a6211263184 None admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 15:51:34.731546 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b3d04fe-20e1-4ccb-99c9-b9f3ce3da95d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 15:51:34.732148 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b3d04fe-20e1-4ccb-99c9-b9f3ce3da95d None admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:51:34.732427 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b3d04fe-20e1-4ccb-99c9-b9f3ce3da95d None admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:51:34.732700 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b3d04fe-20e1-4ccb-99c9-b9f3ce3da95d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:51:34.804056 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b3d04fe-20e1-4ccb-99c9-b9f3ce3da95d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 15:51:34.805506 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b3d04fe-20e1-4ccb-99c9-b9f3ce3da95d None admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 15:51:34.894982 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=100 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 15:51:34.905088 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 652/1282] 10.222.0.15 () {58 vars in 1282 bytes} [Mon Aug 22 15:51:34 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 313 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 22 15:51:34.911201 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69520d9e-fd7f-4c6e-a65a-af8510301f5d None admin] Authenticating user token {{(pid=81767) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 15:51:34.976576 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG dbcounter [-] [81767] Writing DB stats keystone:SELECT=100 {{(pid=81767) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 15:51:35.022799 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69520d9e-fd7f-4c6e-a65a-af8510301f5d None admin] Validating token access rules against request {{(pid=81767) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 15:51:35.056416 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-730c1b69-b88c-4393-ad91-68c39168b236 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81767) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 15:51:35.056975 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-730c1b69-b88c-4393-ad91-68c39168b236 None admin] REQUEST_METHOD: `GET` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:51:35.057231 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-730c1b69-b88c-4393-ad91-68c39168b236 None admin] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:51:35.057495 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-730c1b69-b88c-4393-ad91-68c39168b236 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:51:35.069296 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-730c1b69-b88c-4393-ad91-68c39168b236 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 15:51:35.070672 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-730c1b69-b88c-4393-ad91-68c39168b236 None admin] RBAC: Authorization granted {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 15:51:35.086035 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application [None req-730c1b69-b88c-4393-ad91-68c39168b236 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 22 15:51:35.086035 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 15:51:35.086035 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 15:51:35.086035 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 15:51:35.086035 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 15:51:35.086035 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 15:51:35.086035 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 15:51:35.086035 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 15:51:35.086035 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 15:51:35.086035 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 15:51:35.086035 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 15:51:35.086035 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 15:51:35.086035 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 22 15:51:35.086035 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 22 15:51:35.086035 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 22 15:51:35.086035 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 22 15:51:35.087220 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 15:51:35.087220 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 15:51:35.087220 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 15:51:35.087220 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 15:51:35.087220 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 15:51:35.087220 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 15:51:35.087220 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 15:51:35.087220 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application with Lock( Aug 22 15:51:35.087220 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 15:51:35.087220 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return self._enter() Aug 22 15:51:35.087220 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 15:51:35.087220 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 15:51:35.087220 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 15:51:35.087220 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return self.creator() Aug 22 15:51:35.087220 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 15:51:35.087220 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application created_value = creator( Aug 22 15:51:35.087220 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 22 15:51:35.088257 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 22 15:51:35.088257 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 22 15:51:35.088257 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application Aug 22 15:51:35.088257 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 631/1283] 10.222.0.15 () {56 vars in 1108 bytes} [Mon Aug 22 15:51:34 2022] GET /identity/v3/domains/Default => generated 87 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 15:51:35.093562 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b3d04fe-20e1-4ccb-99c9-b9f3ce3da95d None admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 15:51:35.200540 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b3d04fe-20e1-4ccb-99c9-b9f3ce3da95d None admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 15:51:35.235485 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d76d4ee-b70f-4a70-aa6b-1219f3f2c9ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 15:51:35.236083 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d76d4ee-b70f-4a70-aa6b-1219f3f2c9ef None admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:51:35.236361 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d76d4ee-b70f-4a70-aa6b-1219f3f2c9ef None admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:51:35.236631 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d76d4ee-b70f-4a70-aa6b-1219f3f2c9ef None admin] PATH_INFO: `/v3/domains` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:51:35.237264 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d76d4ee-b70f-4a70-aa6b-1219f3f2c9ef None admin] RBAC: Adding query filter params (name=Default) {{(pid=81768) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 15:51:35.237567 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d76d4ee-b70f-4a70-aa6b-1219f3f2c9ef None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 15:51:35.239632 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d76d4ee-b70f-4a70-aa6b-1219f3f2c9ef None admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 15:51:35.250894 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 653/1284] 10.222.0.15 () {56 vars in 1109 bytes} [Mon Aug 22 15:51:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 15:51:35.256766 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-730c1b69-b88c-4393-ad91-68c39168b236 None admin] Authenticating user token {{(pid=81767) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 15:51:35.364048 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-730c1b69-b88c-4393-ad91-68c39168b236 None admin] Validating token access rules against request {{(pid=81767) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 15:51:35.382289 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG dbcounter [-] [81767] Writing DB stats keystone:SELECT=100 {{(pid=81767) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 15:51:35.403246 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb809409-2722-4778-8f35-ff1cd3c5ec25 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81767) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 15:51:35.403900 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb809409-2722-4778-8f35-ff1cd3c5ec25 None admin] REQUEST_METHOD: `GET` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:51:35.404176 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb809409-2722-4778-8f35-ff1cd3c5ec25 None admin] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:51:35.404445 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb809409-2722-4778-8f35-ff1cd3c5ec25 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:51:35.412008 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb809409-2722-4778-8f35-ff1cd3c5ec25 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 15:51:35.413655 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb809409-2722-4778-8f35-ff1cd3c5ec25 None admin] RBAC: Authorization granted {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 15:51:35.425192 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application [None req-cb809409-2722-4778-8f35-ff1cd3c5ec25 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 22 15:51:35.425192 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 15:51:35.425192 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 15:51:35.425192 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 15:51:35.425192 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 15:51:35.425192 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 15:51:35.425192 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 15:51:35.425192 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 15:51:35.425192 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 15:51:35.425192 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 15:51:35.425192 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 15:51:35.425192 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 15:51:35.425192 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 22 15:51:35.425192 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 22 15:51:35.425192 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 22 15:51:35.426510 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 22 15:51:35.426510 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 15:51:35.426510 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 15:51:35.426510 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 15:51:35.426510 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 15:51:35.426510 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 15:51:35.426510 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 15:51:35.426510 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 15:51:35.426510 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application with Lock( Aug 22 15:51:35.426510 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 15:51:35.426510 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return self._enter() Aug 22 15:51:35.426510 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 15:51:35.426510 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 15:51:35.426510 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 15:51:35.426510 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return self.creator() Aug 22 15:51:35.426510 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 15:51:35.426510 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application created_value = creator( Aug 22 15:51:35.427653 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 22 15:51:35.427653 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 22 15:51:35.427653 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 22 15:51:35.427653 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 22 15:51:35.427653 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 22 15:51:35.427653 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 22 15:51:35.427653 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 22 15:51:35.427653 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application Aug 22 15:51:35.427653 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 632/1285] 10.222.0.15 () {56 vars in 1111 bytes} [Mon Aug 22 15:51:35 2022] GET /identity/v3/projects/service => generated 88 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 15:51:35.432369 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d76d4ee-b70f-4a70-aa6b-1219f3f2c9ef None admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 15:51:35.485651 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=100 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 15:51:35.538008 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d76d4ee-b70f-4a70-aa6b-1219f3f2c9ef None admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 15:51:35.572899 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc0ddc9a-61ba-4d0f-b035-b13bcfa0205c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 15:51:35.573472 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc0ddc9a-61ba-4d0f-b035-b13bcfa0205c None admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:51:35.573749 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc0ddc9a-61ba-4d0f-b035-b13bcfa0205c None admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:51:35.574026 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc0ddc9a-61ba-4d0f-b035-b13bcfa0205c None admin] PATH_INFO: `/v3/projects` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:51:35.574747 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc0ddc9a-61ba-4d0f-b035-b13bcfa0205c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81768) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 15:51:35.575051 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc0ddc9a-61ba-4d0f-b035-b13bcfa0205c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 15:51:35.576564 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc0ddc9a-61ba-4d0f-b035-b13bcfa0205c None admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 15:51:35.590331 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 654/1286] 10.222.0.15 () {56 vars in 1148 bytes} [Mon Aug 22 15:51:35 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 15:51:35.596713 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb809409-2722-4778-8f35-ff1cd3c5ec25 None admin] Authenticating user token {{(pid=81767) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 15:51:35.702067 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb809409-2722-4778-8f35-ff1cd3c5ec25 None admin] Validating token access rules against request {{(pid=81767) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 15:51:35.736742 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21cdcac4-4cad-40f7-9441-c41e479532ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81767) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 15:51:35.737332 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21cdcac4-4cad-40f7-9441-c41e479532ef None admin] REQUEST_METHOD: `GET` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:51:35.737607 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21cdcac4-4cad-40f7-9441-c41e479532ef None admin] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:51:35.737873 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21cdcac4-4cad-40f7-9441-c41e479532ef None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:51:35.738588 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21cdcac4-4cad-40f7-9441-c41e479532ef None admin] RBAC: Adding query filter params (role.id=9d9848cfd22e4b089db443c350e56670, scope.project.id=c4ee0413f9ef4d4fb1f95a660babe090, user.id=08159bf8edda4246bb463f7b11b0894b) {{(pid=81767) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 15:51:35.738888 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21cdcac4-4cad-40f7-9441-c41e479532ef None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 15:51:35.740331 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21cdcac4-4cad-40f7-9441-c41e479532ef None admin] RBAC: Authorization granted {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 15:51:35.746921 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 633/1287] 10.222.0.15 () {56 vars in 1374 bytes} [Mon Aug 22 15:51:35 2022] GET /identity/v3/role_assignments?user.id=08159bf8edda4246bb463f7b11b0894b&scope.project.id=c4ee0413f9ef4d4fb1f95a660babe090&role.id=9d9848cfd22e4b089db443c350e56670 => generated 608 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 15:51:37.690678 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f8b89c6-a4bc-4f82-8643-e722090948ed None None] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:51:37.691212 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f8b89c6-a4bc-4f82-8643-e722090948ed None None] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:51:37.691212 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f8b89c6-a4bc-4f82-8643-e722090948ed None None] PATH_INFO: `/` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:51:37.693067 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 655/1288] 10.222.0.15 () {54 vars in 927 bytes} [Mon Aug 22 15:51:37 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 15:51:37.698429 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52361943-ca69-4f81-80d3-2391ad322908 None None] REQUEST_METHOD: `POST` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:51:37.698865 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52361943-ca69-4f81-80d3-2391ad322908 None None] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:51:37.698969 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52361943-ca69-4f81-80d3-2391ad322908 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:51:37.717548 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.auth.core [None req-52361943-ca69-4f81-80d3-2391ad322908 None None] MFA Rules not processed for user `6b2199ae0fb0480aa917ada391f5ec54`. Rule list: `[]` (Enabled: `True`). {{(pid=81767) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 15:51:37.745649 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.fernet_utils [None req-52361943-ca69-4f81-80d3-2391ad322908 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81767) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 15:51:37.748325 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG dbcounter [-] [81767] Writing DB stats keystone:SELECT=100 {{(pid=81767) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 15:51:37.784890 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 634/1289] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Aug 22 15:51:37 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 22 15:51:38.055060 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f8b89c6-a4bc-4f82-8643-e722090948ed None None] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 15:51:38.159903 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f8b89c6-a4bc-4f82-8643-e722090948ed None None] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 15:51:38.170852 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=100 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 15:51:38.195492 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-770b04b9-3e33-43c1-b100-881ddfe5ef0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 15:51:38.196144 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-770b04b9-3e33-43c1-b100-881ddfe5ef0a None admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:51:38.196417 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-770b04b9-3e33-43c1-b100-881ddfe5ef0a None admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:51:38.196682 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-770b04b9-3e33-43c1-b100-881ddfe5ef0a None admin] PATH_INFO: `/v3/services/placement` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:51:38.197257 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-770b04b9-3e33-43c1-b100-881ddfe5ef0a None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 15:51:38.198622 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-770b04b9-3e33-43c1-b100-881ddfe5ef0a None admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 15:51:38.208577 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application [None req-770b04b9-3e33-43c1-b100-881ddfe5ef0a None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Aug 22 15:51:38.208577 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 15:51:38.208577 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 22 15:51:38.208577 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 22 15:51:38.208577 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 22 15:51:38.208577 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 22 15:51:38.208577 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 22 15:51:38.208577 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 22 15:51:38.208577 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 22 15:51:38.208577 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application Aug 22 15:51:38.208577 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 22 15:51:38.208577 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application Aug 22 15:51:38.208577 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 15:51:38.208577 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 15:51:38.208577 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 15:51:38.208577 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 15:51:38.208577 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 15:51:38.209884 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 15:51:38.209884 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 15:51:38.209884 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 15:51:38.209884 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 15:51:38.209884 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 15:51:38.209884 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 15:51:38.209884 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 22 15:51:38.209884 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 22 15:51:38.209884 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 22 15:51:38.209884 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 22 15:51:38.209884 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 15:51:38.209884 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 15:51:38.209884 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 15:51:38.209884 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 15:51:38.209884 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 15:51:38.209884 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 15:51:38.210950 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 15:51:38.210950 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application with Lock( Aug 22 15:51:38.210950 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 15:51:38.210950 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self._enter() Aug 22 15:51:38.210950 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 15:51:38.210950 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 15:51:38.210950 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 15:51:38.210950 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self.creator() Aug 22 15:51:38.210950 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 15:51:38.210950 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application created_value = creator( Aug 22 15:51:38.210950 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 22 15:51:38.210950 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 22 15:51:38.210950 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 22 15:51:38.210950 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application Aug 22 15:51:38.210950 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 656/1290] 10.222.0.15 () {56 vars in 1117 bytes} [Mon Aug 22 15:51:38 2022] GET /identity/v3/services/placement => generated 90 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 15:51:38.216479 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52361943-ca69-4f81-80d3-2391ad322908 None None] Authenticating user token {{(pid=81767) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 15:51:38.317279 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52361943-ca69-4f81-80d3-2391ad322908 None None] Validating token access rules against request {{(pid=81767) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 15:51:38.351885 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39470dba-070b-4a11-bdcd-21e12ca7f674 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81767) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 15:51:38.352488 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39470dba-070b-4a11-bdcd-21e12ca7f674 None admin] REQUEST_METHOD: `GET` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:51:38.352763 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39470dba-070b-4a11-bdcd-21e12ca7f674 None admin] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:51:38.353032 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39470dba-070b-4a11-bdcd-21e12ca7f674 None admin] PATH_INFO: `/v3/services` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:51:38.353708 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39470dba-070b-4a11-bdcd-21e12ca7f674 None admin] RBAC: Adding query filter params (name=placement) {{(pid=81767) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 15:51:38.354008 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39470dba-070b-4a11-bdcd-21e12ca7f674 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 15:51:38.355452 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39470dba-070b-4a11-bdcd-21e12ca7f674 None admin] RBAC: Authorization granted {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 15:51:38.360673 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 635/1291] 10.222.0.15 () {56 vars in 1116 bytes} [Mon Aug 22 15:51:38 2022] GET /identity/v3/services?name=placement => generated 126 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 15:51:38.366765 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-770b04b9-3e33-43c1-b100-881ddfe5ef0a None admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 15:51:38.470241 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-770b04b9-3e33-43c1-b100-881ddfe5ef0a None admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 15:51:38.504629 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a64deda1-e784-4478-8dd0-5c96d2e6757a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 15:51:38.505244 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a64deda1-e784-4478-8dd0-5c96d2e6757a None admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:51:38.505503 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a64deda1-e784-4478-8dd0-5c96d2e6757a None admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:51:38.505768 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a64deda1-e784-4478-8dd0-5c96d2e6757a None admin] PATH_INFO: `/v3/services` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:51:38.506387 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a64deda1-e784-4478-8dd0-5c96d2e6757a None admin] RBAC: Adding query filter params (type=placement) {{(pid=81768) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 15:51:38.506686 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a64deda1-e784-4478-8dd0-5c96d2e6757a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 15:51:38.508079 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a64deda1-e784-4478-8dd0-5c96d2e6757a None admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 15:51:38.512653 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 657/1292] 10.222.0.15 () {56 vars in 1116 bytes} [Mon Aug 22 15:51:38 2022] GET /identity/v3/services?type=placement => generated 126 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 15:51:40.434500 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c2041ae-16ea-49b7-928e-698c66790fcc None None] REQUEST_METHOD: `GET` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:51:40.435063 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c2041ae-16ea-49b7-928e-698c66790fcc None None] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:51:40.435063 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c2041ae-16ea-49b7-928e-698c66790fcc None None] PATH_INFO: `/` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:51:40.436501 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 636/1293] 10.222.0.15 () {54 vars in 927 bytes} [Mon Aug 22 15:51:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 15:51:40.442644 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a833d34f-9ca8-4262-b846-3f4d2f22b35f None None] REQUEST_METHOD: `POST` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:51:40.443095 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a833d34f-9ca8-4262-b846-3f4d2f22b35f None None] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:51:40.443196 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a833d34f-9ca8-4262-b846-3f4d2f22b35f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:51:40.461756 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.auth.core [None req-a833d34f-9ca8-4262-b846-3f4d2f22b35f None None] MFA Rules not processed for user `6b2199ae0fb0480aa917ada391f5ec54`. Rule list: `[]` (Enabled: `True`). {{(pid=81768) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 15:51:40.462942 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-a833d34f-9ca8-4262-b846-3f4d2f22b35f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 15:51:40.492473 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=100 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 15:51:40.530128 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 658/1294] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Aug 22 15:51:40 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 22 15:51:40.798599 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c2041ae-16ea-49b7-928e-698c66790fcc None None] Authenticating user token {{(pid=81767) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 15:51:40.831046 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG dbcounter [-] [81767] Writing DB stats keystone:SELECT=100 {{(pid=81767) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 15:51:40.902421 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c2041ae-16ea-49b7-928e-698c66790fcc None None] Validating token access rules against request {{(pid=81767) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 15:51:40.936405 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58a1044a-8a7a-4d45-81b5-e5fd251b9121 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81767) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 15:51:40.936784 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58a1044a-8a7a-4d45-81b5-e5fd251b9121 None admin] REQUEST_METHOD: `POST` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:51:40.937319 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58a1044a-8a7a-4d45-81b5-e5fd251b9121 None admin] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:51:40.937593 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58a1044a-8a7a-4d45-81b5-e5fd251b9121 None admin] PATH_INFO: `/v3/services` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:51:40.938229 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58a1044a-8a7a-4d45-81b5-e5fd251b9121 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 15:51:40.939617 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58a1044a-8a7a-4d45-81b5-e5fd251b9121 None admin] RBAC: Authorization granted {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 15:51:41.089918 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 637/1295] 10.222.0.15 () {60 vars in 1141 bytes} [Mon Aug 22 15:51:40 2022] POST /identity/v3/services => generated 246 bytes in 292 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 22 15:51:43.020194 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1fe9d0c-7207-419d-955a-0b7482a2de11 None None] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:51:43.020771 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1fe9d0c-7207-419d-955a-0b7482a2de11 None None] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:51:43.020771 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1fe9d0c-7207-419d-955a-0b7482a2de11 None None] PATH_INFO: `/` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:51:43.022133 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 659/1296] 10.222.0.15 () {54 vars in 927 bytes} [Mon Aug 22 15:51:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 15:51:43.027945 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2f2221a-62f2-4088-8e7c-e9bd7ca87405 None None] REQUEST_METHOD: `POST` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:51:43.028360 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2f2221a-62f2-4088-8e7c-e9bd7ca87405 None None] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:51:43.028501 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2f2221a-62f2-4088-8e7c-e9bd7ca87405 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:51:43.048168 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.auth.core [None req-d2f2221a-62f2-4088-8e7c-e9bd7ca87405 None None] MFA Rules not processed for user `6b2199ae0fb0480aa917ada391f5ec54`. Rule list: `[]` (Enabled: `True`). {{(pid=81767) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 15:51:43.049420 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.fernet_utils [None req-d2f2221a-62f2-4088-8e7c-e9bd7ca87405 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81767) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 15:51:43.128052 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 638/1297] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Aug 22 15:51:43 2022] POST /identity/v3/auth/tokens => generated 2221 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 22 15:51:43.401943 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1fe9d0c-7207-419d-955a-0b7482a2de11 None None] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 15:51:43.515762 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1fe9d0c-7207-419d-955a-0b7482a2de11 None None] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 15:51:43.554322 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6e74278-cf12-4cb2-a299-2cb5b2a6ffa8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 15:51:43.554934 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6e74278-cf12-4cb2-a299-2cb5b2a6ffa8 None admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:51:43.555207 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6e74278-cf12-4cb2-a299-2cb5b2a6ffa8 None admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:51:43.555493 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6e74278-cf12-4cb2-a299-2cb5b2a6ffa8 None admin] PATH_INFO: `/v3/services/placement` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:51:43.556064 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6e74278-cf12-4cb2-a299-2cb5b2a6ffa8 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 15:51:43.557428 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6e74278-cf12-4cb2-a299-2cb5b2a6ffa8 None admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 15:51:43.566505 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application [None req-b6e74278-cf12-4cb2-a299-2cb5b2a6ffa8 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Aug 22 15:51:43.566505 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 15:51:43.566505 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 22 15:51:43.566505 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 22 15:51:43.566505 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 22 15:51:43.566505 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 22 15:51:43.566505 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 22 15:51:43.566505 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 22 15:51:43.566505 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 22 15:51:43.566505 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application Aug 22 15:51:43.566505 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 22 15:51:43.566505 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application Aug 22 15:51:43.566505 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 15:51:43.566505 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 15:51:43.566505 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 15:51:43.566505 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 15:51:43.566505 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 15:51:43.568158 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 15:51:43.568158 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 15:51:43.568158 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 15:51:43.568158 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 15:51:43.568158 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 15:51:43.568158 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 15:51:43.568158 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 22 15:51:43.568158 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 22 15:51:43.568158 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 22 15:51:43.568158 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 22 15:51:43.568158 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 15:51:43.568158 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 15:51:43.568158 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 15:51:43.568158 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 15:51:43.568158 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 15:51:43.568158 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 15:51:43.569245 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 15:51:43.569245 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application with Lock( Aug 22 15:51:43.569245 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 15:51:43.569245 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self._enter() Aug 22 15:51:43.569245 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 15:51:43.569245 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 15:51:43.569245 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 15:51:43.569245 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self.creator() Aug 22 15:51:43.569245 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 15:51:43.569245 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application created_value = creator( Aug 22 15:51:43.569245 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 22 15:51:43.569245 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 22 15:51:43.569245 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 22 15:51:43.569245 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application Aug 22 15:51:43.569245 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 660/1298] 10.222.0.15 () {56 vars in 1117 bytes} [Mon Aug 22 15:51:43 2022] GET /identity/v3/services/placement => generated 90 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 15:51:43.573871 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2f2221a-62f2-4088-8e7c-e9bd7ca87405 None None] Authenticating user token {{(pid=81767) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 15:51:43.595760 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG dbcounter [-] [81767] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81767) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 15:51:43.675706 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2f2221a-62f2-4088-8e7c-e9bd7ca87405 None None] Validating token access rules against request {{(pid=81767) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 15:51:43.710086 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b1ca1a9-d87f-4070-a7e6-d568d568455f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81767) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 15:51:43.710691 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b1ca1a9-d87f-4070-a7e6-d568d568455f None admin] REQUEST_METHOD: `GET` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:51:43.710963 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b1ca1a9-d87f-4070-a7e6-d568d568455f None admin] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:51:43.711231 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b1ca1a9-d87f-4070-a7e6-d568d568455f None admin] PATH_INFO: `/v3/services` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:51:43.711891 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b1ca1a9-d87f-4070-a7e6-d568d568455f None admin] RBAC: Adding query filter params (name=placement) {{(pid=81767) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 15:51:43.712191 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b1ca1a9-d87f-4070-a7e6-d568d568455f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 15:51:43.713552 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b1ca1a9-d87f-4070-a7e6-d568d568455f None admin] RBAC: Authorization granted {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 15:51:43.718812 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 639/1299] 10.222.0.15 () {56 vars in 1116 bytes} [Mon Aug 22 15:51:43 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 22 15:51:43.725555 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6e74278-cf12-4cb2-a299-2cb5b2a6ffa8 None admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 15:51:43.757264 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=100 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 15:51:43.827020 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6e74278-cf12-4cb2-a299-2cb5b2a6ffa8 None admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 15:51:43.862254 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac82467f-d9ed-453a-8608-9393543caacc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 15:51:43.862888 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac82467f-d9ed-453a-8608-9393543caacc None admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:51:43.863166 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac82467f-d9ed-453a-8608-9393543caacc None admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:51:43.863467 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac82467f-d9ed-453a-8608-9393543caacc None admin] PATH_INFO: `/v3/endpoints` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:51:43.864281 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac82467f-d9ed-453a-8608-9393543caacc None admin] RBAC: Adding query filter params (interface=public, service_id=a422a58dff1941d891130162345fc002, region_id=RegionOne) {{(pid=81768) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 15:51:43.864593 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac82467f-d9ed-453a-8608-9393543caacc None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 15:51:43.867117 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac82467f-d9ed-453a-8608-9393543caacc None admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 15:51:43.874157 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 661/1300] 10.222.0.15 () {56 vars in 1251 bytes} [Mon Aug 22 15:51:43 2022] GET /identity/v3/endpoints?service_id=a422a58dff1941d891130162345fc002&interface=public®ion_id=RegionOne => generated 194 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 15:51:43.877424 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b1ca1a9-d87f-4070-a7e6-d568d568455f None admin] Authenticating user token {{(pid=81767) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 15:51:43.964894 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG dbcounter [-] [81767] Writing DB stats keystone:SELECT=100 {{(pid=81767) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 15:51:43.984538 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b1ca1a9-d87f-4070-a7e6-d568d568455f None admin] Validating token access rules against request {{(pid=81767) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 15:51:44.020189 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-678e6695-e358-4cc7-9852-858f6ac08c5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81767) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 15:51:44.020798 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-678e6695-e358-4cc7-9852-858f6ac08c5b None admin] REQUEST_METHOD: `GET` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:51:44.021116 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-678e6695-e358-4cc7-9852-858f6ac08c5b None admin] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:51:44.039351 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-678e6695-e358-4cc7-9852-858f6ac08c5b None admin] PATH_INFO: `/v3/services` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:51:44.039351 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-678e6695-e358-4cc7-9852-858f6ac08c5b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 15:51:44.039351 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-678e6695-e358-4cc7-9852-858f6ac08c5b None admin] RBAC: Authorization granted {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 15:51:44.039351 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 640/1301] 10.222.0.15 () {56 vars in 1087 bytes} [Mon Aug 22 15:51:43 2022] GET /identity/v3/services => generated 2191 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 22 15:51:46.009928 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-355a77d8-8140-4d74-b770-e377340566b9 None None] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:51:46.010454 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-355a77d8-8140-4d74-b770-e377340566b9 None None] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:51:46.010454 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-355a77d8-8140-4d74-b770-e377340566b9 None None] PATH_INFO: `/` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:51:46.011839 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 662/1302] 10.222.0.15 () {54 vars in 927 bytes} [Mon Aug 22 15:51:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 15:51:46.018066 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc014ff7-d556-4a0f-919c-d004ca05a994 None None] REQUEST_METHOD: `POST` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:51:46.018509 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc014ff7-d556-4a0f-919c-d004ca05a994 None None] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:51:46.018945 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc014ff7-d556-4a0f-919c-d004ca05a994 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:51:46.038870 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.auth.core [None req-fc014ff7-d556-4a0f-919c-d004ca05a994 None None] MFA Rules not processed for user `6b2199ae0fb0480aa917ada391f5ec54`. Rule list: `[]` (Enabled: `True`). {{(pid=81767) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 15:51:46.040106 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.fernet_utils [None req-fc014ff7-d556-4a0f-919c-d004ca05a994 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81767) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 15:51:46.109761 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 641/1303] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Aug 22 15:51:46 2022] POST /identity/v3/auth/tokens => generated 2221 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 22 15:51:46.392577 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-355a77d8-8140-4d74-b770-e377340566b9 None None] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 15:51:46.494274 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=100 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 15:51:46.500952 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-355a77d8-8140-4d74-b770-e377340566b9 None None] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 15:51:46.535768 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa79f03b-ecf9-4c87-9f6c-0bace82b4b08 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 15:51:46.536405 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa79f03b-ecf9-4c87-9f6c-0bace82b4b08 None admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:51:46.536728 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa79f03b-ecf9-4c87-9f6c-0bace82b4b08 None admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:51:46.536996 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa79f03b-ecf9-4c87-9f6c-0bace82b4b08 None admin] PATH_INFO: `/v3/services/placement` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:51:46.537571 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa79f03b-ecf9-4c87-9f6c-0bace82b4b08 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 15:51:46.538967 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa79f03b-ecf9-4c87-9f6c-0bace82b4b08 None admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 15:51:46.548239 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application [None req-fa79f03b-ecf9-4c87-9f6c-0bace82b4b08 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Aug 22 15:51:46.548239 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 15:51:46.548239 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 22 15:51:46.548239 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 22 15:51:46.548239 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 22 15:51:46.548239 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 22 15:51:46.548239 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 22 15:51:46.548239 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 22 15:51:46.548239 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 22 15:51:46.548239 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application Aug 22 15:51:46.548239 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 22 15:51:46.548239 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application Aug 22 15:51:46.548239 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 15:51:46.548239 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 15:51:46.548239 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 15:51:46.548239 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 15:51:46.548239 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 15:51:46.549714 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 15:51:46.549714 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 15:51:46.549714 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 15:51:46.549714 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 15:51:46.549714 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 15:51:46.549714 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 15:51:46.549714 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 22 15:51:46.549714 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 22 15:51:46.549714 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 22 15:51:46.549714 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 22 15:51:46.549714 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 15:51:46.549714 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 15:51:46.549714 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 15:51:46.549714 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 15:51:46.549714 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 15:51:46.549714 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 15:51:46.550618 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 15:51:46.550618 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application with Lock( Aug 22 15:51:46.550618 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 15:51:46.550618 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self._enter() Aug 22 15:51:46.550618 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 15:51:46.550618 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 15:51:46.550618 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 15:51:46.550618 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self.creator() Aug 22 15:51:46.550618 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 15:51:46.550618 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application created_value = creator( Aug 22 15:51:46.550618 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 22 15:51:46.550618 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 22 15:51:46.550618 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 22 15:51:46.550618 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application Aug 22 15:51:46.550618 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 663/1304] 10.222.0.15 () {56 vars in 1117 bytes} [Mon Aug 22 15:51:46 2022] GET /identity/v3/services/placement => generated 90 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 15:51:46.556042 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc014ff7-d556-4a0f-919c-d004ca05a994 None None] Authenticating user token {{(pid=81767) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 15:51:46.640578 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG dbcounter [-] [81767] Writing DB stats keystone:SELECT=100 {{(pid=81767) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 15:51:46.663568 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc014ff7-d556-4a0f-919c-d004ca05a994 None None] Validating token access rules against request {{(pid=81767) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 15:51:46.699292 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba07043b-9ee7-4beb-8d22-627028c23d26 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81767) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 15:51:46.699918 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba07043b-9ee7-4beb-8d22-627028c23d26 None admin] REQUEST_METHOD: `GET` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:51:46.700200 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba07043b-9ee7-4beb-8d22-627028c23d26 None admin] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:51:46.700485 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba07043b-9ee7-4beb-8d22-627028c23d26 None admin] PATH_INFO: `/v3/services` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:51:46.701176 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba07043b-9ee7-4beb-8d22-627028c23d26 None admin] RBAC: Adding query filter params (name=placement) {{(pid=81767) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 15:51:46.701497 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba07043b-9ee7-4beb-8d22-627028c23d26 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 15:51:46.702971 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba07043b-9ee7-4beb-8d22-627028c23d26 None admin] RBAC: Authorization granted {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 15:51:46.708239 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 642/1305] 10.222.0.15 () {56 vars in 1116 bytes} [Mon Aug 22 15:51:46 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 15:51:46.714117 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa79f03b-ecf9-4c87-9f6c-0bace82b4b08 None admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 15:51:46.818738 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa79f03b-ecf9-4c87-9f6c-0bace82b4b08 None admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 15:51:46.852637 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-250fc369-004c-4649-989d-c349082b5f01 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 15:51:46.853240 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-250fc369-004c-4649-989d-c349082b5f01 None admin] REQUEST_METHOD: `POST` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 15:51:46.853515 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-250fc369-004c-4649-989d-c349082b5f01 None admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 15:51:46.853783 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-250fc369-004c-4649-989d-c349082b5f01 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 15:51:46.854435 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-250fc369-004c-4649-989d-c349082b5f01 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 15:51:46.855833 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-250fc369-004c-4649-989d-c349082b5f01 None admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 15:51:47.079672 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 664/1306] 10.222.0.15 () {60 vars in 1144 bytes} [Mon Aug 22 15:51:46 2022] POST /identity/v3/endpoints => generated 331 bytes in 359 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 22 15:51:56.799583 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG dbcounter [-] [81767] Writing DB stats keystone:SELECT=30 {{(pid=81767) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 15:51:57.067505 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=91,keystone:INSERT=1 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 16:01:57.272427 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-836cda0c-6a4c-47a3-99e7-1e5fddeb9b9a None None] REQUEST_METHOD: `GET` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:01:57.274145 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-836cda0c-6a4c-47a3-99e7-1e5fddeb9b9a None None] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:01:57.274873 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-836cda0c-6a4c-47a3-99e7-1e5fddeb9b9a None None] PATH_INFO: `/` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:01:57.277468 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 643/1307] 10.222.0.15 () {54 vars in 927 bytes} [Mon Aug 22 16:01:57 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 16:01:57.286426 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c96dd84-a71a-4d12-a760-7de9bf702b50 None None] REQUEST_METHOD: `POST` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:01:57.287184 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c96dd84-a71a-4d12-a760-7de9bf702b50 None None] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:01:57.287572 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c96dd84-a71a-4d12-a760-7de9bf702b50 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:01:57.572033 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.auth.core [None req-7c96dd84-a71a-4d12-a760-7de9bf702b50 None None] MFA Rules not processed for user `6b2199ae0fb0480aa917ada391f5ec54`. Rule list: `[]` (Enabled: `True`). {{(pid=81768) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 16:01:57.574313 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-7c96dd84-a71a-4d12-a760-7de9bf702b50 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 16:01:57.864746 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 665/1308] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Aug 22 16:01:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 580 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 16:01:58.433515 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50bd175d-aa62-429e-a3bb-b93aeb547bcd None None] REQUEST_METHOD: `GET` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:01:58.434196 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50bd175d-aa62-429e-a3bb-b93aeb547bcd None None] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:01:58.434573 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50bd175d-aa62-429e-a3bb-b93aeb547bcd None None] PATH_INFO: `/` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:01:58.438340 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 644/1309] 10.222.0.15 () {54 vars in 965 bytes} [Mon Aug 22 16:01:58 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 16:01:58.449815 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b5ed4ca-0f6b-430c-b45e-59d70722f06a None None] REQUEST_METHOD: `POST` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:01:58.450474 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b5ed4ca-0f6b-430c-b45e-59d70722f06a None None] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:01:58.450769 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b5ed4ca-0f6b-430c-b45e-59d70722f06a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:01:58.639962 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.auth.core [None req-8b5ed4ca-0f6b-430c-b45e-59d70722f06a None None] MFA Rules not processed for user `234687adad594f1aa45775a565ffbdc7`. Rule list: `[]` (Enabled: `True`). {{(pid=81768) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 16:01:58.642263 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-8b5ed4ca-0f6b-430c-b45e-59d70722f06a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 16:01:58.691656 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=100 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 16:01:58.827903 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 666/1310] 10.222.0.15 () {58 vars in 1062 bytes} [Mon Aug 22 16:01:58 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 380 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 16:01:58.829293 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50bd175d-aa62-429e-a3bb-b93aeb547bcd None None] Authenticating user token {{(pid=81767) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 16:01:58.833098 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.fernet_utils [None req-50bd175d-aa62-429e-a3bb-b93aeb547bcd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81767) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 16:01:59.096257 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50bd175d-aa62-429e-a3bb-b93aeb547bcd None None] Validating token access rules against request {{(pid=81767) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 16:01:59.267077 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d2c48e9-ece0-40c4-b9f6-df9171b1c482 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '234687adad594f1aa45775a565ffbdc7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4ee0413f9ef4d4fb1f95a660babe090', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81767) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 16:01:59.268431 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d2c48e9-ece0-40c4-b9f6-df9171b1c482 service neutron] REQUEST_METHOD: `GET` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:01:59.268931 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d2c48e9-ece0-40c4-b9f6-df9171b1c482 service neutron] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:01:59.269342 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d2c48e9-ece0-40c4-b9f6-df9171b1c482 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:01:59.273248 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.fernet_utils [None req-3d2c48e9-ece0-40c4-b9f6-df9171b1c482 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81767) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 16:01:59.417438 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG dbcounter [-] [81767] Writing DB stats keystone:SELECT=100 {{(pid=81767) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 16:01:59.508214 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d2c48e9-ece0-40c4-b9f6-df9171b1c482 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 16:01:59.509888 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d2c48e9-ece0-40c4-b9f6-df9171b1c482 service neutron] RBAC: Authorization granted {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 16:01:59.535658 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 645/1311] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Aug 22 16:01:58 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 707 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 16:02:02.686974 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da56880-d4a9-4c5c-908b-8ada76768586 None None] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:02:02.687597 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da56880-d4a9-4c5c-908b-8ada76768586 None None] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:02:02.688019 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da56880-d4a9-4c5c-908b-8ada76768586 None None] PATH_INFO: `/` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:02:02.690822 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 667/1312] 10.222.0.15 () {54 vars in 927 bytes} [Mon Aug 22 16:02:02 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 16:02:02.701478 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-910889a1-a7d7-4e73-8867-62029569f083 None None] REQUEST_METHOD: `POST` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:02:02.702493 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-910889a1-a7d7-4e73-8867-62029569f083 None None] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:02:02.702794 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-910889a1-a7d7-4e73-8867-62029569f083 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:02:02.850200 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.auth.core [None req-910889a1-a7d7-4e73-8867-62029569f083 None None] MFA Rules not processed for user `6b2199ae0fb0480aa917ada391f5ec54`. Rule list: `[]` (Enabled: `True`). {{(pid=81767) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 16:02:02.852072 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.fernet_utils [None req-910889a1-a7d7-4e73-8867-62029569f083 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81767) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 16:02:03.016157 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 646/1313] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Aug 22 16:02:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 316 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 16:02:03.519042 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3da56880-d4a9-4c5c-908b-8ada76768586 None None] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 16:02:03.521972 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-3da56880-d4a9-4c5c-908b-8ada76768586 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 16:02:03.726558 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3da56880-d4a9-4c5c-908b-8ada76768586 None None] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 16:02:03.873850 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=100 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 16:02:03.879036 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d36965f-1229-4ba5-bc03-c42eb6a36150 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '234687adad594f1aa45775a565ffbdc7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4ee0413f9ef4d4fb1f95a660babe090', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 16:02:03.880117 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d36965f-1229-4ba5-bc03-c42eb6a36150 service neutron] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:02:03.881368 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d36965f-1229-4ba5-bc03-c42eb6a36150 service neutron] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:02:03.881788 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d36965f-1229-4ba5-bc03-c42eb6a36150 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:02:03.885114 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-6d36965f-1229-4ba5-bc03-c42eb6a36150 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 16:02:04.062887 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d36965f-1229-4ba5-bc03-c42eb6a36150 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 16:02:04.065735 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d36965f-1229-4ba5-bc03-c42eb6a36150 service neutron] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 16:02:04.098973 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 668/1314] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Aug 22 16:02:03 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 581 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 22 16:02:06.939031 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee7e9aa-196e-459c-96dd-6f57a1b26407 None None] REQUEST_METHOD: `GET` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:02:06.939707 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee7e9aa-196e-459c-96dd-6f57a1b26407 None None] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:02:06.939707 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee7e9aa-196e-459c-96dd-6f57a1b26407 None None] PATH_INFO: `/` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:02:06.941015 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 647/1315] 10.222.0.15 () {54 vars in 927 bytes} [Mon Aug 22 16:02:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 22 16:02:06.949839 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db6de601-061f-4f3a-b36d-24fe5b6c09ca None None] REQUEST_METHOD: `POST` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:02:06.950398 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db6de601-061f-4f3a-b36d-24fe5b6c09ca None None] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:02:06.950656 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db6de601-061f-4f3a-b36d-24fe5b6c09ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:02:07.072872 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.auth.core [None req-db6de601-061f-4f3a-b36d-24fe5b6c09ca None None] MFA Rules not processed for user `ad6aab8e70974f939bda68c2060aace5`. Rule list: `[]` (Enabled: `True`). {{(pid=81768) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 16:02:07.074197 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-db6de601-061f-4f3a-b36d-24fe5b6c09ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 16:02:07.137367 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=100 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 16:02:07.198281 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 669/1316] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Aug 22 16:02:06 2022] POST /identity/v3/auth/tokens => generated 3243 bytes in 250 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 16:02:07.507138 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ee7e9aa-196e-459c-96dd-6f57a1b26407 None None] Authenticating user token {{(pid=81767) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 16:02:07.508726 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.fernet_utils [None req-1ee7e9aa-196e-459c-96dd-6f57a1b26407 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81767) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 16:02:07.522219 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG dbcounter [-] [81767] Writing DB stats keystone:SELECT=100 {{(pid=81767) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 16:02:07.626031 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ee7e9aa-196e-459c-96dd-6f57a1b26407 None None] Validating token access rules against request {{(pid=81767) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 16:02:07.746800 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cec3ed40-6698-46ff-8ff9-84e002678751 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '234687adad594f1aa45775a565ffbdc7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4ee0413f9ef4d4fb1f95a660babe090', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81767) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 16:02:07.747644 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec3ed40-6698-46ff-8ff9-84e002678751 service neutron] REQUEST_METHOD: `GET` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:02:07.748117 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec3ed40-6698-46ff-8ff9-84e002678751 service neutron] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:02:07.748573 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec3ed40-6698-46ff-8ff9-84e002678751 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:02:07.750599 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.fernet_utils [None req-cec3ed40-6698-46ff-8ff9-84e002678751 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81767) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 16:02:07.865558 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG dbcounter [-] [81767] Writing DB stats keystone:SELECT=100 {{(pid=81767) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 16:02:07.953985 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cec3ed40-6698-46ff-8ff9-84e002678751 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 16:02:07.956374 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cec3ed40-6698-46ff-8ff9-84e002678751 service neutron] RBAC: Authorization granted {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 16:02:07.990181 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 648/1317] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Aug 22 16:02:07 2022] GET /identity/v3/auth/tokens => generated 3243 bytes in 483 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 16:02:11.399010 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-011445d0-9344-4100-b093-3855ff4e396b None None] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:02:11.399777 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-011445d0-9344-4100-b093-3855ff4e396b None None] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:02:11.400047 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-011445d0-9344-4100-b093-3855ff4e396b None None] PATH_INFO: `/` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:02:11.402378 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 670/1318] 10.222.0.15 () {54 vars in 927 bytes} [Mon Aug 22 16:02:11 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 16:02:11.411748 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16df19d9-3041-42ee-b429-44365ec72c9f None None] REQUEST_METHOD: `POST` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:02:11.412482 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16df19d9-3041-42ee-b429-44365ec72c9f None None] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:02:11.412673 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16df19d9-3041-42ee-b429-44365ec72c9f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:02:11.537957 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.auth.core [None req-16df19d9-3041-42ee-b429-44365ec72c9f None None] MFA Rules not processed for user `6b2199ae0fb0480aa917ada391f5ec54`. Rule list: `[]` (Enabled: `True`). {{(pid=81767) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 16:02:11.539891 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.fernet_utils [None req-16df19d9-3041-42ee-b429-44365ec72c9f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81767) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 16:02:11.749190 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 649/1319] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Aug 22 16:02:11 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 339 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 16:02:12.252253 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-011445d0-9344-4100-b093-3855ff4e396b None None] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 16:02:12.254737 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-011445d0-9344-4100-b093-3855ff4e396b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 16:02:12.461119 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-011445d0-9344-4100-b093-3855ff4e396b None None] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 16:02:12.614113 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a9a2aa5-9707-460f-908b-eadffaa93d9d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '234687adad594f1aa45775a565ffbdc7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4ee0413f9ef4d4fb1f95a660babe090', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 16:02:12.615732 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a9a2aa5-9707-460f-908b-eadffaa93d9d service neutron] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:02:12.616199 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a9a2aa5-9707-460f-908b-eadffaa93d9d service neutron] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:02:12.616746 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a9a2aa5-9707-460f-908b-eadffaa93d9d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:02:12.620155 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-2a9a2aa5-9707-460f-908b-eadffaa93d9d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 16:02:12.635540 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=100 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 16:02:12.844015 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a9a2aa5-9707-460f-908b-eadffaa93d9d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 16:02:12.847134 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a9a2aa5-9707-460f-908b-eadffaa93d9d service neutron] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 16:02:12.885420 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 671/1320] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Aug 22 16:02:12 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 634 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 16:02:16.832726 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92542bcd-89e2-4a21-af7a-1376fcda5714 None None] REQUEST_METHOD: `GET` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:02:16.832726 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92542bcd-89e2-4a21-af7a-1376fcda5714 None None] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:02:16.832726 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92542bcd-89e2-4a21-af7a-1376fcda5714 None None] PATH_INFO: `/` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:02:16.838302 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 650/1321] 10.222.0.15 () {54 vars in 927 bytes} [Mon Aug 22 16:02:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 22 16:02:16.840441 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4514e76-2e2b-475b-9dea-8548513c5ace None None] REQUEST_METHOD: `POST` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:02:16.841220 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4514e76-2e2b-475b-9dea-8548513c5ace None None] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:02:16.841220 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4514e76-2e2b-475b-9dea-8548513c5ace None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:02:16.931494 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.auth.core [None req-d4514e76-2e2b-475b-9dea-8548513c5ace None None] MFA Rules not processed for user `6b2199ae0fb0480aa917ada391f5ec54`. Rule list: `[]` (Enabled: `True`). {{(pid=81768) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 16:02:16.932853 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-d4514e76-2e2b-475b-9dea-8548513c5ace None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 16:02:17.026526 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=100 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 16:02:17.084078 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 672/1322] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Aug 22 16:02:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 245 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 22 16:02:17.428299 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ce382a6-3609-484c-9d02-a523f30b8d3e None None] REQUEST_METHOD: `GET` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:02:17.428697 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ce382a6-3609-484c-9d02-a523f30b8d3e None None] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:02:17.429030 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ce382a6-3609-484c-9d02-a523f30b8d3e None None] PATH_INFO: `/` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:02:17.430640 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 651/1323] 10.222.0.15 () {54 vars in 965 bytes} [Mon Aug 22 16:02:17 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 16:02:17.437482 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9917ac09-f1ae-4047-b490-ef1975737955 None None] REQUEST_METHOD: `POST` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:02:17.437911 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9917ac09-f1ae-4047-b490-ef1975737955 None None] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:02:17.438202 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9917ac09-f1ae-4047-b490-ef1975737955 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:02:17.552419 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.auth.core [None req-9917ac09-f1ae-4047-b490-ef1975737955 None None] MFA Rules not processed for user `234687adad594f1aa45775a565ffbdc7`. Rule list: `[]` (Enabled: `True`). {{(pid=81768) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 16:02:17.554298 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-9917ac09-f1ae-4047-b490-ef1975737955 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 16:02:17.701575 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 673/1324] 10.222.0.15 () {58 vars in 1062 bytes} [Mon Aug 22 16:02:17 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 265 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 22 16:02:17.713943 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ce382a6-3609-484c-9d02-a523f30b8d3e None None] Authenticating user token {{(pid=81767) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 16:02:17.715673 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.fernet_utils [None req-1ce382a6-3609-484c-9d02-a523f30b8d3e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81767) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 16:02:17.742963 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG dbcounter [-] [81767] Writing DB stats keystone:SELECT=100 {{(pid=81767) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 16:02:17.883617 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ce382a6-3609-484c-9d02-a523f30b8d3e None None] Validating token access rules against request {{(pid=81767) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 16:02:18.030715 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9d37d82-b66c-44fb-b831-9cfc5461b2ab service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '234687adad594f1aa45775a565ffbdc7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4ee0413f9ef4d4fb1f95a660babe090', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81767) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 16:02:18.031369 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d37d82-b66c-44fb-b831-9cfc5461b2ab service neutron] REQUEST_METHOD: `GET` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:02:18.031747 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d37d82-b66c-44fb-b831-9cfc5461b2ab service neutron] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:02:18.032083 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d37d82-b66c-44fb-b831-9cfc5461b2ab service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:02:18.034188 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.fernet_utils [None req-b9d37d82-b66c-44fb-b831-9cfc5461b2ab service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81767) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 16:02:18.159638 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG dbcounter [-] [81767] Writing DB stats keystone:SELECT=100 {{(pid=81767) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 16:02:18.233640 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9d37d82-b66c-44fb-b831-9cfc5461b2ab service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 16:02:18.236101 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9d37d82-b66c-44fb-b831-9cfc5461b2ab service neutron] RBAC: Authorization granted {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 16:02:18.272310 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 652/1325] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Aug 22 16:02:17 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 559 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 16:02:21.058267 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88e48d60-643d-4751-ab73-da1f3fde2335 None None] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:02:21.059089 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88e48d60-643d-4751-ab73-da1f3fde2335 None None] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:02:21.059089 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88e48d60-643d-4751-ab73-da1f3fde2335 None None] PATH_INFO: `/` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:02:21.060185 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 674/1326] 10.222.0.15 () {54 vars in 927 bytes} [Mon Aug 22 16:02:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 22 16:02:21.069250 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1c00825-aa91-446a-989c-74069bc7b6a2 None None] REQUEST_METHOD: `POST` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:02:21.069882 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1c00825-aa91-446a-989c-74069bc7b6a2 None None] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:02:21.070196 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1c00825-aa91-446a-989c-74069bc7b6a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:02:21.175372 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.auth.core [None req-e1c00825-aa91-446a-989c-74069bc7b6a2 None None] MFA Rules not processed for user `6b2199ae0fb0480aa917ada391f5ec54`. Rule list: `[]` (Enabled: `True`). {{(pid=81767) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 16:02:21.177552 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.fernet_utils [None req-e1c00825-aa91-446a-989c-74069bc7b6a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81767) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 16:02:21.358911 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 653/1327] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Aug 22 16:02:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 290 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 16:02:21.761811 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88e48d60-643d-4751-ab73-da1f3fde2335 None None] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 16:02:21.765046 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-88e48d60-643d-4751-ab73-da1f3fde2335 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 16:02:21.870751 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=100 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 16:02:21.967210 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88e48d60-643d-4751-ab73-da1f3fde2335 None None] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 16:02:22.119911 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e91084a-acd2-4c7c-965c-2c95e3e102db service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '234687adad594f1aa45775a565ffbdc7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4ee0413f9ef4d4fb1f95a660babe090', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 16:02:22.121010 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e91084a-acd2-4c7c-965c-2c95e3e102db service neutron] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:02:22.121450 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e91084a-acd2-4c7c-965c-2c95e3e102db service neutron] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:02:22.121921 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e91084a-acd2-4c7c-965c-2c95e3e102db service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:02:22.125068 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-1e91084a-acd2-4c7c-965c-2c95e3e102db service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 16:02:22.303118 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=100 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 16:02:22.306706 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e91084a-acd2-4c7c-965c-2c95e3e102db service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 16:02:22.308359 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e91084a-acd2-4c7c-965c-2c95e3e102db service neutron] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 16:02:22.339331 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 675/1328] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Aug 22 16:02:21 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 579 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 16:02:25.142192 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-040d5a5a-ea1f-499e-b073-cda47fbc3c35 None None] REQUEST_METHOD: `GET` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:02:25.142821 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-040d5a5a-ea1f-499e-b073-cda47fbc3c35 None None] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:02:25.143070 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-040d5a5a-ea1f-499e-b073-cda47fbc3c35 None None] PATH_INFO: `/` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:02:25.145760 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 654/1329] 10.222.0.15 () {54 vars in 927 bytes} [Mon Aug 22 16:02:25 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 16:02:25.151959 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-887a394c-d730-45cd-8ccb-21f5fddda94b None None] REQUEST_METHOD: `POST` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:02:25.152804 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-887a394c-d730-45cd-8ccb-21f5fddda94b None None] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:02:25.152977 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-887a394c-d730-45cd-8ccb-21f5fddda94b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:02:25.237722 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.auth.core [None req-887a394c-d730-45cd-8ccb-21f5fddda94b None None] MFA Rules not processed for user `ad6aab8e70974f939bda68c2060aace5`. Rule list: `[]` (Enabled: `True`). {{(pid=81768) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 16:02:25.239865 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-887a394c-d730-45cd-8ccb-21f5fddda94b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 16:02:25.402341 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 676/1330] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Aug 22 16:02:25 2022] POST /identity/v3/auth/tokens => generated 3243 bytes in 252 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 16:02:25.748209 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-040d5a5a-ea1f-499e-b073-cda47fbc3c35 None None] Authenticating user token {{(pid=81767) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 16:02:25.749746 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.fernet_utils [None req-040d5a5a-ea1f-499e-b073-cda47fbc3c35 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81767) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 16:02:25.807826 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG dbcounter [-] [81767] Writing DB stats keystone:SELECT=100 {{(pid=81767) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 16:02:25.918563 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-040d5a5a-ea1f-499e-b073-cda47fbc3c35 None None] Validating token access rules against request {{(pid=81767) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 16:02:26.025573 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6482fda9-fe65-4089-9a61-2319994a7bfe service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '234687adad594f1aa45775a565ffbdc7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4ee0413f9ef4d4fb1f95a660babe090', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81767) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 16:02:26.026592 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6482fda9-fe65-4089-9a61-2319994a7bfe service neutron] REQUEST_METHOD: `GET` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:02:26.027055 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6482fda9-fe65-4089-9a61-2319994a7bfe service neutron] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:02:26.027525 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6482fda9-fe65-4089-9a61-2319994a7bfe service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:02:26.029520 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.fernet_utils [None req-6482fda9-fe65-4089-9a61-2319994a7bfe service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81767) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 16:02:26.148682 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG dbcounter [-] [81767] Writing DB stats keystone:SELECT=100 {{(pid=81767) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 16:02:26.197884 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6482fda9-fe65-4089-9a61-2319994a7bfe service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 16:02:26.199467 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6482fda9-fe65-4089-9a61-2319994a7bfe service neutron] RBAC: Authorization granted {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 16:02:26.224017 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 655/1331] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Aug 22 16:02:25 2022] GET /identity/v3/auth/tokens => generated 3243 bytes in 476 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 22 16:02:29.711739 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16f04a59-3f13-4d02-b149-ca7c9880b985 None None] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:02:29.712288 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16f04a59-3f13-4d02-b149-ca7c9880b985 None None] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:02:29.712288 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16f04a59-3f13-4d02-b149-ca7c9880b985 None None] PATH_INFO: `/` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:02:29.713494 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 677/1332] 10.222.0.15 () {54 vars in 927 bytes} [Mon Aug 22 16:02:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 16:02:29.720089 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1e83887-cc68-4138-8f1c-1648baff4c1b None None] REQUEST_METHOD: `POST` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:02:29.720869 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1e83887-cc68-4138-8f1c-1648baff4c1b None None] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:02:29.721169 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1e83887-cc68-4138-8f1c-1648baff4c1b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:02:29.798833 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.auth.core [None req-b1e83887-cc68-4138-8f1c-1648baff4c1b None None] MFA Rules not processed for user `6b2199ae0fb0480aa917ada391f5ec54`. Rule list: `[]` (Enabled: `True`). {{(pid=81767) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 16:02:29.800128 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.fernet_utils [None req-b1e83887-cc68-4138-8f1c-1648baff4c1b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81767) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 16:02:29.907815 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 656/1333] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Aug 22 16:02:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 22 16:02:30.191156 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16f04a59-3f13-4d02-b149-ca7c9880b985 None None] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 16:02:30.193502 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-16f04a59-3f13-4d02-b149-ca7c9880b985 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 16:02:30.268881 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=100 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 16:02:30.315938 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16f04a59-3f13-4d02-b149-ca7c9880b985 None None] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 16:02:30.399083 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0be11517-f528-4ef9-997f-9db44e06f6d2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '234687adad594f1aa45775a565ffbdc7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4ee0413f9ef4d4fb1f95a660babe090', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 16:02:30.399800 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0be11517-f528-4ef9-997f-9db44e06f6d2 service neutron] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:02:30.400070 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0be11517-f528-4ef9-997f-9db44e06f6d2 service neutron] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:02:30.400338 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0be11517-f528-4ef9-997f-9db44e06f6d2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:02:30.402209 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-0be11517-f528-4ef9-997f-9db44e06f6d2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 16:02:30.579255 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0be11517-f528-4ef9-997f-9db44e06f6d2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 16:02:30.581703 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0be11517-f528-4ef9-997f-9db44e06f6d2 service neutron] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 16:02:30.599519 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=100 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 16:02:30.615910 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 678/1334] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Aug 22 16:02:30 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 425 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 16:02:33.026630 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e922db-2dd0-4761-8bc0-93b2ed6fa25b None None] REQUEST_METHOD: `GET` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:02:33.027238 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e922db-2dd0-4761-8bc0-93b2ed6fa25b None None] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:02:33.027566 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e922db-2dd0-4761-8bc0-93b2ed6fa25b None None] PATH_INFO: `/` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:02:33.029595 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 657/1335] 10.222.0.15 () {54 vars in 927 bytes} [Mon Aug 22 16:02:33 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 16:02:33.036002 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a17649d-19c0-4622-a522-05234bbc1b59 None None] REQUEST_METHOD: `POST` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:02:33.036571 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a17649d-19c0-4622-a522-05234bbc1b59 None None] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:02:33.036679 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a17649d-19c0-4622-a522-05234bbc1b59 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:02:33.120107 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.auth.core [None req-2a17649d-19c0-4622-a522-05234bbc1b59 None None] MFA Rules not processed for user `6b2199ae0fb0480aa917ada391f5ec54`. Rule list: `[]` (Enabled: `True`). {{(pid=81768) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 16:02:33.121247 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-2a17649d-19c0-4622-a522-05234bbc1b59 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 16:02:33.228337 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 679/1336] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Aug 22 16:02:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 22 16:02:33.527018 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0e922db-2dd0-4761-8bc0-93b2ed6fa25b None None] Authenticating user token {{(pid=81767) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 16:02:33.528627 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.fernet_utils [None req-e0e922db-2dd0-4761-8bc0-93b2ed6fa25b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81767) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 16:02:33.581851 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG dbcounter [-] [81767] Writing DB stats keystone:SELECT=100 {{(pid=81767) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 16:02:33.687831 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0e922db-2dd0-4761-8bc0-93b2ed6fa25b None None] Validating token access rules against request {{(pid=81767) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 16:02:33.793226 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0967e7d4-c930-4a02-866f-a8ebb43830ca service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '234687adad594f1aa45775a565ffbdc7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4ee0413f9ef4d4fb1f95a660babe090', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81767) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 16:02:33.793983 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0967e7d4-c930-4a02-866f-a8ebb43830ca service neutron] REQUEST_METHOD: `GET` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:02:33.794353 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0967e7d4-c930-4a02-866f-a8ebb43830ca service neutron] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:02:33.794729 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0967e7d4-c930-4a02-866f-a8ebb43830ca service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:02:33.796681 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.fernet_utils [None req-0967e7d4-c930-4a02-866f-a8ebb43830ca service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81767) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 16:02:33.920474 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG dbcounter [-] [81767] Writing DB stats keystone:SELECT=100 {{(pid=81767) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 16:02:33.972008 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0967e7d4-c930-4a02-866f-a8ebb43830ca service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 16:02:33.975244 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0967e7d4-c930-4a02-866f-a8ebb43830ca service neutron] RBAC: Authorization granted {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 16:02:34.012976 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 658/1337] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Aug 22 16:02:33 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 486 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 22 16:02:36.839368 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18dfe1e4-b1dd-4304-8e16-ded68e0a58eb None None] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:02:36.840043 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18dfe1e4-b1dd-4304-8e16-ded68e0a58eb None None] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:02:36.840248 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18dfe1e4-b1dd-4304-8e16-ded68e0a58eb None None] PATH_INFO: `/` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:02:36.842266 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 680/1338] 10.222.0.15 () {54 vars in 927 bytes} [Mon Aug 22 16:02:36 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 16:02:36.852311 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fc53714-2a36-4ec1-8e21-77bd6ae750b9 None None] REQUEST_METHOD: `POST` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:02:36.853135 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fc53714-2a36-4ec1-8e21-77bd6ae750b9 None None] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:02:36.853292 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fc53714-2a36-4ec1-8e21-77bd6ae750b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:02:36.941107 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.auth.core [None req-1fc53714-2a36-4ec1-8e21-77bd6ae750b9 None None] MFA Rules not processed for user `6b2199ae0fb0480aa917ada391f5ec54`. Rule list: `[]` (Enabled: `True`). {{(pid=81767) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 16:02:36.943480 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.fernet_utils [None req-1fc53714-2a36-4ec1-8e21-77bd6ae750b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81767) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 16:02:37.065085 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 659/1339] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Aug 22 16:02:36 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 214 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 16:02:37.372352 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18dfe1e4-b1dd-4304-8e16-ded68e0a58eb None None] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 16:02:37.375256 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-18dfe1e4-b1dd-4304-8e16-ded68e0a58eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 16:02:37.481435 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=100 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 16:02:37.514712 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18dfe1e4-b1dd-4304-8e16-ded68e0a58eb None None] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 16:02:37.601296 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c861af5a-0af2-4438-bcff-b0781ce13354 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '234687adad594f1aa45775a565ffbdc7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4ee0413f9ef4d4fb1f95a660babe090', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 16:02:37.601832 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c861af5a-0af2-4438-bcff-b0781ce13354 service neutron] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:02:37.602110 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c861af5a-0af2-4438-bcff-b0781ce13354 service neutron] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:02:37.602415 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c861af5a-0af2-4438-bcff-b0781ce13354 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:02:37.604584 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-c861af5a-0af2-4438-bcff-b0781ce13354 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 16:02:37.726894 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c861af5a-0af2-4438-bcff-b0781ce13354 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 16:02:37.728414 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c861af5a-0af2-4438-bcff-b0781ce13354 service neutron] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 16:02:37.747845 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 681/1340] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Aug 22 16:02:37 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 376 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 16:02:42.933317 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-885b679f-90b7-4ca3-b108-cd1b3ca277c1 None None] REQUEST_METHOD: `GET` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:02:42.934021 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-885b679f-90b7-4ca3-b108-cd1b3ca277c1 None None] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:02:42.934021 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-885b679f-90b7-4ca3-b108-cd1b3ca277c1 None None] PATH_INFO: `/` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:02:42.935518 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 660/1341] 10.222.0.15 () {54 vars in 927 bytes} [Mon Aug 22 16:02:42 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 16:02:42.942605 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-024634e2-9cde-4d6a-8470-672ffe19a8e7 None None] REQUEST_METHOD: `POST` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:02:42.943512 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-024634e2-9cde-4d6a-8470-672ffe19a8e7 None None] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:02:42.943681 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-024634e2-9cde-4d6a-8470-672ffe19a8e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:02:42.950808 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=100 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 16:02:43.033471 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.auth.core [None req-024634e2-9cde-4d6a-8470-672ffe19a8e7 None None] MFA Rules not processed for user `6b2199ae0fb0480aa917ada391f5ec54`. Rule list: `[]` (Enabled: `True`). {{(pid=81768) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 16:02:43.036617 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-024634e2-9cde-4d6a-8470-672ffe19a8e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 16:02:43.172611 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 682/1342] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Aug 22 16:02:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 232 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 22 16:02:43.496984 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-885b679f-90b7-4ca3-b108-cd1b3ca277c1 None None] Authenticating user token {{(pid=81767) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 16:02:43.499121 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.fernet_utils [None req-885b679f-90b7-4ca3-b108-cd1b3ca277c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81767) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 16:02:43.577693 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG dbcounter [-] [81767] Writing DB stats keystone:SELECT=100 {{(pid=81767) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 16:02:43.657782 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-885b679f-90b7-4ca3-b108-cd1b3ca277c1 None None] Validating token access rules against request {{(pid=81767) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 16:02:43.762994 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6073af20-a1a1-4f54-b724-4e12842938c2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '234687adad594f1aa45775a565ffbdc7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4ee0413f9ef4d4fb1f95a660babe090', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81767) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 16:02:43.763532 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6073af20-a1a1-4f54-b724-4e12842938c2 service neutron] REQUEST_METHOD: `GET` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:02:43.763879 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6073af20-a1a1-4f54-b724-4e12842938c2 service neutron] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:02:43.764154 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6073af20-a1a1-4f54-b724-4e12842938c2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:02:43.765928 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.fernet_utils [None req-6073af20-a1a1-4f54-b724-4e12842938c2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81767) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 16:02:43.884678 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG dbcounter [-] [81767] Writing DB stats keystone:SELECT=100 {{(pid=81767) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 16:02:43.905818 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6073af20-a1a1-4f54-b724-4e12842938c2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 16:02:43.907550 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6073af20-a1a1-4f54-b724-4e12842938c2 service neutron] RBAC: Authorization granted {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 16:02:43.930213 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 661/1343] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Aug 22 16:02:43 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 434 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 16:02:47.845199 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12e396e0-f814-4f5f-8aed-31598c962ca4 None None] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:02:47.846048 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12e396e0-f814-4f5f-8aed-31598c962ca4 None None] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:02:47.846048 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12e396e0-f814-4f5f-8aed-31598c962ca4 None None] PATH_INFO: `/` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:02:47.849958 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 683/1344] 10.222.0.15 () {54 vars in 927 bytes} [Mon Aug 22 16:02:47 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 16:02:47.855217 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-740b4627-9020-462e-9d55-b7742f98eb76 None None] REQUEST_METHOD: `POST` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:02:47.856077 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-740b4627-9020-462e-9d55-b7742f98eb76 None None] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:02:47.856077 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-740b4627-9020-462e-9d55-b7742f98eb76 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:02:47.962928 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.auth.core [None req-740b4627-9020-462e-9d55-b7742f98eb76 None None] MFA Rules not processed for user `6b2199ae0fb0480aa917ada391f5ec54`. Rule list: `[]` (Enabled: `True`). {{(pid=81767) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 16:02:47.965088 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.fernet_utils [None req-740b4627-9020-462e-9d55-b7742f98eb76 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81767) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 16:02:48.113267 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 662/1345] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Aug 22 16:02:47 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 260 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 22 16:02:48.526855 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12e396e0-f814-4f5f-8aed-31598c962ca4 None None] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 16:02:48.529750 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-12e396e0-f814-4f5f-8aed-31598c962ca4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 16:02:48.663760 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=100 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 16:02:48.687545 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12e396e0-f814-4f5f-8aed-31598c962ca4 None None] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 16:02:48.795145 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-125b5f27-2b11-490f-9168-50abd191c1af service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '234687adad594f1aa45775a565ffbdc7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4ee0413f9ef4d4fb1f95a660babe090', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 16:02:48.796177 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-125b5f27-2b11-490f-9168-50abd191c1af service neutron] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:02:48.796458 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-125b5f27-2b11-490f-9168-50abd191c1af service neutron] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:02:48.796739 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-125b5f27-2b11-490f-9168-50abd191c1af service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:02:48.798502 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-125b5f27-2b11-490f-9168-50abd191c1af service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 16:02:48.932912 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-125b5f27-2b11-490f-9168-50abd191c1af service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 16:02:48.934378 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-125b5f27-2b11-490f-9168-50abd191c1af service neutron] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 16:02:48.957245 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 684/1346] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Aug 22 16:02:48 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 431 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 16:02:54.133616 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3a9875a-4a5d-41ca-82f7-bd996abd0ef2 None None] REQUEST_METHOD: `GET` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:02:54.134449 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3a9875a-4a5d-41ca-82f7-bd996abd0ef2 None None] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:02:54.134449 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3a9875a-4a5d-41ca-82f7-bd996abd0ef2 None None] PATH_INFO: `/` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:02:54.135648 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 663/1347] 10.222.0.15 () {54 vars in 927 bytes} [Mon Aug 22 16:02:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 22 16:02:54.142883 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-693b7e6a-45a4-4d6d-b191-63acdc6541b4 None None] REQUEST_METHOD: `POST` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:02:54.143517 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-693b7e6a-45a4-4d6d-b191-63acdc6541b4 None None] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:02:54.144031 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-693b7e6a-45a4-4d6d-b191-63acdc6541b4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:02:54.173790 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=100 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 16:02:54.261449 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.auth.core [None req-693b7e6a-45a4-4d6d-b191-63acdc6541b4 None None] MFA Rules not processed for user `6b2199ae0fb0480aa917ada391f5ec54`. Rule list: `[]` (Enabled: `True`). {{(pid=81768) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 16:02:54.262700 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-693b7e6a-45a4-4d6d-b191-63acdc6541b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 16:02:54.429422 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 685/1348] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Aug 22 16:02:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 289 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 16:02:54.717615 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3a9875a-4a5d-41ca-82f7-bd996abd0ef2 None None] Authenticating user token {{(pid=81767) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 16:02:54.720586 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.fernet_utils [None req-a3a9875a-4a5d-41ca-82f7-bd996abd0ef2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81767) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 16:02:54.818742 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG dbcounter [-] [81767] Writing DB stats keystone:SELECT=100 {{(pid=81767) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 16:02:54.935511 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3a9875a-4a5d-41ca-82f7-bd996abd0ef2 None None] Validating token access rules against request {{(pid=81767) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 16:02:55.059332 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1847f9c-937d-4f25-810d-809380ca8a41 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '234687adad594f1aa45775a565ffbdc7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4ee0413f9ef4d4fb1f95a660babe090', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81767) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 16:02:55.060381 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1847f9c-937d-4f25-810d-809380ca8a41 service neutron] REQUEST_METHOD: `GET` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:02:55.060879 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1847f9c-937d-4f25-810d-809380ca8a41 service neutron] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:02:55.061376 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1847f9c-937d-4f25-810d-809380ca8a41 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:02:55.064558 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.fernet_utils [None req-e1847f9c-937d-4f25-810d-809380ca8a41 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81767) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 16:02:55.220647 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG dbcounter [-] [81767] Writing DB stats keystone:SELECT=100 {{(pid=81767) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 16:02:55.225422 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1847f9c-937d-4f25-810d-809380ca8a41 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 16:02:55.226888 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1847f9c-937d-4f25-810d-809380ca8a41 service neutron] RBAC: Authorization granted {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 16:02:55.247046 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 664/1349] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Aug 22 16:02:54 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 530 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 16:03:01.951977 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9360a9c3-4bb1-4bf7-90ad-6cb84dc50dc7 None None] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:03:01.953547 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9360a9c3-4bb1-4bf7-90ad-6cb84dc50dc7 None None] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:03:01.954222 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9360a9c3-4bb1-4bf7-90ad-6cb84dc50dc7 None None] PATH_INFO: `/` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:03:01.957562 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 686/1350] 10.222.0.15 () {54 vars in 921 bytes} [Mon Aug 22 16:03:01 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 16:03:01.966598 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a5098e-cbdc-42da-b850-6df843fa2ed2 None None] REQUEST_METHOD: `POST` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:03:01.967864 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a5098e-cbdc-42da-b850-6df843fa2ed2 None None] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:03:01.968565 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a5098e-cbdc-42da-b850-6df843fa2ed2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:03:02.141326 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.auth.core [None req-c3a5098e-cbdc-42da-b850-6df843fa2ed2 None None] MFA Rules not processed for user `08159bf8edda4246bb463f7b11b0894b`. Rule list: `[]` (Enabled: `True`). {{(pid=81767) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 16:03:02.143758 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.fernet_utils [None req-c3a5098e-cbdc-42da-b850-6df843fa2ed2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81767) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 16:03:02.427202 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 665/1351] 10.222.0.15 () {58 vars in 1018 bytes} [Mon Aug 22 16:03:01 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 462 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 16:03:04.426955 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=59 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 16:03:07.914362 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c92b9b8-6578-4656-9e3e-e0a71fe915c2 None None] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:03:07.914362 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c92b9b8-6578-4656-9e3e-e0a71fe915c2 None None] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:03:07.914362 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c92b9b8-6578-4656-9e3e-e0a71fe915c2 None None] PATH_INFO: `/` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:03:07.916320 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 687/1352] 10.222.0.15 () {54 vars in 921 bytes} [Mon Aug 22 16:03:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 22 16:03:07.923119 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b25de52-7052-4802-af29-d4c38063ecc9 None None] REQUEST_METHOD: `POST` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:03:07.924257 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b25de52-7052-4802-af29-d4c38063ecc9 None None] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:03:07.925082 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b25de52-7052-4802-af29-d4c38063ecc9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:03:08.036770 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG dbcounter [-] [81767] Writing DB stats keystone:SELECT=100 {{(pid=81767) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 16:03:08.083939 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.auth.core [None req-3b25de52-7052-4802-af29-d4c38063ecc9 None None] MFA Rules not processed for user `08159bf8edda4246bb463f7b11b0894b`. Rule list: `[]` (Enabled: `True`). {{(pid=81767) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 16:03:08.086341 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.fernet_utils [None req-3b25de52-7052-4802-af29-d4c38063ecc9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81767) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 16:03:08.311546 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 666/1353] 10.222.0.15 () {58 vars in 1018 bytes} [Mon Aug 22 16:03:07 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 390 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 22 16:03:09.795521 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3a66d45-4ad9-41c1-bce6-5fbdf8eaa916 None None] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:03:09.796340 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3a66d45-4ad9-41c1-bce6-5fbdf8eaa916 None None] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:03:09.796340 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3a66d45-4ad9-41c1-bce6-5fbdf8eaa916 None None] PATH_INFO: `/` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:03:09.797800 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 688/1354] 10.222.0.15 () {54 vars in 921 bytes} [Mon Aug 22 16:03:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 22 16:03:09.805257 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-486dc0d7-fbde-4696-8733-fc06db9c5004 None None] REQUEST_METHOD: `POST` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:03:09.805859 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-486dc0d7-fbde-4696-8733-fc06db9c5004 None None] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:03:09.806198 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-486dc0d7-fbde-4696-8733-fc06db9c5004 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:03:09.913508 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.auth.core [None req-486dc0d7-fbde-4696-8733-fc06db9c5004 None None] MFA Rules not processed for user `08159bf8edda4246bb463f7b11b0894b`. Rule list: `[]` (Enabled: `True`). {{(pid=81767) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 16:03:09.914753 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.fernet_utils [None req-486dc0d7-fbde-4696-8733-fc06db9c5004 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81767) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 16:03:09.971990 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG dbcounter [-] [81767] Writing DB stats keystone:SELECT=100 {{(pid=81767) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 16:03:10.088292 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 667/1355] 10.222.0.15 () {58 vars in 1018 bytes} [Mon Aug 22 16:03:09 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 285 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 16:03:13.877323 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7255d9a1-61eb-42eb-9b5c-dc44ba85fe9e None None] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:03:13.877975 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7255d9a1-61eb-42eb-9b5c-dc44ba85fe9e None None] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:03:13.877975 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7255d9a1-61eb-42eb-9b5c-dc44ba85fe9e None None] PATH_INFO: `/` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:03:13.879265 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 689/1356] 10.222.0.15 () {54 vars in 927 bytes} [Mon Aug 22 16:03:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 16:03:13.885764 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc08e5eb-e1f4-4dcb-8a2b-e7a46356865d None None] REQUEST_METHOD: `POST` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:03:13.886590 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc08e5eb-e1f4-4dcb-8a2b-e7a46356865d None None] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:03:13.886754 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc08e5eb-e1f4-4dcb-8a2b-e7a46356865d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:03:13.971726 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.auth.core [None req-cc08e5eb-e1f4-4dcb-8a2b-e7a46356865d None None] MFA Rules not processed for user `6b2199ae0fb0480aa917ada391f5ec54`. Rule list: `[]` (Enabled: `True`). {{(pid=81767) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 16:03:13.973590 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.fernet_utils [None req-cc08e5eb-e1f4-4dcb-8a2b-e7a46356865d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81767) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 16:03:14.130263 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 668/1357] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Aug 22 16:03:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 246 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 16:03:14.714367 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22ce18c1-a7e7-4d27-a53d-cca085e3a74e None None] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:03:14.714690 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22ce18c1-a7e7-4d27-a53d-cca085e3a74e None None] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:03:14.714910 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22ce18c1-a7e7-4d27-a53d-cca085e3a74e None None] PATH_INFO: `/` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:03:14.716758 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 690/1358] 10.222.0.15 () {54 vars in 962 bytes} [Mon Aug 22 16:03:14 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 16:03:14.732279 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5686076d-2bef-416f-aa9b-045dda49db05 None None] REQUEST_METHOD: `POST` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:03:14.733020 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5686076d-2bef-416f-aa9b-045dda49db05 None None] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:03:14.733644 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5686076d-2bef-416f-aa9b-045dda49db05 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:03:14.766891 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG dbcounter [-] [81767] Writing DB stats keystone:SELECT=100 {{(pid=81767) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 16:03:14.847796 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.auth.core [None req-5686076d-2bef-416f-aa9b-045dda49db05 None None] MFA Rules not processed for user `e4ef569e41fb46d0b721bb1c7663616f`. Rule list: `[]` (Enabled: `True`). {{(pid=81767) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 16:03:14.849995 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.fernet_utils [None req-5686076d-2bef-416f-aa9b-045dda49db05 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81767) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 16:03:15.011173 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 669/1359] 10.222.0.15 () {58 vars in 1059 bytes} [Mon Aug 22 16:03:14 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 281 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 16:03:15.028774 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22ce18c1-a7e7-4d27-a53d-cca085e3a74e None None] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 16:03:15.032869 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-22ce18c1-a7e7-4d27-a53d-cca085e3a74e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 16:03:15.225717 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22ce18c1-a7e7-4d27-a53d-cca085e3a74e None None] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 16:03:15.316429 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=100 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 16:03:15.374293 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8bdf84e-2b81-47b2-812b-8311c9325a6c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4ef569e41fb46d0b721bb1c7663616f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4ee0413f9ef4d4fb1f95a660babe090', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 16:03:15.375015 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8bdf84e-2b81-47b2-812b-8311c9325a6c service nova] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:03:15.375398 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8bdf84e-2b81-47b2-812b-8311c9325a6c service nova] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:03:15.375749 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8bdf84e-2b81-47b2-812b-8311c9325a6c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:03:15.377525 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-c8bdf84e-2b81-47b2-812b-8311c9325a6c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 16:03:15.499835 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8bdf84e-2b81-47b2-812b-8311c9325a6c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 16:03:15.502189 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8bdf84e-2b81-47b2-812b-8311c9325a6c service nova] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 16:03:15.527892 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 691/1360] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Aug 22 16:03:15 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 500 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 22 16:03:17.848869 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6849ac9-245b-4286-80e5-42635972f51b None None] REQUEST_METHOD: `GET` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:03:17.849444 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6849ac9-245b-4286-80e5-42635972f51b None None] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:03:17.849444 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6849ac9-245b-4286-80e5-42635972f51b None None] PATH_INFO: `/` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:03:17.850954 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 670/1361] 10.222.0.15 () {54 vars in 927 bytes} [Mon Aug 22 16:03:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 16:03:17.856445 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ce0149f-41b8-40df-9d94-a1726f13ad87 None None] REQUEST_METHOD: `POST` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:03:17.856998 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ce0149f-41b8-40df-9d94-a1726f13ad87 None None] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:03:17.856998 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ce0149f-41b8-40df-9d94-a1726f13ad87 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:03:17.940298 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.auth.core [None req-9ce0149f-41b8-40df-9d94-a1726f13ad87 None None] MFA Rules not processed for user `6b2199ae0fb0480aa917ada391f5ec54`. Rule list: `[]` (Enabled: `True`). {{(pid=81768) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 16:03:17.942089 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-9ce0149f-41b8-40df-9d94-a1726f13ad87 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 16:03:18.009577 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=100 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 16:03:18.131187 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 692/1362] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Aug 22 16:03:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 276 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 16:03:18.444740 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6849ac9-245b-4286-80e5-42635972f51b None None] Authenticating user token {{(pid=81767) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 16:03:18.446742 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.fernet_utils [None req-c6849ac9-245b-4286-80e5-42635972f51b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81767) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 16:03:18.537121 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG dbcounter [-] [81767] Writing DB stats keystone:SELECT=100 {{(pid=81767) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 16:03:18.658255 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6849ac9-245b-4286-80e5-42635972f51b None None] Validating token access rules against request {{(pid=81767) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 16:03:18.867871 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a92bdc6-2729-4a8e-b947-d874b866016e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4ef569e41fb46d0b721bb1c7663616f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4ee0413f9ef4d4fb1f95a660babe090', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81767) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 16:03:18.868753 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a92bdc6-2729-4a8e-b947-d874b866016e service nova] REQUEST_METHOD: `GET` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:03:18.868753 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a92bdc6-2729-4a8e-b947-d874b866016e service nova] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:03:18.869063 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a92bdc6-2729-4a8e-b947-d874b866016e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:03:18.870736 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.fernet_utils [None req-2a92bdc6-2729-4a8e-b947-d874b866016e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81767) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 16:03:18.897885 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG dbcounter [-] [81767] Writing DB stats keystone:SELECT=100 {{(pid=81767) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 16:03:19.036879 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a92bdc6-2729-4a8e-b947-d874b866016e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 16:03:19.039737 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a92bdc6-2729-4a8e-b947-d874b866016e service nova] RBAC: Authorization granted {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 16:03:19.077272 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 671/1363] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Aug 22 16:03:18 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 633 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 16:03:21.900880 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5adbac0-173b-4ad7-bac7-001d985473a0 None None] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:03:21.901628 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5adbac0-173b-4ad7-bac7-001d985473a0 None None] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:03:21.901628 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5adbac0-173b-4ad7-bac7-001d985473a0 None None] PATH_INFO: `/` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:03:21.903106 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 693/1364] 10.222.0.15 () {54 vars in 927 bytes} [Mon Aug 22 16:03:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 16:03:21.909738 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0941fcf9-2c42-4265-8e4f-e0acfa9ffd2b None None] REQUEST_METHOD: `POST` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:03:21.910557 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0941fcf9-2c42-4265-8e4f-e0acfa9ffd2b None None] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:03:21.910557 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0941fcf9-2c42-4265-8e4f-e0acfa9ffd2b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:03:21.990282 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.auth.core [None req-0941fcf9-2c42-4265-8e4f-e0acfa9ffd2b None None] MFA Rules not processed for user `6b2199ae0fb0480aa917ada391f5ec54`. Rule list: `[]` (Enabled: `True`). {{(pid=81767) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 16:03:21.992456 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.fernet_utils [None req-0941fcf9-2c42-4265-8e4f-e0acfa9ffd2b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81767) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 16:03:22.103426 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG dbcounter [-] [81767] Writing DB stats keystone:SELECT=100 {{(pid=81767) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 16:03:22.106553 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 672/1365] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Aug 22 16:03:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 198 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 16:03:22.459049 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dedfaf2b-54cd-48f6-a109-92de52fd6bd3 None None] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:03:22.459624 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dedfaf2b-54cd-48f6-a109-92de52fd6bd3 None None] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:03:22.460071 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dedfaf2b-54cd-48f6-a109-92de52fd6bd3 None None] PATH_INFO: `/` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:03:22.461959 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 694/1366] 10.222.0.15 () {54 vars in 962 bytes} [Mon Aug 22 16:03:22 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 16:03:22.468468 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ebf62cd-c0a2-4d2d-910c-bce0964def77 None None] REQUEST_METHOD: `POST` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:03:22.468838 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ebf62cd-c0a2-4d2d-910c-bce0964def77 None None] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:03:22.469159 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ebf62cd-c0a2-4d2d-910c-bce0964def77 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:03:22.597229 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.auth.core [None req-1ebf62cd-c0a2-4d2d-910c-bce0964def77 None None] MFA Rules not processed for user `e4ef569e41fb46d0b721bb1c7663616f`. Rule list: `[]` (Enabled: `True`). {{(pid=81767) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 16:03:22.598898 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.fernet_utils [None req-1ebf62cd-c0a2-4d2d-910c-bce0964def77 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81767) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 16:03:22.741680 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 673/1367] 10.222.0.15 () {58 vars in 1059 bytes} [Mon Aug 22 16:03:22 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 274 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 16:03:22.749836 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dedfaf2b-54cd-48f6-a109-92de52fd6bd3 None None] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 16:03:22.751405 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-dedfaf2b-54cd-48f6-a109-92de52fd6bd3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 16:03:22.918245 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dedfaf2b-54cd-48f6-a109-92de52fd6bd3 None None] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 16:03:22.940696 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=100 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 16:03:23.047393 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-286b0347-917a-4adf-92ab-abec10c68245 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4ef569e41fb46d0b721bb1c7663616f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4ee0413f9ef4d4fb1f95a660babe090', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 16:03:23.048000 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-286b0347-917a-4adf-92ab-abec10c68245 service nova] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:03:23.048277 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-286b0347-917a-4adf-92ab-abec10c68245 service nova] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:03:23.048540 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-286b0347-917a-4adf-92ab-abec10c68245 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:03:23.050150 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-286b0347-917a-4adf-92ab-abec10c68245 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 16:03:23.145732 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-286b0347-917a-4adf-92ab-abec10c68245 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 16:03:23.147241 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-286b0347-917a-4adf-92ab-abec10c68245 service nova] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 16:03:23.166146 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 695/1368] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Aug 22 16:03:22 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 417 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 16:03:23.332336 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5325eb5a-ed9b-4fd9-a925-f119e840c3fe None None] REQUEST_METHOD: `GET` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:03:23.333129 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5325eb5a-ed9b-4fd9-a925-f119e840c3fe None None] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:03:23.333272 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5325eb5a-ed9b-4fd9-a925-f119e840c3fe None None] PATH_INFO: `/` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:03:23.338946 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 674/1369] 10.222.0.15 () {54 vars in 927 bytes} [Mon Aug 22 16:03:23 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 16:03:23.341276 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3126e79-363f-4c50-8851-624813e97e04 None None] REQUEST_METHOD: `POST` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:03:23.341551 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3126e79-363f-4c50-8851-624813e97e04 None None] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:03:23.341819 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3126e79-363f-4c50-8851-624813e97e04 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:03:23.381035 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=100 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 16:03:23.420567 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.auth.core [None req-a3126e79-363f-4c50-8851-624813e97e04 None None] MFA Rules not processed for user `6b2199ae0fb0480aa917ada391f5ec54`. Rule list: `[]` (Enabled: `True`). {{(pid=81768) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 16:03:23.421757 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-a3126e79-363f-4c50-8851-624813e97e04 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 16:03:23.558157 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 696/1370] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Aug 22 16:03:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 218 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 16:03:24.141462 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e2520df-d3a7-4f63-83f5-0e20c1c0a5f0 None None] REQUEST_METHOD: `GET` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:03:24.141797 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e2520df-d3a7-4f63-83f5-0e20c1c0a5f0 None None] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:03:24.142046 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e2520df-d3a7-4f63-83f5-0e20c1c0a5f0 None None] PATH_INFO: `/` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:03:24.143538 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 675/1371] 10.222.0.15 () {54 vars in 964 bytes} [Mon Aug 22 16:03:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 16:03:24.156433 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94db3e4f-7330-4292-a58e-a82318791630 None None] REQUEST_METHOD: `POST` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:03:24.157228 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94db3e4f-7330-4292-a58e-a82318791630 None None] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:03:24.157599 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94db3e4f-7330-4292-a58e-a82318791630 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:03:24.247111 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.auth.core [None req-94db3e4f-7330-4292-a58e-a82318791630 None None] MFA Rules not processed for user `79706d480f864998b9f8594be407ceab`. Rule list: `[]` (Enabled: `True`). {{(pid=81768) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 16:03:24.248518 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-94db3e4f-7330-4292-a58e-a82318791630 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 16:03:24.300633 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=100 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 16:03:24.334673 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 697/1372] 10.222.0.15 () {58 vars in 1061 bytes} [Mon Aug 22 16:03:24 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 16:03:24.347903 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e2520df-d3a7-4f63-83f5-0e20c1c0a5f0 None None] Authenticating user token {{(pid=81767) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 16:03:24.347903 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.fernet_utils [None req-1e2520df-d3a7-4f63-83f5-0e20c1c0a5f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81767) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 16:03:24.400651 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG dbcounter [-] [81767] Writing DB stats keystone:SELECT=100 {{(pid=81767) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 16:03:24.460100 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e2520df-d3a7-4f63-83f5-0e20c1c0a5f0 None None] Validating token access rules against request {{(pid=81767) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 16:03:24.553893 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9f24421-df36-46ed-a01e-d3612a9e91fc service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79706d480f864998b9f8594be407ceab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4ee0413f9ef4d4fb1f95a660babe090', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81767) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 16:03:24.554607 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9f24421-df36-46ed-a01e-d3612a9e91fc service cinder] REQUEST_METHOD: `GET` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:03:24.555030 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9f24421-df36-46ed-a01e-d3612a9e91fc service cinder] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:03:24.555498 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9f24421-df36-46ed-a01e-d3612a9e91fc service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:03:24.557138 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.fernet_utils [None req-d9f24421-df36-46ed-a01e-d3612a9e91fc service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81767) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 16:03:24.670582 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG dbcounter [-] [81767] Writing DB stats keystone:SELECT=100 {{(pid=81767) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 16:03:24.688141 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9f24421-df36-46ed-a01e-d3612a9e91fc service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 16:03:24.689480 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9f24421-df36-46ed-a01e-d3612a9e91fc service cinder] RBAC: Authorization granted {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 16:03:24.716498 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 676/1373] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Aug 22 16:03:24 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 370 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 16:03:25.457721 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cfea17f-309c-4ba1-93a6-21ff54b300af None None] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:03:25.458319 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cfea17f-309c-4ba1-93a6-21ff54b300af None None] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:03:25.458319 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cfea17f-309c-4ba1-93a6-21ff54b300af None None] PATH_INFO: `/` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:03:25.459673 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 698/1374] 10.222.0.15 () {54 vars in 927 bytes} [Mon Aug 22 16:03:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 16:03:25.465746 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd301721-dd69-4bb6-b002-65e089c8fd6f None None] REQUEST_METHOD: `POST` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:03:25.466239 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd301721-dd69-4bb6-b002-65e089c8fd6f None None] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:03:25.466368 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd301721-dd69-4bb6-b002-65e089c8fd6f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:03:25.540012 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.auth.core [None req-cd301721-dd69-4bb6-b002-65e089c8fd6f None None] MFA Rules not processed for user `6b2199ae0fb0480aa917ada391f5ec54`. Rule list: `[]` (Enabled: `True`). {{(pid=81767) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 16:03:25.541248 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.fernet_utils [None req-cd301721-dd69-4bb6-b002-65e089c8fd6f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81767) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 16:03:25.663770 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 677/1375] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Aug 22 16:03:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 199 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 16:03:25.967630 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cfea17f-309c-4ba1-93a6-21ff54b300af None None] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 16:03:25.969240 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-3cfea17f-309c-4ba1-93a6-21ff54b300af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 16:03:26.142496 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cfea17f-309c-4ba1-93a6-21ff54b300af None None] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 16:03:26.211133 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=100 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 16:03:26.322977 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-434b1706-4d86-4eb6-8741-c345df5eb959 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4ef569e41fb46d0b721bb1c7663616f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4ee0413f9ef4d4fb1f95a660babe090', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 16:03:26.323961 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-434b1706-4d86-4eb6-8741-c345df5eb959 service nova] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:03:26.324394 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-434b1706-4d86-4eb6-8741-c345df5eb959 service nova] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:03:26.325501 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-434b1706-4d86-4eb6-8741-c345df5eb959 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:03:26.327579 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-434b1706-4d86-4eb6-8741-c345df5eb959 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 16:03:26.486578 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-434b1706-4d86-4eb6-8741-c345df5eb959 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 16:03:26.488922 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-434b1706-4d86-4eb6-8741-c345df5eb959 service nova] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 16:03:26.519504 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 699/1376] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Aug 22 16:03:25 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 552 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 22 16:03:28.856419 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25643bfd-6fb3-4707-b304-da8fdf4954cb None None] REQUEST_METHOD: `GET` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:03:28.857325 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25643bfd-6fb3-4707-b304-da8fdf4954cb None None] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:03:28.857325 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25643bfd-6fb3-4707-b304-da8fdf4954cb None None] PATH_INFO: `/` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:03:28.858933 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 678/1377] 10.222.0.15 () {54 vars in 927 bytes} [Mon Aug 22 16:03:28 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 16:03:28.867664 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cffae87c-af8c-438d-ad90-dcc65ad3f69c None None] REQUEST_METHOD: `POST` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:03:28.869054 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cffae87c-af8c-438d-ad90-dcc65ad3f69c None None] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:03:28.869580 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cffae87c-af8c-438d-ad90-dcc65ad3f69c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:03:28.975927 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.auth.core [None req-cffae87c-af8c-438d-ad90-dcc65ad3f69c None None] MFA Rules not processed for user `6b2199ae0fb0480aa917ada391f5ec54`. Rule list: `[]` (Enabled: `True`). {{(pid=81768) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 16:03:28.977321 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-cffae87c-af8c-438d-ad90-dcc65ad3f69c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 16:03:28.986856 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=100 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 16:03:29.133793 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 700/1378] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Aug 22 16:03:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 268 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 16:03:29.675387 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25643bfd-6fb3-4707-b304-da8fdf4954cb None None] Authenticating user token {{(pid=81767) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 16:03:29.677336 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.fernet_utils [None req-25643bfd-6fb3-4707-b304-da8fdf4954cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81767) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 16:03:29.774975 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG dbcounter [-] [81767] Writing DB stats keystone:SELECT=100 {{(pid=81767) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 16:03:29.883743 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25643bfd-6fb3-4707-b304-da8fdf4954cb None None] Validating token access rules against request {{(pid=81767) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 16:03:30.061138 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd72f744-fb41-461c-bace-aef9e0e7e88e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4ef569e41fb46d0b721bb1c7663616f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4ee0413f9ef4d4fb1f95a660babe090', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81767) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 16:03:30.067899 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd72f744-fb41-461c-bace-aef9e0e7e88e service nova] REQUEST_METHOD: `GET` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:03:30.067899 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd72f744-fb41-461c-bace-aef9e0e7e88e service nova] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:03:30.067899 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd72f744-fb41-461c-bace-aef9e0e7e88e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:03:30.067899 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.fernet_utils [None req-cd72f744-fb41-461c-bace-aef9e0e7e88e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81767) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 16:03:30.087057 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG dbcounter [-] [81767] Writing DB stats keystone:SELECT=100 {{(pid=81767) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 16:03:30.180551 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd72f744-fb41-461c-bace-aef9e0e7e88e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 16:03:30.182589 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd72f744-fb41-461c-bace-aef9e0e7e88e service nova] RBAC: Authorization granted {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 16:03:30.218142 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 679/1379] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Aug 22 16:03:29 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 544 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 16:03:32.595957 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61c4fe48-db96-4ef0-b956-3b2853a4decd None None] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:03:32.596809 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61c4fe48-db96-4ef0-b956-3b2853a4decd None None] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:03:32.596809 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61c4fe48-db96-4ef0-b956-3b2853a4decd None None] PATH_INFO: `/` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:03:32.598809 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 701/1380] 10.222.0.15 () {54 vars in 927 bytes} [Mon Aug 22 16:03:32 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 16:03:32.605983 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed4855ed-4a4e-4e8a-be31-18cf8813655c None None] REQUEST_METHOD: `POST` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:03:32.606555 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed4855ed-4a4e-4e8a-be31-18cf8813655c None None] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:03:32.606555 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed4855ed-4a4e-4e8a-be31-18cf8813655c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:03:32.715680 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.auth.core [None req-ed4855ed-4a4e-4e8a-be31-18cf8813655c None None] MFA Rules not processed for user `6b2199ae0fb0480aa917ada391f5ec54`. Rule list: `[]` (Enabled: `True`). {{(pid=81767) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 16:03:32.717009 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.fernet_utils [None req-ed4855ed-4a4e-4e8a-be31-18cf8813655c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81767) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 16:03:32.840863 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG dbcounter [-] [81767] Writing DB stats keystone:SELECT=100 {{(pid=81767) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 16:03:32.858943 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 680/1381] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Aug 22 16:03:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 249 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 16:03:33.219572 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61c4fe48-db96-4ef0-b956-3b2853a4decd None None] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 16:03:33.222041 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-61c4fe48-db96-4ef0-b956-3b2853a4decd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 16:03:33.406386 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=100 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 16:03:33.419651 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61c4fe48-db96-4ef0-b956-3b2853a4decd None None] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 16:03:33.559042 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e47d28d8-f4e7-4d41-92e0-a07d7584ca07 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4ef569e41fb46d0b721bb1c7663616f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4ee0413f9ef4d4fb1f95a660babe090', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 16:03:33.562344 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e47d28d8-f4e7-4d41-92e0-a07d7584ca07 service nova] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:03:33.562344 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e47d28d8-f4e7-4d41-92e0-a07d7584ca07 service nova] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:03:33.562344 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e47d28d8-f4e7-4d41-92e0-a07d7584ca07 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:03:33.562344 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-e47d28d8-f4e7-4d41-92e0-a07d7584ca07 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 16:03:33.674278 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e47d28d8-f4e7-4d41-92e0-a07d7584ca07 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 16:03:33.676112 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e47d28d8-f4e7-4d41-92e0-a07d7584ca07 service nova] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 16:03:33.692620 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=100 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 16:03:33.703988 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 702/1382] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Aug 22 16:03:33 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 485 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 22 16:03:36.271350 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1778704-2fae-4bab-ab72-1b2082f303f2 None None] REQUEST_METHOD: `GET` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:03:36.272488 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1778704-2fae-4bab-ab72-1b2082f303f2 None None] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:03:36.272488 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1778704-2fae-4bab-ab72-1b2082f303f2 None None] PATH_INFO: `/` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:03:36.274342 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 681/1383] 10.222.0.15 () {54 vars in 927 bytes} [Mon Aug 22 16:03:36 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 16:03:36.282174 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ed3261b-25d5-47db-b4fe-5c4868053246 None None] REQUEST_METHOD: `POST` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:03:36.282785 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ed3261b-25d5-47db-b4fe-5c4868053246 None None] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:03:36.283171 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ed3261b-25d5-47db-b4fe-5c4868053246 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:03:36.397102 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.auth.core [None req-4ed3261b-25d5-47db-b4fe-5c4868053246 None None] MFA Rules not processed for user `6b2199ae0fb0480aa917ada391f5ec54`. Rule list: `[]` (Enabled: `True`). {{(pid=81768) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 16:03:36.398386 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-4ed3261b-25d5-47db-b4fe-5c4868053246 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 16:03:36.535202 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 703/1384] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Aug 22 16:03:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 255 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 16:03:36.977715 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1778704-2fae-4bab-ab72-1b2082f303f2 None None] Authenticating user token {{(pid=81767) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 16:03:36.979905 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.fernet_utils [None req-e1778704-2fae-4bab-ab72-1b2082f303f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81767) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 16:03:37.206849 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1778704-2fae-4bab-ab72-1b2082f303f2 None None] Validating token access rules against request {{(pid=81767) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 16:03:37.329744 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG dbcounter [-] [81767] Writing DB stats keystone:SELECT=100 {{(pid=81767) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 16:03:37.406663 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8afaaa1-9ebe-4ec6-8c18-2f5636c030db service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4ef569e41fb46d0b721bb1c7663616f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4ee0413f9ef4d4fb1f95a660babe090', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81767) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 16:03:37.407324 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8afaaa1-9ebe-4ec6-8c18-2f5636c030db service nova] REQUEST_METHOD: `GET` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:03:37.407617 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8afaaa1-9ebe-4ec6-8c18-2f5636c030db service nova] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:03:37.407913 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8afaaa1-9ebe-4ec6-8c18-2f5636c030db service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:03:37.409817 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.fernet_utils [None req-c8afaaa1-9ebe-4ec6-8c18-2f5636c030db service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81767) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 16:03:37.557184 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8afaaa1-9ebe-4ec6-8c18-2f5636c030db service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 16:03:37.559094 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8afaaa1-9ebe-4ec6-8c18-2f5636c030db service nova] RBAC: Authorization granted {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 16:03:37.586353 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 682/1385] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Aug 22 16:03:36 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 609 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 16:03:39.676811 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df5dfc4e-b2b9-4ec2-a173-5520600fb0f8 None None] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:03:39.677473 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df5dfc4e-b2b9-4ec2-a173-5520600fb0f8 None None] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:03:39.677473 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df5dfc4e-b2b9-4ec2-a173-5520600fb0f8 None None] PATH_INFO: `/` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:03:39.678897 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 704/1386] 10.222.0.15 () {54 vars in 927 bytes} [Mon Aug 22 16:03:39 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 16:03:39.685056 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e940594-3bbb-4037-be98-e7c3bb26ac90 None None] REQUEST_METHOD: `POST` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:03:39.685611 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e940594-3bbb-4037-be98-e7c3bb26ac90 None None] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:03:39.685843 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e940594-3bbb-4037-be98-e7c3bb26ac90 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:03:39.773641 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.auth.core [None req-7e940594-3bbb-4037-be98-e7c3bb26ac90 None None] MFA Rules not processed for user `6b2199ae0fb0480aa917ada391f5ec54`. Rule list: `[]` (Enabled: `True`). {{(pid=81767) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 16:03:39.774936 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.fernet_utils [None req-7e940594-3bbb-4037-be98-e7c3bb26ac90 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81767) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 16:03:39.812708 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG dbcounter [-] [81767] Writing DB stats keystone:SELECT=100 {{(pid=81767) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 16:03:39.883872 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 683/1387] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Aug 22 16:03:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 200 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 16:03:40.182246 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df5dfc4e-b2b9-4ec2-a173-5520600fb0f8 None None] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 16:03:40.183813 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-df5dfc4e-b2b9-4ec2-a173-5520600fb0f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 16:03:40.271590 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=100 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 16:03:40.361191 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df5dfc4e-b2b9-4ec2-a173-5520600fb0f8 None None] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 16:03:40.520772 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d45dbb6a-9b25-4376-a3ef-eb1eea0987ff service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4ef569e41fb46d0b721bb1c7663616f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4ee0413f9ef4d4fb1f95a660babe090', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 16:03:40.521309 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d45dbb6a-9b25-4376-a3ef-eb1eea0987ff service nova] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:03:40.521624 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d45dbb6a-9b25-4376-a3ef-eb1eea0987ff service nova] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:03:40.522020 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d45dbb6a-9b25-4376-a3ef-eb1eea0987ff service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:03:40.524562 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-d45dbb6a-9b25-4376-a3ef-eb1eea0987ff service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 16:03:40.587229 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=100 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 16:03:40.673690 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d45dbb6a-9b25-4376-a3ef-eb1eea0987ff service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 16:03:40.676630 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d45dbb6a-9b25-4376-a3ef-eb1eea0987ff service nova] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 16:03:40.702047 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 705/1388] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Aug 22 16:03:40 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 520 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 16:03:42.171422 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e103a100-09fc-4b9e-8119-84a9d046b7a9 None None] REQUEST_METHOD: `GET` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:03:42.175742 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e103a100-09fc-4b9e-8119-84a9d046b7a9 None None] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:03:42.175742 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e103a100-09fc-4b9e-8119-84a9d046b7a9 None None] PATH_INFO: `/` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:03:42.175742 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 684/1389] 10.222.0.15 () {54 vars in 927 bytes} [Mon Aug 22 16:03:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 22 16:03:42.180251 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-557b2010-f8ee-4533-a8ff-0f064c1ab66c None None] REQUEST_METHOD: `POST` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:03:42.181106 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-557b2010-f8ee-4533-a8ff-0f064c1ab66c None None] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:03:42.181106 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-557b2010-f8ee-4533-a8ff-0f064c1ab66c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:03:42.287867 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.auth.core [None req-557b2010-f8ee-4533-a8ff-0f064c1ab66c None None] MFA Rules not processed for user `6b2199ae0fb0480aa917ada391f5ec54`. Rule list: `[]` (Enabled: `True`). {{(pid=81768) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 16:03:42.289953 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-557b2010-f8ee-4533-a8ff-0f064c1ab66c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 16:03:42.445478 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 706/1390] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Aug 22 16:03:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 266 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 22 16:03:42.778836 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6cfa3a1-82a2-41c0-a678-0f6ab4aa31b0 None None] REQUEST_METHOD: `GET` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:03:42.779369 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6cfa3a1-82a2-41c0-a678-0f6ab4aa31b0 None None] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:03:42.780131 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6cfa3a1-82a2-41c0-a678-0f6ab4aa31b0 None None] PATH_INFO: `/` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:03:42.783045 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 685/1391] 10.222.0.15 () {54 vars in 967 bytes} [Mon Aug 22 16:03:42 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 16:03:42.792146 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd4eaa61-dbe2-4e2b-a899-c3b83da4051d None None] REQUEST_METHOD: `POST` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:03:42.792705 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd4eaa61-dbe2-4e2b-a899-c3b83da4051d None None] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:03:42.793157 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd4eaa61-dbe2-4e2b-a899-c3b83da4051d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:03:42.823058 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=100 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 16:03:42.917665 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.auth.core [None req-fd4eaa61-dbe2-4e2b-a899-c3b83da4051d None None] MFA Rules not processed for user `91f565473c824a15930e2b12863c1ed3`. Rule list: `[]` (Enabled: `True`). {{(pid=81768) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 16:03:42.918996 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-fd4eaa61-dbe2-4e2b-a899-c3b83da4051d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 16:03:42.985580 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f59776-028e-4ddc-bf11-1b807d76fa53 None None] REQUEST_METHOD: `GET` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:03:42.985849 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f59776-028e-4ddc-bf11-1b807d76fa53 None None] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:03:42.986041 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f59776-028e-4ddc-bf11-1b807d76fa53 None None] PATH_INFO: `/` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:03:42.987696 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 686/1392] 10.222.0.15 () {54 vars in 927 bytes} [Mon Aug 22 16:03:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 22 16:03:42.993425 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b94f4473-1bde-40a5-ac7f-8fd325dc53fa None None] REQUEST_METHOD: `POST` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:03:42.993744 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b94f4473-1bde-40a5-ac7f-8fd325dc53fa None None] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:03:42.993962 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b94f4473-1bde-40a5-ac7f-8fd325dc53fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:03:43.014708 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 707/1393] 10.222.0.15 () {58 vars in 1064 bytes} [Mon Aug 22 16:03:42 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 225 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 16:03:43.023221 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd4eaa61-dbe2-4e2b-a899-c3b83da4051d None None] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 16:03:43.031532 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd4eaa61-dbe2-4e2b-a899-c3b83da4051d None None] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 16:03:43.071814 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.auth.core [None req-b94f4473-1bde-40a5-ac7f-8fd325dc53fa None None] MFA Rules not processed for user `6b2199ae0fb0480aa917ada391f5ec54`. Rule list: `[]` (Enabled: `True`). {{(pid=81767) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 16:03:43.073071 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.fernet_utils [None req-b94f4473-1bde-40a5-ac7f-8fd325dc53fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81767) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 16:03:43.132983 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de323298-e9a3-47d6-aaac-03f94f478090 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91f565473c824a15930e2b12863c1ed3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4ee0413f9ef4d4fb1f95a660babe090', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 16:03:43.133592 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de323298-e9a3-47d6-aaac-03f94f478090 service glance] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:03:43.133867 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de323298-e9a3-47d6-aaac-03f94f478090 service glance] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:03:43.134244 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de323298-e9a3-47d6-aaac-03f94f478090 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:03:43.136410 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-de323298-e9a3-47d6-aaac-03f94f478090 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 16:03:43.185000 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=100 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 16:03:43.196487 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 687/1394] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Aug 22 16:03:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 204 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 22 16:03:43.325047 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de323298-e9a3-47d6-aaac-03f94f478090 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 16:03:43.327898 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de323298-e9a3-47d6-aaac-03f94f478090 service glance] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 16:03:43.367036 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 708/1395] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Aug 22 16:03:43 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 344 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 22 16:03:43.508861 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b94f4473-1bde-40a5-ac7f-8fd325dc53fa None None] Authenticating user token {{(pid=81767) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 16:03:43.510496 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.fernet_utils [None req-b94f4473-1bde-40a5-ac7f-8fd325dc53fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81767) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 16:03:43.526784 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG dbcounter [-] [81767] Writing DB stats keystone:SELECT=100 {{(pid=81767) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 16:03:43.675414 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b94f4473-1bde-40a5-ac7f-8fd325dc53fa None None] Validating token access rules against request {{(pid=81767) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 16:03:43.837263 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-769f8c32-f27f-4110-a519-ccca27cb826d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4ef569e41fb46d0b721bb1c7663616f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4ee0413f9ef4d4fb1f95a660babe090', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81767) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 16:03:43.837892 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-769f8c32-f27f-4110-a519-ccca27cb826d service nova] REQUEST_METHOD: `GET` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:03:43.838175 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-769f8c32-f27f-4110-a519-ccca27cb826d service nova] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:03:43.838449 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-769f8c32-f27f-4110-a519-ccca27cb826d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:03:43.840227 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.fernet_utils [None req-769f8c32-f27f-4110-a519-ccca27cb826d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81767) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 16:03:43.851507 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG dbcounter [-] [81767] Writing DB stats keystone:SELECT=100 {{(pid=81767) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 16:03:43.958674 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-769f8c32-f27f-4110-a519-ccca27cb826d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 16:03:43.962163 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-769f8c32-f27f-4110-a519-ccca27cb826d service nova] RBAC: Authorization granted {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 16:03:44.000440 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 688/1396] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Aug 22 16:03:43 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 492 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 16:03:44.473863 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04a16a52-2be7-4004-8296-8b0fc9a6e52f None None] REQUEST_METHOD: `POST` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:03:44.474519 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04a16a52-2be7-4004-8296-8b0fc9a6e52f None None] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:03:44.474519 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04a16a52-2be7-4004-8296-8b0fc9a6e52f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:03:44.561429 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.auth.core [None req-04a16a52-2be7-4004-8296-8b0fc9a6e52f None None] MFA Rules not processed for user `058e76354dce412cb78b2875c88f2791`. Rule list: `[]` (Enabled: `True`). {{(pid=81768) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 16:03:44.564196 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-04a16a52-2be7-4004-8296-8b0fc9a6e52f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 16:03:44.671829 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=100 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 16:03:44.680688 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 709/1397] 10.222.0.15 () {58 vars in 1009 bytes} [Mon Aug 22 16:03:44 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 209 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 16:03:44.733742 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4310a972-1ab5-4cf1-a254-941d44e2989d None None] REQUEST_METHOD: `GET` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:03:44.734281 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4310a972-1ab5-4cf1-a254-941d44e2989d None None] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:03:44.734496 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4310a972-1ab5-4cf1-a254-941d44e2989d None None] PATH_INFO: `/` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:03:44.735997 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 689/1398] 10.222.0.15 () {54 vars in 943 bytes} [Mon Aug 22 16:03:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 16:03:44.743763 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d07b809-ee29-46cf-a3ae-93204611381e None None] REQUEST_METHOD: `POST` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:03:44.744040 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d07b809-ee29-46cf-a3ae-93204611381e None None] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:03:44.744309 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d07b809-ee29-46cf-a3ae-93204611381e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:03:44.855713 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.auth.core [None req-9d07b809-ee29-46cf-a3ae-93204611381e None None] MFA Rules not processed for user `6a09d128dee9438283633e51f4834643`. Rule list: `[]` (Enabled: `True`). {{(pid=81768) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 16:03:44.856941 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-9d07b809-ee29-46cf-a3ae-93204611381e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 16:03:45.010162 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 710/1399] 10.222.0.15 () {58 vars in 1040 bytes} [Mon Aug 22 16:03:44 2022] POST /identity/v3/auth/tokens => generated 3304 bytes in 267 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 22 16:03:45.020489 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4310a972-1ab5-4cf1-a254-941d44e2989d None None] Authenticating user token {{(pid=81767) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 16:03:45.022092 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.fernet_utils [None req-4310a972-1ab5-4cf1-a254-941d44e2989d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81767) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 16:03:45.226864 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG dbcounter [-] [81767] Writing DB stats keystone:SELECT=100 {{(pid=81767) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 16:03:45.229965 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4310a972-1ab5-4cf1-a254-941d44e2989d None None] Validating token access rules against request {{(pid=81767) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 16:03:45.507395 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0d7ec47-1dab-4677-85cb-5039ce59df00 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a09d128dee9438283633e51f4834643', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4ee0413f9ef4d4fb1f95a660babe090', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81767) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 16:03:45.508037 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d7ec47-1dab-4677-85cb-5039ce59df00 service swift] REQUEST_METHOD: `GET` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:03:45.508280 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d7ec47-1dab-4677-85cb-5039ce59df00 service swift] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:03:45.508555 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d7ec47-1dab-4677-85cb-5039ce59df00 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:03:45.510240 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.fernet_utils [None req-d0d7ec47-1dab-4677-85cb-5039ce59df00 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81767) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 16:03:45.615473 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0d7ec47-1dab-4677-85cb-5039ce59df00 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 16:03:45.618520 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0d7ec47-1dab-4677-85cb-5039ce59df00 service swift] RBAC: Authorization granted {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 16:03:45.635430 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 690/1400] 10.222.0.15 () {60 vars in 1384 bytes} [Mon Aug 22 16:03:45 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 615 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Aug 22 16:03:46.240946 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d853d88-793e-4073-bcc5-762b7119e59f None None] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:03:46.241696 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d853d88-793e-4073-bcc5-762b7119e59f None None] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:03:46.241696 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d853d88-793e-4073-bcc5-762b7119e59f None None] PATH_INFO: `/` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:03:46.243086 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 711/1401] 10.222.0.15 () {54 vars in 927 bytes} [Mon Aug 22 16:03:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 16:03:46.249956 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d123ebf6-5798-4efe-baec-021daf26f1b3 None None] REQUEST_METHOD: `POST` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:03:46.249956 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d123ebf6-5798-4efe-baec-021daf26f1b3 None None] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:03:46.249956 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d123ebf6-5798-4efe-baec-021daf26f1b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:03:46.305675 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG dbcounter [-] [81767] Writing DB stats keystone:SELECT=100 {{(pid=81767) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 16:03:46.353135 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.auth.core [None req-d123ebf6-5798-4efe-baec-021daf26f1b3 None None] MFA Rules not processed for user `6b2199ae0fb0480aa917ada391f5ec54`. Rule list: `[]` (Enabled: `True`). {{(pid=81767) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 16:03:46.359955 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.fernet_utils [None req-d123ebf6-5798-4efe-baec-021daf26f1b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81767) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 16:03:46.551538 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 691/1402] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Aug 22 16:03:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 304 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 22 16:03:46.870894 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d853d88-793e-4073-bcc5-762b7119e59f None None] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 16:03:46.872871 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-9d853d88-793e-4073-bcc5-762b7119e59f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 16:03:46.971917 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=100 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 16:03:47.144007 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d853d88-793e-4073-bcc5-762b7119e59f None None] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 16:03:47.318306 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf8aa8cc-6250-493a-887e-f62762504913 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4ef569e41fb46d0b721bb1c7663616f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4ee0413f9ef4d4fb1f95a660babe090', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 16:03:47.319423 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf8aa8cc-6250-493a-887e-f62762504913 service nova] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:03:47.319423 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf8aa8cc-6250-493a-887e-f62762504913 service nova] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:03:47.319719 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf8aa8cc-6250-493a-887e-f62762504913 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:03:47.323909 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-bf8aa8cc-6250-493a-887e-f62762504913 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 16:03:47.340903 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=100 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 16:03:47.434124 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf8aa8cc-6250-493a-887e-f62762504913 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 16:03:47.436129 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf8aa8cc-6250-493a-887e-f62762504913 service nova] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 16:03:47.459913 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 712/1403] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Aug 22 16:03:46 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 589 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 16:03:50.858989 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-086d98c2-2902-48a6-99d3-d70cb1d786f6 None None] REQUEST_METHOD: `GET` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:03:50.859599 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-086d98c2-2902-48a6-99d3-d70cb1d786f6 None None] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:03:50.859599 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-086d98c2-2902-48a6-99d3-d70cb1d786f6 None None] PATH_INFO: `/` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:03:50.861097 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 692/1404] 10.222.0.15 () {54 vars in 927 bytes} [Mon Aug 22 16:03:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 16:03:50.867552 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-089ed392-8909-4982-9f01-63d6d1519016 None None] REQUEST_METHOD: `POST` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:03:50.868160 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-089ed392-8909-4982-9f01-63d6d1519016 None None] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:03:50.868541 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-089ed392-8909-4982-9f01-63d6d1519016 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:03:50.966840 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.auth.core [None req-089ed392-8909-4982-9f01-63d6d1519016 None None] MFA Rules not processed for user `6b2199ae0fb0480aa917ada391f5ec54`. Rule list: `[]` (Enabled: `True`). {{(pid=81768) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 16:03:50.968798 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-089ed392-8909-4982-9f01-63d6d1519016 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 16:03:51.126793 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=100 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 16:03:51.135576 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 713/1405] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Aug 22 16:03:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 270 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 16:03:51.475397 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-086d98c2-2902-48a6-99d3-d70cb1d786f6 None None] Authenticating user token {{(pid=81767) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 16:03:51.477268 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.fernet_utils [None req-086d98c2-2902-48a6-99d3-d70cb1d786f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81767) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 16:03:51.653459 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG dbcounter [-] [81767] Writing DB stats keystone:SELECT=100 {{(pid=81767) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 16:03:51.757817 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-086d98c2-2902-48a6-99d3-d70cb1d786f6 None None] Validating token access rules against request {{(pid=81767) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 16:03:51.906767 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f665da50-df47-4fb6-b469-738f78ac8735 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4ef569e41fb46d0b721bb1c7663616f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4ee0413f9ef4d4fb1f95a660babe090', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81767) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 16:03:51.907530 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f665da50-df47-4fb6-b469-738f78ac8735 service nova] REQUEST_METHOD: `GET` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:03:51.907702 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f665da50-df47-4fb6-b469-738f78ac8735 service nova] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:03:51.911652 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f665da50-df47-4fb6-b469-738f78ac8735 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:03:51.911652 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.fernet_utils [None req-f665da50-df47-4fb6-b469-738f78ac8735 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81767) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 16:03:52.020576 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG dbcounter [-] [81767] Writing DB stats keystone:SELECT=100 {{(pid=81767) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 16:03:52.040526 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f665da50-df47-4fb6-b469-738f78ac8735 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 16:03:52.042333 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f665da50-df47-4fb6-b469-738f78ac8735 service nova] RBAC: Authorization granted {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 16:03:52.078181 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 693/1406] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Aug 22 16:03:51 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 603 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 22 16:03:54.367205 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-073e0bbc-d0b7-40c6-bc01-b93178cebab2 None None] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:03:54.368165 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-073e0bbc-d0b7-40c6-bc01-b93178cebab2 None None] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:03:54.368800 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-073e0bbc-d0b7-40c6-bc01-b93178cebab2 None None] PATH_INFO: `/` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:03:54.371483 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 714/1407] 10.222.0.15 () {54 vars in 927 bytes} [Mon Aug 22 16:03:54 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 16:03:54.377979 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d186df29-c8b4-4083-a3ab-81c4f87217d9 None None] REQUEST_METHOD: `POST` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:03:54.379058 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d186df29-c8b4-4083-a3ab-81c4f87217d9 None None] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:03:54.379058 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d186df29-c8b4-4083-a3ab-81c4f87217d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:03:54.473656 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.auth.core [None req-d186df29-c8b4-4083-a3ab-81c4f87217d9 None None] MFA Rules not processed for user `6b2199ae0fb0480aa917ada391f5ec54`. Rule list: `[]` (Enabled: `True`). {{(pid=81767) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 16:03:54.475159 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.fernet_utils [None req-d186df29-c8b4-4083-a3ab-81c4f87217d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81767) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 16:03:54.619051 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 694/1408] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Aug 22 16:03:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 243 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 16:03:54.919356 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-073e0bbc-d0b7-40c6-bc01-b93178cebab2 None None] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 16:03:54.921669 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-073e0bbc-d0b7-40c6-bc01-b93178cebab2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 16:03:55.170911 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-073e0bbc-d0b7-40c6-bc01-b93178cebab2 None None] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 16:03:55.297591 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=100 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 16:03:55.381679 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ef2e9e0-6395-4aec-a69d-360dcff490b1 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4ef569e41fb46d0b721bb1c7663616f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4ee0413f9ef4d4fb1f95a660babe090', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 16:03:55.382644 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ef2e9e0-6395-4aec-a69d-360dcff490b1 service nova] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:03:55.382935 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ef2e9e0-6395-4aec-a69d-360dcff490b1 service nova] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:03:55.383212 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ef2e9e0-6395-4aec-a69d-360dcff490b1 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:03:55.385365 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-0ef2e9e0-6395-4aec-a69d-360dcff490b1 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 16:03:55.522351 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ef2e9e0-6395-4aec-a69d-360dcff490b1 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 16:03:55.524336 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ef2e9e0-6395-4aec-a69d-360dcff490b1 service nova] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 16:03:55.546880 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 715/1409] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Aug 22 16:03:54 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 628 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 22 16:04:04.616730 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG dbcounter [-] [81767] Writing DB stats keystone:SELECT=78 {{(pid=81767) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 16:04:05.543754 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=65 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 16:04:10.687236 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49e5f3b2-bb75-49bc-8aa6-c15ce5d2f018 None None] REQUEST_METHOD: `GET` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:04:10.687812 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49e5f3b2-bb75-49bc-8aa6-c15ce5d2f018 None None] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:04:10.687812 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49e5f3b2-bb75-49bc-8aa6-c15ce5d2f018 None None] PATH_INFO: `/` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:04:10.689686 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 695/1410] 10.222.0.15 () {54 vars in 918 bytes} [Mon Aug 22 16:04:10 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 22 16:04:10.695431 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaf72d82-e1b7-4ff6-b00f-e4139f39445e None None] REQUEST_METHOD: `POST` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:04:10.695961 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaf72d82-e1b7-4ff6-b00f-e4139f39445e None None] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:04:10.695961 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaf72d82-e1b7-4ff6-b00f-e4139f39445e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:04:10.780592 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.auth.core [None req-eaf72d82-e1b7-4ff6-b00f-e4139f39445e None None] MFA Rules not processed for user `08159bf8edda4246bb463f7b11b0894b`. Rule list: `[]` (Enabled: `True`). {{(pid=81768) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 16:04:10.781828 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-eaf72d82-e1b7-4ff6-b00f-e4139f39445e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 16:04:10.903623 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 716/1411] 10.222.0.15 () {58 vars in 1015 bytes} [Mon Aug 22 16:04:10 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 209 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 16:04:10.918896 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbde0135-d4cd-4814-9c87-2c296a811b63 None None] REQUEST_METHOD: `GET` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:04:10.919178 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbde0135-d4cd-4814-9c87-2c296a811b63 None None] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:04:10.919475 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbde0135-d4cd-4814-9c87-2c296a811b63 None None] PATH_INFO: `/` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:04:10.920803 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 696/1412] 10.222.0.15 () {54 vars in 918 bytes} [Mon Aug 22 16:04:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 16:04:10.926101 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97122a44-03f0-4b42-8c89-f3146931e299 None None] REQUEST_METHOD: `POST` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:04:10.926392 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97122a44-03f0-4b42-8c89-f3146931e299 None None] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:04:10.926672 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97122a44-03f0-4b42-8c89-f3146931e299 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:04:11.007967 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=100 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 16:04:11.011199 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.auth.core [None req-97122a44-03f0-4b42-8c89-f3146931e299 None None] MFA Rules not processed for user `e4ef569e41fb46d0b721bb1c7663616f`. Rule list: `[]` (Enabled: `True`). {{(pid=81768) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 16:04:11.012477 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-97122a44-03f0-4b42-8c89-f3146931e299 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 16:04:11.139371 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 717/1413] 10.222.0.15 () {58 vars in 1015 bytes} [Mon Aug 22 16:04:10 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 214 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 16:04:13.635693 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00a28dd2-032c-4d06-8ddc-18184b74be6c None None] REQUEST_METHOD: `GET` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:04:13.636233 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00a28dd2-032c-4d06-8ddc-18184b74be6c None None] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:04:13.636233 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00a28dd2-032c-4d06-8ddc-18184b74be6c None None] PATH_INFO: `/` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:04:13.637518 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 697/1414] 127.0.0.1 () {54 vars in 917 bytes} [Mon Aug 22 16:04:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 16:04:13.649066 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba575288-e1a4-4e20-ab0b-68a22c8e7610 None None] REQUEST_METHOD: `POST` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:04:13.649671 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba575288-e1a4-4e20-ab0b-68a22c8e7610 None None] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:04:13.649831 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba575288-e1a4-4e20-ab0b-68a22c8e7610 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:04:13.728946 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.auth.core [None req-ba575288-e1a4-4e20-ab0b-68a22c8e7610 None None] MFA Rules not processed for user `6b2199ae0fb0480aa917ada391f5ec54`. Rule list: `[]` (Enabled: `True`). {{(pid=81768) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 16:04:13.730188 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-ba575288-e1a4-4e20-ab0b-68a22c8e7610 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 16:04:13.791006 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=100 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 16:04:13.835736 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 718/1415] 127.0.0.1 () {58 vars in 1014 bytes} [Mon Aug 22 16:04:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 16:04:14.132997 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00a28dd2-032c-4d06-8ddc-18184b74be6c None None] Authenticating user token {{(pid=81767) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 16:04:14.134537 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.fernet_utils [None req-00a28dd2-032c-4d06-8ddc-18184b74be6c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81767) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 16:04:14.295368 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00a28dd2-032c-4d06-8ddc-18184b74be6c None None] Validating token access rules against request {{(pid=81767) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 16:04:14.390341 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG dbcounter [-] [81767] Writing DB stats keystone:SELECT=100 {{(pid=81767) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 16:04:14.438071 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c529ff54-8c99-4bc8-8231-392daf779e9f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4ef569e41fb46d0b721bb1c7663616f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4ee0413f9ef4d4fb1f95a660babe090', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81767) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 16:04:14.438687 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c529ff54-8c99-4bc8-8231-392daf779e9f service nova] REQUEST_METHOD: `GET` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:04:14.439014 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c529ff54-8c99-4bc8-8231-392daf779e9f service nova] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:04:14.439378 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c529ff54-8c99-4bc8-8231-392daf779e9f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:04:14.440972 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.fernet_utils [None req-c529ff54-8c99-4bc8-8231-392daf779e9f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81767) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 16:04:14.560064 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c529ff54-8c99-4bc8-8231-392daf779e9f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 16:04:14.561675 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c529ff54-8c99-4bc8-8231-392daf779e9f service nova] RBAC: Authorization granted {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 16:04:14.581977 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 698/1416] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Aug 22 16:04:14 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 449 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 16:04:15.515860 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b398a682-6719-483c-aca7-7678b1d7ee0b None None] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:04:15.516415 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b398a682-6719-483c-aca7-7678b1d7ee0b None None] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:04:15.516415 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b398a682-6719-483c-aca7-7678b1d7ee0b None None] PATH_INFO: `/` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:04:15.517824 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 719/1417] 127.0.0.1 () {54 vars in 901 bytes} [Mon Aug 22 16:04:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 16:04:15.523575 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e0e8b44-e7db-4e8b-8dac-c7f15b68e98f None None] REQUEST_METHOD: `POST` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:04:15.524223 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e0e8b44-e7db-4e8b-8dac-c7f15b68e98f None None] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:04:15.524223 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e0e8b44-e7db-4e8b-8dac-c7f15b68e98f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:04:15.607597 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.auth.core [None req-7e0e8b44-e7db-4e8b-8dac-c7f15b68e98f None None] MFA Rules not processed for user `6b2199ae0fb0480aa917ada391f5ec54`. Rule list: `[]` (Enabled: `True`). {{(pid=81767) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 16:04:15.608733 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.fernet_utils [None req-7e0e8b44-e7db-4e8b-8dac-c7f15b68e98f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81767) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 16:04:15.650759 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG dbcounter [-] [81767] Writing DB stats keystone:SELECT=100 {{(pid=81767) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 16:04:15.729263 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 699/1418] 127.0.0.1 () {58 vars in 998 bytes} [Mon Aug 22 16:04:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 207 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 16:04:15.884509 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b398a682-6719-483c-aca7-7678b1d7ee0b None None] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 16:04:15.886475 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-b398a682-6719-483c-aca7-7678b1d7ee0b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 16:04:16.043206 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b398a682-6719-483c-aca7-7678b1d7ee0b None None] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 16:04:16.088811 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=100 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 16:04:16.167991 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb9dcd62-ac69-48b5-bd69-5eed966714bf service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4ef569e41fb46d0b721bb1c7663616f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4ee0413f9ef4d4fb1f95a660babe090', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 16:04:16.168534 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb9dcd62-ac69-48b5-bd69-5eed966714bf service nova] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:04:16.168810 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb9dcd62-ac69-48b5-bd69-5eed966714bf service nova] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:04:16.169075 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb9dcd62-ac69-48b5-bd69-5eed966714bf service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:04:16.170713 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-bb9dcd62-ac69-48b5-bd69-5eed966714bf service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 16:04:16.274922 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb9dcd62-ac69-48b5-bd69-5eed966714bf service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 16:04:16.276589 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb9dcd62-ac69-48b5-bd69-5eed966714bf service nova] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 16:04:16.297584 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 720/1419] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Aug 22 16:04:15 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 414 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 16:04:17.286197 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51156aa3-7117-45d1-9759-f803a53273fe None None] REQUEST_METHOD: `GET` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:04:17.286723 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51156aa3-7117-45d1-9759-f803a53273fe None None] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:04:17.286723 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51156aa3-7117-45d1-9759-f803a53273fe None None] PATH_INFO: `/` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:04:17.288172 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 700/1420] 127.0.0.1 () {54 vars in 901 bytes} [Mon Aug 22 16:04:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 16:04:17.293487 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8b82a36-b0ff-4cd0-8b63-2d4d647167ac None None] REQUEST_METHOD: `POST` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:04:17.294144 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8b82a36-b0ff-4cd0-8b63-2d4d647167ac None None] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:04:17.294144 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8b82a36-b0ff-4cd0-8b63-2d4d647167ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:04:17.354054 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=100 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 16:04:17.377508 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.auth.core [None req-d8b82a36-b0ff-4cd0-8b63-2d4d647167ac None None] MFA Rules not processed for user `6b2199ae0fb0480aa917ada391f5ec54`. Rule list: `[]` (Enabled: `True`). {{(pid=81768) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 16:04:17.378597 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-d8b82a36-b0ff-4cd0-8b63-2d4d647167ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 16:04:17.476422 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 721/1421] 127.0.0.1 () {58 vars in 998 bytes} [Mon Aug 22 16:04:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 16:04:17.633823 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51156aa3-7117-45d1-9759-f803a53273fe None None] Authenticating user token {{(pid=81767) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 16:04:17.635374 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.fernet_utils [None req-51156aa3-7117-45d1-9759-f803a53273fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81767) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 16:04:17.807700 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51156aa3-7117-45d1-9759-f803a53273fe None None] Validating token access rules against request {{(pid=81767) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 16:04:17.820701 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG dbcounter [-] [81767] Writing DB stats keystone:SELECT=100 {{(pid=81767) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 16:04:17.933921 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32795aa7-3d7e-4c01-83fc-82d3e381017f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4ef569e41fb46d0b721bb1c7663616f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4ee0413f9ef4d4fb1f95a660babe090', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81767) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 16:04:17.934446 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32795aa7-3d7e-4c01-83fc-82d3e381017f service nova] REQUEST_METHOD: `GET` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:04:17.934734 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32795aa7-3d7e-4c01-83fc-82d3e381017f service nova] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:04:17.935005 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32795aa7-3d7e-4c01-83fc-82d3e381017f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:04:17.936674 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.fernet_utils [None req-32795aa7-3d7e-4c01-83fc-82d3e381017f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81767) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 16:04:18.041296 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32795aa7-3d7e-4c01-83fc-82d3e381017f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 16:04:18.043267 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32795aa7-3d7e-4c01-83fc-82d3e381017f service nova] RBAC: Authorization granted {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 16:04:18.063000 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 701/1422] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Aug 22 16:04:17 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 430 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 16:04:18.963579 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6ca3539-ee50-4dfa-8c68-e863dae3f925 None None] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:04:18.964125 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6ca3539-ee50-4dfa-8c68-e863dae3f925 None None] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:04:18.964125 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6ca3539-ee50-4dfa-8c68-e863dae3f925 None None] PATH_INFO: `/` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:04:18.965470 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 722/1423] 127.0.0.1 () {54 vars in 901 bytes} [Mon Aug 22 16:04:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 16:04:18.971043 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-601ac557-1331-476c-bcfb-0ac0139c17d0 None None] REQUEST_METHOD: `POST` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:04:18.971617 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-601ac557-1331-476c-bcfb-0ac0139c17d0 None None] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:04:18.971617 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-601ac557-1331-476c-bcfb-0ac0139c17d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:04:18.992474 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG dbcounter [-] [81767] Writing DB stats keystone:SELECT=100 {{(pid=81767) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 16:04:19.057435 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.auth.core [None req-601ac557-1331-476c-bcfb-0ac0139c17d0 None None] MFA Rules not processed for user `6b2199ae0fb0480aa917ada391f5ec54`. Rule list: `[]` (Enabled: `True`). {{(pid=81767) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 16:04:19.058663 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.fernet_utils [None req-601ac557-1331-476c-bcfb-0ac0139c17d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81767) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 16:04:19.187170 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 702/1424] 127.0.0.1 () {58 vars in 998 bytes} [Mon Aug 22 16:04:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 217 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 16:04:19.341058 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6ca3539-ee50-4dfa-8c68-e863dae3f925 None None] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 16:04:19.342715 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-c6ca3539-ee50-4dfa-8c68-e863dae3f925 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 16:04:19.466999 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=100 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 16:04:19.519369 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6ca3539-ee50-4dfa-8c68-e863dae3f925 None None] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 16:04:19.641696 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-581e3422-61e1-4e05-9636-a59b0c404689 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4ef569e41fb46d0b721bb1c7663616f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4ee0413f9ef4d4fb1f95a660babe090', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 16:04:19.642309 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-581e3422-61e1-4e05-9636-a59b0c404689 service nova] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:04:19.642585 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-581e3422-61e1-4e05-9636-a59b0c404689 service nova] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:04:19.642855 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-581e3422-61e1-4e05-9636-a59b0c404689 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:04:19.644560 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-581e3422-61e1-4e05-9636-a59b0c404689 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 16:04:19.727529 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=100 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 16:04:19.742070 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-581e3422-61e1-4e05-9636-a59b0c404689 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 16:04:19.743759 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-581e3422-61e1-4e05-9636-a59b0c404689 service nova] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 16:04:19.763073 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 723/1425] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Aug 22 16:04:19 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 422 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 16:04:20.651520 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67fd2fd6-399d-467f-a07c-30307ec95183 None None] REQUEST_METHOD: `GET` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:04:20.652069 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67fd2fd6-399d-467f-a07c-30307ec95183 None None] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:04:20.652069 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67fd2fd6-399d-467f-a07c-30307ec95183 None None] PATH_INFO: `/` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:04:20.653326 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 703/1426] 127.0.0.1 () {54 vars in 901 bytes} [Mon Aug 22 16:04:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 16:04:20.658395 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a54e6c9-d98b-487a-b12a-e35026007767 None None] REQUEST_METHOD: `POST` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:04:20.658836 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a54e6c9-d98b-487a-b12a-e35026007767 None None] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:04:20.658942 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a54e6c9-d98b-487a-b12a-e35026007767 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:04:20.756065 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.auth.core [None req-2a54e6c9-d98b-487a-b12a-e35026007767 None None] MFA Rules not processed for user `6b2199ae0fb0480aa917ada391f5ec54`. Rule list: `[]` (Enabled: `True`). {{(pid=81768) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 16:04:20.757304 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-2a54e6c9-d98b-487a-b12a-e35026007767 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 16:04:20.867592 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 724/1427] 127.0.0.1 () {58 vars in 998 bytes} [Mon Aug 22 16:04:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 210 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 16:04:21.022147 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67fd2fd6-399d-467f-a07c-30307ec95183 None None] Authenticating user token {{(pid=81767) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 16:04:21.023714 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.fernet_utils [None req-67fd2fd6-399d-467f-a07c-30307ec95183 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81767) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 16:04:21.114164 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG dbcounter [-] [81767] Writing DB stats keystone:SELECT=100 {{(pid=81767) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 16:04:21.177181 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67fd2fd6-399d-467f-a07c-30307ec95183 None None] Validating token access rules against request {{(pid=81767) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 16:04:21.313284 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0530f54-24a9-4281-9099-8b78cb40635b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4ef569e41fb46d0b721bb1c7663616f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4ee0413f9ef4d4fb1f95a660babe090', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81767) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 16:04:21.313981 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0530f54-24a9-4281-9099-8b78cb40635b service nova] REQUEST_METHOD: `GET` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:04:21.314258 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0530f54-24a9-4281-9099-8b78cb40635b service nova] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:04:21.314528 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0530f54-24a9-4281-9099-8b78cb40635b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:04:21.316212 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.fernet_utils [None req-e0530f54-24a9-4281-9099-8b78cb40635b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81767) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 16:04:21.377198 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG dbcounter [-] [81767] Writing DB stats keystone:SELECT=100 {{(pid=81767) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 16:04:21.423197 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0530f54-24a9-4281-9099-8b78cb40635b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 16:04:21.424880 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0530f54-24a9-4281-9099-8b78cb40635b service nova] RBAC: Authorization granted {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 16:04:21.445348 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 704/1428] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Aug 22 16:04:21 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 424 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 16:04:22.340634 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74b688d2-526e-4837-8419-5c6c73e8eb67 None None] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:04:22.340634 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74b688d2-526e-4837-8419-5c6c73e8eb67 None None] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:04:22.341127 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74b688d2-526e-4837-8419-5c6c73e8eb67 None None] PATH_INFO: `/` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:04:22.342182 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 725/1429] 127.0.0.1 () {54 vars in 901 bytes} [Mon Aug 22 16:04:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 22 16:04:22.348435 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-432e4591-1151-45a2-a6ac-087b5f8edca3 None None] REQUEST_METHOD: `POST` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:04:22.348962 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-432e4591-1151-45a2-a6ac-087b5f8edca3 None None] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:04:22.348962 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-432e4591-1151-45a2-a6ac-087b5f8edca3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:04:22.448159 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.auth.core [None req-432e4591-1151-45a2-a6ac-087b5f8edca3 None None] MFA Rules not processed for user `6b2199ae0fb0480aa917ada391f5ec54`. Rule list: `[]` (Enabled: `True`). {{(pid=81767) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 16:04:22.450232 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.fernet_utils [None req-432e4591-1151-45a2-a6ac-087b5f8edca3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81767) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 16:04:22.568079 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 705/1430] 127.0.0.1 () {58 vars in 998 bytes} [Mon Aug 22 16:04:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 221 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 16:04:22.726042 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74b688d2-526e-4837-8419-5c6c73e8eb67 None None] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 16:04:22.727583 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-74b688d2-526e-4837-8419-5c6c73e8eb67 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 16:04:22.781037 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=100 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 16:04:22.881999 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74b688d2-526e-4837-8419-5c6c73e8eb67 None None] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 16:04:23.006114 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da6b9684-aaf9-4ef5-b644-709455c90656 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4ef569e41fb46d0b721bb1c7663616f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4ee0413f9ef4d4fb1f95a660babe090', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 16:04:23.006639 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da6b9684-aaf9-4ef5-b644-709455c90656 service nova] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:04:23.006924 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da6b9684-aaf9-4ef5-b644-709455c90656 service nova] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:04:23.007198 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da6b9684-aaf9-4ef5-b644-709455c90656 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:04:23.008932 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-da6b9684-aaf9-4ef5-b644-709455c90656 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 16:04:23.017868 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=100 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 16:04:23.111685 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da6b9684-aaf9-4ef5-b644-709455c90656 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 16:04:23.113334 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da6b9684-aaf9-4ef5-b644-709455c90656 service nova] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 16:04:23.133527 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 726/1431] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Aug 22 16:04:22 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 408 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 16:04:24.024280 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68e9d1cd-236c-42f4-bae6-8911cc5c1711 None None] REQUEST_METHOD: `GET` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:04:24.024843 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68e9d1cd-236c-42f4-bae6-8911cc5c1711 None None] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:04:24.024843 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68e9d1cd-236c-42f4-bae6-8911cc5c1711 None None] PATH_INFO: `/` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:04:24.026392 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 706/1432] 127.0.0.1 () {54 vars in 901 bytes} [Mon Aug 22 16:04:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 16:04:24.032106 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7d1be83-3693-4542-90bc-e56e78803770 None None] REQUEST_METHOD: `POST` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:04:24.033353 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7d1be83-3693-4542-90bc-e56e78803770 None None] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:04:24.033353 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7d1be83-3693-4542-90bc-e56e78803770 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:04:24.133159 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.auth.core [None req-c7d1be83-3693-4542-90bc-e56e78803770 None None] MFA Rules not processed for user `6b2199ae0fb0480aa917ada391f5ec54`. Rule list: `[]` (Enabled: `True`). {{(pid=81768) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 16:04:24.134537 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-c7d1be83-3693-4542-90bc-e56e78803770 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 16:04:24.222367 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=100 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 16:04:24.255695 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 727/1433] 127.0.0.1 () {58 vars in 998 bytes} [Mon Aug 22 16:04:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 224 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 16:04:24.410676 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68e9d1cd-236c-42f4-bae6-8911cc5c1711 None None] Authenticating user token {{(pid=81767) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 16:04:24.412239 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.fernet_utils [None req-68e9d1cd-236c-42f4-bae6-8911cc5c1711 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81767) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 16:04:24.431160 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG dbcounter [-] [81767] Writing DB stats keystone:SELECT=100 {{(pid=81767) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 16:04:24.581757 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68e9d1cd-236c-42f4-bae6-8911cc5c1711 None None] Validating token access rules against request {{(pid=81767) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 16:04:24.683391 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG dbcounter [-] [81767] Writing DB stats keystone:SELECT=100 {{(pid=81767) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 16:04:24.712745 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b5e6f2b-3ac2-4102-8f21-03723f2b2793 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4ef569e41fb46d0b721bb1c7663616f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4ee0413f9ef4d4fb1f95a660babe090', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81767) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 16:04:24.713364 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b5e6f2b-3ac2-4102-8f21-03723f2b2793 service nova] REQUEST_METHOD: `GET` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:04:24.713640 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b5e6f2b-3ac2-4102-8f21-03723f2b2793 service nova] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:04:24.713910 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b5e6f2b-3ac2-4102-8f21-03723f2b2793 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:04:24.715579 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.fernet_utils [None req-0b5e6f2b-3ac2-4102-8f21-03723f2b2793 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81767) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 16:04:24.824928 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b5e6f2b-3ac2-4102-8f21-03723f2b2793 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 16:04:24.826755 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b5e6f2b-3ac2-4102-8f21-03723f2b2793 service nova] RBAC: Authorization granted {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 16:04:24.847976 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 707/1434] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Aug 22 16:04:24 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 438 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 22 16:04:25.751028 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd4ded1-3d1c-477b-a264-dff4dd22042e None None] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:04:25.751671 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd4ded1-3d1c-477b-a264-dff4dd22042e None None] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:04:25.751671 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd4ded1-3d1c-477b-a264-dff4dd22042e None None] PATH_INFO: `/` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:04:25.753050 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 728/1435] 127.0.0.1 () {54 vars in 901 bytes} [Mon Aug 22 16:04:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 16:04:25.758755 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88710553-10a8-4940-8a9c-5dff49e2c1b3 None None] REQUEST_METHOD: `POST` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:04:25.759254 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88710553-10a8-4940-8a9c-5dff49e2c1b3 None None] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:04:25.759404 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88710553-10a8-4940-8a9c-5dff49e2c1b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:04:25.856431 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.auth.core [None req-88710553-10a8-4940-8a9c-5dff49e2c1b3 None None] MFA Rules not processed for user `6b2199ae0fb0480aa917ada391f5ec54`. Rule list: `[]` (Enabled: `True`). {{(pid=81767) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 16:04:25.857629 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.fernet_utils [None req-88710553-10a8-4940-8a9c-5dff49e2c1b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81767) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 16:04:25.903922 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG dbcounter [-] [81767] Writing DB stats keystone:SELECT=100 {{(pid=81767) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 16:04:25.967752 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 708/1436] 127.0.0.1 () {58 vars in 998 bytes} [Mon Aug 22 16:04:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 210 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 16:04:26.135704 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fd4ded1-3d1c-477b-a264-dff4dd22042e None None] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 16:04:26.138102 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-4fd4ded1-3d1c-477b-a264-dff4dd22042e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 16:04:26.312044 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fd4ded1-3d1c-477b-a264-dff4dd22042e None None] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 16:04:26.372531 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=100 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 16:04:26.433812 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0aec57f-6c3a-4929-bcd4-2533afc52fc3 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4ef569e41fb46d0b721bb1c7663616f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4ee0413f9ef4d4fb1f95a660babe090', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 16:04:26.434650 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0aec57f-6c3a-4929-bcd4-2533afc52fc3 service nova] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:04:26.435475 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0aec57f-6c3a-4929-bcd4-2533afc52fc3 service nova] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:04:26.435767 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0aec57f-6c3a-4929-bcd4-2533afc52fc3 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:04:26.437794 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-c0aec57f-6c3a-4929-bcd4-2533afc52fc3 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 16:04:26.538803 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0aec57f-6c3a-4929-bcd4-2533afc52fc3 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 16:04:26.540523 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0aec57f-6c3a-4929-bcd4-2533afc52fc3 service nova] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 16:04:26.560265 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 729/1437] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Aug 22 16:04:26 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 425 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 16:04:27.451682 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f10c5d20-4ceb-4adc-af15-950f54952e56 None None] REQUEST_METHOD: `GET` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:04:27.452274 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f10c5d20-4ceb-4adc-af15-950f54952e56 None None] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:04:27.452274 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f10c5d20-4ceb-4adc-af15-950f54952e56 None None] PATH_INFO: `/` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:04:27.453738 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 709/1438] 127.0.0.1 () {54 vars in 901 bytes} [Mon Aug 22 16:04:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 16:04:27.460083 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7054c73-1c2d-4490-ae23-6c418812280e None None] REQUEST_METHOD: `POST` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:04:27.460919 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7054c73-1c2d-4490-ae23-6c418812280e None None] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:04:27.461094 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7054c73-1c2d-4490-ae23-6c418812280e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:04:27.589229 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.auth.core [None req-c7054c73-1c2d-4490-ae23-6c418812280e None None] MFA Rules not processed for user `6b2199ae0fb0480aa917ada391f5ec54`. Rule list: `[]` (Enabled: `True`). {{(pid=81768) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 16:04:27.590389 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-c7054c73-1c2d-4490-ae23-6c418812280e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 16:04:27.592799 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=100 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 16:04:27.701656 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 730/1439] 127.0.0.1 () {58 vars in 998 bytes} [Mon Aug 22 16:04:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 243 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 16:04:27.862294 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f10c5d20-4ceb-4adc-af15-950f54952e56 None None] Authenticating user token {{(pid=81767) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 16:04:27.865401 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.fernet_utils [None req-f10c5d20-4ceb-4adc-af15-950f54952e56 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81767) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 16:04:28.024159 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f10c5d20-4ceb-4adc-af15-950f54952e56 None None] Validating token access rules against request {{(pid=81767) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 16:04:28.060051 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG dbcounter [-] [81767] Writing DB stats keystone:SELECT=100 {{(pid=81767) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 16:04:28.153474 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-978b652b-20ac-4026-8004-39f75f8bbc0a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4ef569e41fb46d0b721bb1c7663616f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4ee0413f9ef4d4fb1f95a660babe090', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81767) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 16:04:28.153985 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-978b652b-20ac-4026-8004-39f75f8bbc0a service nova] REQUEST_METHOD: `GET` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:04:28.154262 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-978b652b-20ac-4026-8004-39f75f8bbc0a service nova] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:04:28.154526 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-978b652b-20ac-4026-8004-39f75f8bbc0a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:04:28.156147 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.fernet_utils [None req-978b652b-20ac-4026-8004-39f75f8bbc0a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81767) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 16:04:28.259415 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-978b652b-20ac-4026-8004-39f75f8bbc0a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 16:04:28.261067 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-978b652b-20ac-4026-8004-39f75f8bbc0a service nova] RBAC: Authorization granted {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 16:04:28.281083 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 710/1440] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Aug 22 16:04:27 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 419 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 22 16:04:30.198724 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7adea8d0-44df-48f4-82b4-5d610432ea7c None None] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:04:30.199415 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7adea8d0-44df-48f4-82b4-5d610432ea7c None None] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:04:30.199415 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7adea8d0-44df-48f4-82b4-5d610432ea7c None None] PATH_INFO: `/` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:04:30.200700 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 731/1441] 127.0.0.1 () {54 vars in 917 bytes} [Mon Aug 22 16:04:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 22 16:04:30.206666 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5ccbdc7-27e4-4812-952a-869a8b068019 None None] REQUEST_METHOD: `POST` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:04:30.209170 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5ccbdc7-27e4-4812-952a-869a8b068019 None None] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:04:30.209170 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5ccbdc7-27e4-4812-952a-869a8b068019 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:04:30.254967 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG dbcounter [-] [81767] Writing DB stats keystone:SELECT=100 {{(pid=81767) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 16:04:30.291259 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.auth.core [None req-c5ccbdc7-27e4-4812-952a-869a8b068019 None None] MFA Rules not processed for user `6b2199ae0fb0480aa917ada391f5ec54`. Rule list: `[]` (Enabled: `True`). {{(pid=81767) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 16:04:30.292521 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.fernet_utils [None req-c5ccbdc7-27e4-4812-952a-869a8b068019 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81767) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 16:04:30.408444 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 711/1442] 127.0.0.1 () {58 vars in 1014 bytes} [Mon Aug 22 16:04:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 203 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 16:04:30.696847 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7adea8d0-44df-48f4-82b4-5d610432ea7c None None] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 16:04:30.698533 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-7adea8d0-44df-48f4-82b4-5d610432ea7c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 16:04:30.812790 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7adea8d0-44df-48f4-82b4-5d610432ea7c None None] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 16:04:30.848750 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=100 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 16:04:30.895205 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85481b1d-6e66-45d7-a6df-b531de7e137a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '234687adad594f1aa45775a565ffbdc7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4ee0413f9ef4d4fb1f95a660babe090', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 16:04:30.895844 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85481b1d-6e66-45d7-a6df-b531de7e137a service neutron] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:04:30.896123 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85481b1d-6e66-45d7-a6df-b531de7e137a service neutron] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:04:30.896391 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85481b1d-6e66-45d7-a6df-b531de7e137a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:04:30.898030 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-85481b1d-6e66-45d7-a6df-b531de7e137a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 16:04:31.035549 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85481b1d-6e66-45d7-a6df-b531de7e137a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 16:04:31.037217 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85481b1d-6e66-45d7-a6df-b531de7e137a service neutron] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 16:04:31.057660 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 732/1443] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Aug 22 16:04:30 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 361 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 16:04:33.238280 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ce8de1-ff8c-405f-a63d-756aa68885cd None None] REQUEST_METHOD: `GET` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:04:33.238823 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ce8de1-ff8c-405f-a63d-756aa68885cd None None] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:04:33.238823 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ce8de1-ff8c-405f-a63d-756aa68885cd None None] PATH_INFO: `/` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:04:33.240113 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 712/1444] 127.0.0.1 () {54 vars in 917 bytes} [Mon Aug 22 16:04:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 16:04:33.245734 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cba89af-fe4e-42a7-9573-76807868e420 None None] REQUEST_METHOD: `POST` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:04:33.246216 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cba89af-fe4e-42a7-9573-76807868e420 None None] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:04:33.246313 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cba89af-fe4e-42a7-9573-76807868e420 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:04:33.319955 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.auth.core [None req-6cba89af-fe4e-42a7-9573-76807868e420 None None] MFA Rules not processed for user `6b2199ae0fb0480aa917ada391f5ec54`. Rule list: `[]` (Enabled: `True`). {{(pid=81768) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 16:04:33.321239 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-6cba89af-fe4e-42a7-9573-76807868e420 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 16:04:33.336344 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=100 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 16:04:33.434992 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 733/1445] 127.0.0.1 () {58 vars in 1014 bytes} [Mon Aug 22 16:04:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 16:04:33.729175 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12ce8de1-ff8c-405f-a63d-756aa68885cd None None] Authenticating user token {{(pid=81767) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 16:04:33.730704 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.fernet_utils [None req-12ce8de1-ff8c-405f-a63d-756aa68885cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81767) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 16:04:33.844359 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12ce8de1-ff8c-405f-a63d-756aa68885cd None None] Validating token access rules against request {{(pid=81767) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 16:04:33.861154 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG dbcounter [-] [81767] Writing DB stats keystone:SELECT=100 {{(pid=81767) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 16:04:33.939910 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfe48785-07d8-4236-8a6e-2e1a89b6ff9d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '234687adad594f1aa45775a565ffbdc7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4ee0413f9ef4d4fb1f95a660babe090', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81767) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 16:04:33.940533 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfe48785-07d8-4236-8a6e-2e1a89b6ff9d service neutron] REQUEST_METHOD: `GET` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:04:33.940814 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfe48785-07d8-4236-8a6e-2e1a89b6ff9d service neutron] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:04:33.941082 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfe48785-07d8-4236-8a6e-2e1a89b6ff9d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:04:33.942742 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.fernet_utils [None req-cfe48785-07d8-4236-8a6e-2e1a89b6ff9d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81767) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 16:04:34.079876 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfe48785-07d8-4236-8a6e-2e1a89b6ff9d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 16:04:34.081322 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfe48785-07d8-4236-8a6e-2e1a89b6ff9d service neutron] RBAC: Authorization granted {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 16:04:34.102365 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 713/1446] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Aug 22 16:04:33 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 374 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 16:04:36.722481 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32b00645-fc79-483f-9cbb-23fd8cc48ba2 None None] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:04:36.723095 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32b00645-fc79-483f-9cbb-23fd8cc48ba2 None None] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:04:36.723095 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32b00645-fc79-483f-9cbb-23fd8cc48ba2 None None] PATH_INFO: `/` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:04:36.724495 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 734/1447] 127.0.0.1 () {54 vars in 917 bytes} [Mon Aug 22 16:04:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 16:04:36.731225 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-423d729f-43b1-43f7-9c6b-f4ea9c1a0d8d None None] REQUEST_METHOD: `POST` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:04:36.731805 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-423d729f-43b1-43f7-9c6b-f4ea9c1a0d8d None None] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:04:36.731805 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-423d729f-43b1-43f7-9c6b-f4ea9c1a0d8d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:04:36.800697 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG dbcounter [-] [81767] Writing DB stats keystone:SELECT=100 {{(pid=81767) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 16:04:36.823849 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.auth.core [None req-423d729f-43b1-43f7-9c6b-f4ea9c1a0d8d None None] MFA Rules not processed for user `6b2199ae0fb0480aa917ada391f5ec54`. Rule list: `[]` (Enabled: `True`). {{(pid=81767) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 16:04:36.825094 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.fernet_utils [None req-423d729f-43b1-43f7-9c6b-f4ea9c1a0d8d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81767) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 16:04:36.930902 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 714/1448] 127.0.0.1 () {58 vars in 1014 bytes} [Mon Aug 22 16:04:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 201 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 16:04:37.221724 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32b00645-fc79-483f-9cbb-23fd8cc48ba2 None None] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 16:04:37.223341 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-32b00645-fc79-483f-9cbb-23fd8cc48ba2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 16:04:37.342495 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32b00645-fc79-483f-9cbb-23fd8cc48ba2 None None] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 16:04:37.385419 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=100 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 16:04:37.421992 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd701507-cf93-4e4d-a965-647e0a3cc4ed service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '234687adad594f1aa45775a565ffbdc7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4ee0413f9ef4d4fb1f95a660babe090', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 16:04:37.423380 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd701507-cf93-4e4d-a965-647e0a3cc4ed service neutron] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:04:37.423593 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd701507-cf93-4e4d-a965-647e0a3cc4ed service neutron] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:04:37.423866 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd701507-cf93-4e4d-a965-647e0a3cc4ed service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:04:37.425575 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-cd701507-cf93-4e4d-a965-647e0a3cc4ed service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 16:04:37.544722 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd701507-cf93-4e4d-a965-647e0a3cc4ed service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 16:04:37.546245 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd701507-cf93-4e4d-a965-647e0a3cc4ed service neutron] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 16:04:37.567076 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 735/1449] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Aug 22 16:04:37 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 346 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 16:04:39.798322 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c46d30b6-7c9a-4385-95d7-0beb6aa7d81e None None] REQUEST_METHOD: `GET` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:04:39.799115 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c46d30b6-7c9a-4385-95d7-0beb6aa7d81e None None] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:04:39.799115 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c46d30b6-7c9a-4385-95d7-0beb6aa7d81e None None] PATH_INFO: `/` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:04:39.800413 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 715/1450] 127.0.0.1 () {54 vars in 917 bytes} [Mon Aug 22 16:04:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 16:04:39.806745 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91009968-43c6-43ee-8d7f-eb11fd82f3c4 None None] REQUEST_METHOD: `POST` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:04:39.807374 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91009968-43c6-43ee-8d7f-eb11fd82f3c4 None None] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:04:39.807494 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91009968-43c6-43ee-8d7f-eb11fd82f3c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:04:39.909722 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.auth.core [None req-91009968-43c6-43ee-8d7f-eb11fd82f3c4 None None] MFA Rules not processed for user `6b2199ae0fb0480aa917ada391f5ec54`. Rule list: `[]` (Enabled: `True`). {{(pid=81768) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 16:04:39.911035 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-91009968-43c6-43ee-8d7f-eb11fd82f3c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 16:04:39.938929 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=100 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 16:04:40.048503 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 736/1451] 127.0.0.1 () {58 vars in 1014 bytes} [Mon Aug 22 16:04:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 243 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 16:04:40.342655 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c46d30b6-7c9a-4385-95d7-0beb6aa7d81e None None] Authenticating user token {{(pid=81767) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 16:04:40.344244 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.fernet_utils [None req-c46d30b6-7c9a-4385-95d7-0beb6aa7d81e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81767) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 16:04:40.489287 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c46d30b6-7c9a-4385-95d7-0beb6aa7d81e None None] Validating token access rules against request {{(pid=81767) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 16:04:40.511528 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG dbcounter [-] [81767] Writing DB stats keystone:SELECT=100 {{(pid=81767) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 16:04:40.579227 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90fec97e-fba1-46ef-859a-5428bc9fa4f2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '234687adad594f1aa45775a565ffbdc7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4ee0413f9ef4d4fb1f95a660babe090', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81767) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 16:04:40.580183 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90fec97e-fba1-46ef-859a-5428bc9fa4f2 service neutron] REQUEST_METHOD: `GET` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:04:40.580456 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90fec97e-fba1-46ef-859a-5428bc9fa4f2 service neutron] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:04:40.580801 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90fec97e-fba1-46ef-859a-5428bc9fa4f2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:04:40.582515 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.fernet_utils [None req-90fec97e-fba1-46ef-859a-5428bc9fa4f2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81767) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 16:04:40.721916 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90fec97e-fba1-46ef-859a-5428bc9fa4f2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 16:04:40.724091 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90fec97e-fba1-46ef-859a-5428bc9fa4f2 service neutron] RBAC: Authorization granted {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 16:04:40.759778 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 716/1452] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Aug 22 16:04:40 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 418 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 16:04:42.929568 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c3d6c26-5f7c-4778-b863-2a22b2446752 None None] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:04:42.930417 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c3d6c26-5f7c-4778-b863-2a22b2446752 None None] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:04:42.930417 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c3d6c26-5f7c-4778-b863-2a22b2446752 None None] PATH_INFO: `/` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:04:42.931503 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 737/1453] 127.0.0.1 () {54 vars in 917 bytes} [Mon Aug 22 16:04:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 16:04:42.939025 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-511baa07-d748-49c8-8d16-d8a6dfc41b57 None None] REQUEST_METHOD: `POST` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:04:42.939913 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-511baa07-d748-49c8-8d16-d8a6dfc41b57 None None] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:04:42.940090 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-511baa07-d748-49c8-8d16-d8a6dfc41b57 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:04:43.061134 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG dbcounter [-] [81767] Writing DB stats keystone:SELECT=100 {{(pid=81767) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 16:04:43.066634 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.auth.core [None req-511baa07-d748-49c8-8d16-d8a6dfc41b57 None None] MFA Rules not processed for user `6b2199ae0fb0480aa917ada391f5ec54`. Rule list: `[]` (Enabled: `True`). {{(pid=81767) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 16:04:43.068830 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.fernet_utils [None req-511baa07-d748-49c8-8d16-d8a6dfc41b57 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81767) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 16:04:43.222337 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 717/1454] 127.0.0.1 () {58 vars in 1014 bytes} [Mon Aug 22 16:04:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 285 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 16:04:43.519011 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c3d6c26-5f7c-4778-b863-2a22b2446752 None None] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 16:04:43.521835 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-9c3d6c26-5f7c-4778-b863-2a22b2446752 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 16:04:43.696698 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c3d6c26-5f7c-4778-b863-2a22b2446752 None None] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 16:04:43.757850 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=100 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 16:04:43.784905 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6a91d6e-5df8-4504-bfa7-61ad70bb49cd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '234687adad594f1aa45775a565ffbdc7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4ee0413f9ef4d4fb1f95a660babe090', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 16:04:43.785518 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a91d6e-5df8-4504-bfa7-61ad70bb49cd service neutron] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:04:43.785788 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a91d6e-5df8-4504-bfa7-61ad70bb49cd service neutron] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:04:43.786046 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a91d6e-5df8-4504-bfa7-61ad70bb49cd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:04:43.787735 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-a6a91d6e-5df8-4504-bfa7-61ad70bb49cd service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 16:04:43.937400 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6a91d6e-5df8-4504-bfa7-61ad70bb49cd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 16:04:43.940813 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6a91d6e-5df8-4504-bfa7-61ad70bb49cd service neutron] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 16:04:43.978735 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 738/1455] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Aug 22 16:04:43 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 461 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 16:04:46.192533 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69548b29-f3c5-4ef8-9a19-a0ea805519d1 None None] REQUEST_METHOD: `GET` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:04:46.193313 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69548b29-f3c5-4ef8-9a19-a0ea805519d1 None None] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:04:46.193313 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69548b29-f3c5-4ef8-9a19-a0ea805519d1 None None] PATH_INFO: `/` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:04:46.194474 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 718/1456] 127.0.0.1 () {54 vars in 917 bytes} [Mon Aug 22 16:04:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 16:04:46.201422 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46abbf0f-e3fe-4162-9a8c-34c2d9c944e9 None None] REQUEST_METHOD: `POST` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:04:46.202274 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46abbf0f-e3fe-4162-9a8c-34c2d9c944e9 None None] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:04:46.202437 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46abbf0f-e3fe-4162-9a8c-34c2d9c944e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:04:46.274174 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.auth.core [None req-46abbf0f-e3fe-4162-9a8c-34c2d9c944e9 None None] MFA Rules not processed for user `6b2199ae0fb0480aa917ada391f5ec54`. Rule list: `[]` (Enabled: `True`). {{(pid=81768) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 16:04:46.275480 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-46abbf0f-e3fe-4162-9a8c-34c2d9c944e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 16:04:46.326361 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=100 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 16:04:46.413835 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 739/1457] 127.0.0.1 () {58 vars in 1014 bytes} [Mon Aug 22 16:04:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 214 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 16:04:46.703237 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69548b29-f3c5-4ef8-9a19-a0ea805519d1 None None] Authenticating user token {{(pid=81767) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 16:04:46.705656 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.fernet_utils [None req-69548b29-f3c5-4ef8-9a19-a0ea805519d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81767) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 16:04:46.818486 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69548b29-f3c5-4ef8-9a19-a0ea805519d1 None None] Validating token access rules against request {{(pid=81767) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 16:04:46.856356 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG dbcounter [-] [81767] Writing DB stats keystone:SELECT=100 {{(pid=81767) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 16:04:46.909038 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-751e41fe-081d-4f07-83af-f20135ad954f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '234687adad594f1aa45775a565ffbdc7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4ee0413f9ef4d4fb1f95a660babe090', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81767) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 16:04:46.909745 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-751e41fe-081d-4f07-83af-f20135ad954f service neutron] REQUEST_METHOD: `GET` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:04:46.910040 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-751e41fe-081d-4f07-83af-f20135ad954f service neutron] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:04:46.910310 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-751e41fe-081d-4f07-83af-f20135ad954f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:04:46.912502 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.fernet_utils [None req-751e41fe-081d-4f07-83af-f20135ad954f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81767) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 16:04:47.055414 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-751e41fe-081d-4f07-83af-f20135ad954f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 16:04:47.056887 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-751e41fe-081d-4f07-83af-f20135ad954f service neutron] RBAC: Authorization granted {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 16:04:47.079238 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 719/1458] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Aug 22 16:04:46 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 376 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 16:04:49.136059 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f26bf01f-e499-4c86-8a93-21d2b1777030 None None] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:04:49.136912 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f26bf01f-e499-4c86-8a93-21d2b1777030 None None] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:04:49.136912 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f26bf01f-e499-4c86-8a93-21d2b1777030 None None] PATH_INFO: `/` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:04:49.138037 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 740/1459] 127.0.0.1 () {54 vars in 917 bytes} [Mon Aug 22 16:04:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 16:04:49.144558 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4be89bdf-155e-47c5-bf11-b3b9bab85cda None None] REQUEST_METHOD: `POST` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:04:49.145147 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4be89bdf-155e-47c5-bf11-b3b9bab85cda None None] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:04:49.145147 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4be89bdf-155e-47c5-bf11-b3b9bab85cda None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:04:49.220938 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.auth.core [None req-4be89bdf-155e-47c5-bf11-b3b9bab85cda None None] MFA Rules not processed for user `6b2199ae0fb0480aa917ada391f5ec54`. Rule list: `[]` (Enabled: `True`). {{(pid=81767) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 16:04:49.222092 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.fernet_utils [None req-4be89bdf-155e-47c5-bf11-b3b9bab85cda None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81767) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 16:04:49.231213 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG dbcounter [-] [81767] Writing DB stats keystone:SELECT=100 {{(pid=81767) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 16:04:49.335319 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 720/1460] 127.0.0.1 () {58 vars in 1014 bytes} [Mon Aug 22 16:04:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 16:04:49.629711 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f26bf01f-e499-4c86-8a93-21d2b1777030 None None] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 16:04:49.632793 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-f26bf01f-e499-4c86-8a93-21d2b1777030 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 16:04:49.770073 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f26bf01f-e499-4c86-8a93-21d2b1777030 None None] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 16:04:49.836366 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=100 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 16:04:49.857905 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b99cb738-feeb-4ed5-9d07-608c9d5800d4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '234687adad594f1aa45775a565ffbdc7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4ee0413f9ef4d4fb1f95a660babe090', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 16:04:49.858519 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b99cb738-feeb-4ed5-9d07-608c9d5800d4 service neutron] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:04:49.858816 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b99cb738-feeb-4ed5-9d07-608c9d5800d4 service neutron] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:04:49.859071 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b99cb738-feeb-4ed5-9d07-608c9d5800d4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:04:49.861298 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-b99cb738-feeb-4ed5-9d07-608c9d5800d4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 16:04:50.002919 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b99cb738-feeb-4ed5-9d07-608c9d5800d4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 16:04:50.004322 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b99cb738-feeb-4ed5-9d07-608c9d5800d4 service neutron] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 16:04:50.025830 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 741/1461] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Aug 22 16:04:49 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 397 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 16:04:52.186735 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56c4133c-b1d1-4daa-8dce-d564cb22034f None None] REQUEST_METHOD: `GET` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:04:52.187348 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56c4133c-b1d1-4daa-8dce-d564cb22034f None None] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:04:52.187476 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56c4133c-b1d1-4daa-8dce-d564cb22034f None None] PATH_INFO: `/` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:04:52.188961 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 721/1462] 127.0.0.1 () {54 vars in 917 bytes} [Mon Aug 22 16:04:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 16:04:52.194611 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d65b5d5-d6cd-4354-bf8f-1bd843252cc7 None None] REQUEST_METHOD: `POST` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:04:52.195180 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d65b5d5-d6cd-4354-bf8f-1bd843252cc7 None None] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:04:52.195180 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d65b5d5-d6cd-4354-bf8f-1bd843252cc7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:04:52.272445 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.auth.core [None req-1d65b5d5-d6cd-4354-bf8f-1bd843252cc7 None None] MFA Rules not processed for user `6b2199ae0fb0480aa917ada391f5ec54`. Rule list: `[]` (Enabled: `True`). {{(pid=81768) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 16:04:52.273678 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-1d65b5d5-d6cd-4354-bf8f-1bd843252cc7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 16:04:52.326682 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=100 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 16:04:52.396482 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 742/1463] 127.0.0.1 () {58 vars in 1014 bytes} [Mon Aug 22 16:04:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 203 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 16:04:52.688380 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56c4133c-b1d1-4daa-8dce-d564cb22034f None None] Authenticating user token {{(pid=81767) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 16:04:52.689937 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.fernet_utils [None req-56c4133c-b1d1-4daa-8dce-d564cb22034f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81767) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 16:04:52.803673 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56c4133c-b1d1-4daa-8dce-d564cb22034f None None] Validating token access rules against request {{(pid=81767) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 16:04:52.846478 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG dbcounter [-] [81767] Writing DB stats keystone:SELECT=100 {{(pid=81767) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 16:04:52.890585 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4daa1a18-2524-4d67-9871-7203696adec2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '234687adad594f1aa45775a565ffbdc7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4ee0413f9ef4d4fb1f95a660babe090', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81767) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 16:04:52.891227 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4daa1a18-2524-4d67-9871-7203696adec2 service neutron] REQUEST_METHOD: `GET` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:04:52.891528 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4daa1a18-2524-4d67-9871-7203696adec2 service neutron] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:04:52.891793 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4daa1a18-2524-4d67-9871-7203696adec2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:04:52.893401 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.fernet_utils [None req-4daa1a18-2524-4d67-9871-7203696adec2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81767) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 16:04:53.030858 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4daa1a18-2524-4d67-9871-7203696adec2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 16:04:53.033714 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4daa1a18-2524-4d67-9871-7203696adec2 service neutron] RBAC: Authorization granted {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 16:04:53.069589 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 722/1464] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Aug 22 16:04:52 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 382 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 16:04:55.082222 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c31b560-bb60-4542-baa2-105bdb7e570e None None] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:04:55.082903 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c31b560-bb60-4542-baa2-105bdb7e570e None None] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:04:55.082903 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c31b560-bb60-4542-baa2-105bdb7e570e None None] PATH_INFO: `/` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:04:55.084159 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 743/1465] 127.0.0.1 () {54 vars in 917 bytes} [Mon Aug 22 16:04:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 16:04:55.090648 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7c466f9-c978-4a84-b4fc-2857f0e0358d None None] REQUEST_METHOD: `POST` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:04:55.091212 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7c466f9-c978-4a84-b4fc-2857f0e0358d None None] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:04:55.091420 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7c466f9-c978-4a84-b4fc-2857f0e0358d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:04:55.181747 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.auth.core [None req-e7c466f9-c978-4a84-b4fc-2857f0e0358d None None] MFA Rules not processed for user `6b2199ae0fb0480aa917ada391f5ec54`. Rule list: `[]` (Enabled: `True`). {{(pid=81767) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 16:04:55.183621 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.fernet_utils [None req-e7c466f9-c978-4a84-b4fc-2857f0e0358d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81767) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 16:04:55.213065 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG dbcounter [-] [81767] Writing DB stats keystone:SELECT=100 {{(pid=81767) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 16:04:55.331898 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 723/1466] 127.0.0.1 () {58 vars in 1014 bytes} [Mon Aug 22 16:04:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 243 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 22 16:04:55.623778 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c31b560-bb60-4542-baa2-105bdb7e570e None None] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 16:04:55.625920 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-3c31b560-bb60-4542-baa2-105bdb7e570e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 16:04:55.787367 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c31b560-bb60-4542-baa2-105bdb7e570e None None] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 16:04:55.892992 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=100 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 16:04:55.901853 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d6eec75-ec8d-4ac3-bed3-a7d0ff08da69 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '234687adad594f1aa45775a565ffbdc7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4ee0413f9ef4d4fb1f95a660babe090', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 16:04:55.902536 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d6eec75-ec8d-4ac3-bed3-a7d0ff08da69 service neutron] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:04:55.905536 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d6eec75-ec8d-4ac3-bed3-a7d0ff08da69 service neutron] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:04:55.905536 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d6eec75-ec8d-4ac3-bed3-a7d0ff08da69 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:04:55.905536 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-8d6eec75-ec8d-4ac3-bed3-a7d0ff08da69 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 16:04:56.068646 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d6eec75-ec8d-4ac3-bed3-a7d0ff08da69 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 16:04:56.070188 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d6eec75-ec8d-4ac3-bed3-a7d0ff08da69 service neutron] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 16:04:56.093330 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 744/1467] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Aug 22 16:04:55 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 470 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 16:04:58.295484 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76c20076-aa2c-4351-994d-326dcbb766ff None None] REQUEST_METHOD: `GET` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:04:58.296191 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76c20076-aa2c-4351-994d-326dcbb766ff None None] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:04:58.296341 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76c20076-aa2c-4351-994d-326dcbb766ff None None] PATH_INFO: `/` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:04:58.298405 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 724/1468] 127.0.0.1 () {54 vars in 917 bytes} [Mon Aug 22 16:04:58 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 16:04:58.305665 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2047919-00e8-4874-aed9-f30b8764de16 None None] REQUEST_METHOD: `POST` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:04:58.306521 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2047919-00e8-4874-aed9-f30b8764de16 None None] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:04:58.306521 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2047919-00e8-4874-aed9-f30b8764de16 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:04:58.405245 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.auth.core [None req-e2047919-00e8-4874-aed9-f30b8764de16 None None] MFA Rules not processed for user `6b2199ae0fb0480aa917ada391f5ec54`. Rule list: `[]` (Enabled: `True`). {{(pid=81768) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 16:04:58.406570 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-e2047919-00e8-4874-aed9-f30b8764de16 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 16:04:58.503364 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=100 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 16:04:58.599220 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 745/1469] 127.0.0.1 () {58 vars in 1014 bytes} [Mon Aug 22 16:04:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 295 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 22 16:04:58.906776 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76c20076-aa2c-4351-994d-326dcbb766ff None None] Authenticating user token {{(pid=81767) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 16:04:58.908352 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.fernet_utils [None req-76c20076-aa2c-4351-994d-326dcbb766ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81767) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 16:04:59.033971 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76c20076-aa2c-4351-994d-326dcbb766ff None None] Validating token access rules against request {{(pid=81767) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 16:04:59.092128 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG dbcounter [-] [81767] Writing DB stats keystone:SELECT=100 {{(pid=81767) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 16:04:59.121457 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a96cc096-f325-4d9e-aee1-78f5dedcba54 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '234687adad594f1aa45775a565ffbdc7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4ee0413f9ef4d4fb1f95a660babe090', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81767) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 16:04:59.122073 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a96cc096-f325-4d9e-aee1-78f5dedcba54 service neutron] REQUEST_METHOD: `GET` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:04:59.122403 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a96cc096-f325-4d9e-aee1-78f5dedcba54 service neutron] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:04:59.122883 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a96cc096-f325-4d9e-aee1-78f5dedcba54 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:04:59.126404 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.fernet_utils [None req-a96cc096-f325-4d9e-aee1-78f5dedcba54 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81767) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 16:04:59.245317 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a96cc096-f325-4d9e-aee1-78f5dedcba54 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 16:04:59.246791 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a96cc096-f325-4d9e-aee1-78f5dedcba54 service neutron] RBAC: Authorization granted {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 16:04:59.266259 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 725/1470] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Aug 22 16:04:58 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 360 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 16:05:08.594900 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=22 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 16:05:09.264079 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG dbcounter [-] [81767] Writing DB stats keystone:SELECT=66 {{(pid=81767) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 16:05:20.766563 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c1a8a5-7bb6-43b5-bcbe-ed7a511c5921 None None] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:05:20.767153 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c1a8a5-7bb6-43b5-bcbe-ed7a511c5921 None None] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:05:20.767153 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c1a8a5-7bb6-43b5-bcbe-ed7a511c5921 None None] PATH_INFO: `/` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:05:20.768458 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 746/1471] 10.222.0.15 () {54 vars in 927 bytes} [Mon Aug 22 16:05:20 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 16:05:20.774649 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf71efc-e8d0-4b90-abb9-ea1723e6ac9a None None] REQUEST_METHOD: `POST` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:05:20.775221 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf71efc-e8d0-4b90-abb9-ea1723e6ac9a None None] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:05:20.775221 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf71efc-e8d0-4b90-abb9-ea1723e6ac9a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:05:20.843769 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.auth.core [None req-0bf71efc-e8d0-4b90-abb9-ea1723e6ac9a None None] MFA Rules not processed for user `6b2199ae0fb0480aa917ada391f5ec54`. Rule list: `[]` (Enabled: `True`). {{(pid=81767) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 16:05:20.845694 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.fernet_utils [None req-0bf71efc-e8d0-4b90-abb9-ea1723e6ac9a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81767) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 16:05:20.951333 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 726/1472] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Aug 22 16:05:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 22 16:05:21.252757 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7846aa91-5b2b-4873-953e-4ff69a1e7af5 None None] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:05:21.253030 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7846aa91-5b2b-4873-953e-4ff69a1e7af5 None None] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:05:21.253298 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7846aa91-5b2b-4873-953e-4ff69a1e7af5 None None] PATH_INFO: `/` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:05:21.254653 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 747/1473] 10.222.0.15 () {54 vars in 967 bytes} [Mon Aug 22 16:05:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 16:05:21.259697 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae776583-01c3-4c1a-a240-6585e47f763e None None] REQUEST_METHOD: `POST` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:05:21.259984 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae776583-01c3-4c1a-a240-6585e47f763e None None] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:05:21.260262 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae776583-01c3-4c1a-a240-6585e47f763e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:05:21.341273 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.auth.core [None req-ae776583-01c3-4c1a-a240-6585e47f763e None None] MFA Rules not processed for user `91f565473c824a15930e2b12863c1ed3`. Rule list: `[]` (Enabled: `True`). {{(pid=81767) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 16:05:21.342496 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.fernet_utils [None req-ae776583-01c3-4c1a-a240-6585e47f763e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81767) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 16:05:21.364519 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG dbcounter [-] [81767] Writing DB stats keystone:SELECT=100 {{(pid=81767) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 16:05:21.428073 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 727/1474] 10.222.0.15 () {58 vars in 1064 bytes} [Mon Aug 22 16:05:21 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 16:05:21.436002 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7846aa91-5b2b-4873-953e-4ff69a1e7af5 None None] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 16:05:21.438146 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-7846aa91-5b2b-4873-953e-4ff69a1e7af5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 16:05:21.546919 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7846aa91-5b2b-4873-953e-4ff69a1e7af5 None None] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 16:05:21.624004 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aabcd97-f30b-410b-90b7-bc638485a87c service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91f565473c824a15930e2b12863c1ed3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4ee0413f9ef4d4fb1f95a660babe090', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 16:05:21.625204 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aabcd97-f30b-410b-90b7-bc638485a87c service glance] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:05:21.625474 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aabcd97-f30b-410b-90b7-bc638485a87c service glance] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:05:21.625736 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aabcd97-f30b-410b-90b7-bc638485a87c service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:05:21.627437 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-3aabcd97-f30b-410b-90b7-bc638485a87c service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 16:05:21.685904 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=100 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 16:05:21.744288 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aabcd97-f30b-410b-90b7-bc638485a87c service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 16:05:21.745737 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aabcd97-f30b-410b-90b7-bc638485a87c service glance] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 16:05:21.765722 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 748/1475] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Aug 22 16:05:21 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 330 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 16:05:23.846125 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e442860d-ea6f-4416-942d-6783c00935ff None None] REQUEST_METHOD: `GET` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:05:23.846720 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e442860d-ea6f-4416-942d-6783c00935ff None None] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:05:23.846720 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e442860d-ea6f-4416-942d-6783c00935ff None None] PATH_INFO: `/` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:05:23.848028 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 728/1476] 10.222.0.15 () {54 vars in 927 bytes} [Mon Aug 22 16:05:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 16:05:23.853671 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-266e49f9-5f5a-45c7-a208-510d699a5e2b None None] REQUEST_METHOD: `POST` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:05:23.854137 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-266e49f9-5f5a-45c7-a208-510d699a5e2b None None] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:05:23.854262 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-266e49f9-5f5a-45c7-a208-510d699a5e2b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:05:23.932661 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.auth.core [None req-266e49f9-5f5a-45c7-a208-510d699a5e2b None None] MFA Rules not processed for user `6b2199ae0fb0480aa917ada391f5ec54`. Rule list: `[]` (Enabled: `True`). {{(pid=81768) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 16:05:23.933889 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-266e49f9-5f5a-45c7-a208-510d699a5e2b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 16:05:24.042391 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 749/1477] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Aug 22 16:05:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 16:05:24.324624 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e442860d-ea6f-4416-942d-6783c00935ff None None] Authenticating user token {{(pid=81767) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 16:05:24.326180 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.fernet_utils [None req-e442860d-ea6f-4416-942d-6783c00935ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81767) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 16:05:24.459574 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e442860d-ea6f-4416-942d-6783c00935ff None None] Validating token access rules against request {{(pid=81767) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 16:05:24.509978 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG dbcounter [-] [81767] Writing DB stats keystone:SELECT=100 {{(pid=81767) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 16:05:24.560139 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30543ee1-f475-4097-8a7e-5b56563e9b2d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a98b1e4431940ab8d1fc95df2858a6c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81767) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 16:05:24.560748 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30543ee1-f475-4097-8a7e-5b56563e9b2d admin admin] REQUEST_METHOD: `GET` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:05:24.561051 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30543ee1-f475-4097-8a7e-5b56563e9b2d admin admin] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:05:24.561358 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30543ee1-f475-4097-8a7e-5b56563e9b2d admin admin] PATH_INFO: `/v3/projects` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:05:24.562762 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30543ee1-f475-4097-8a7e-5b56563e9b2d admin admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 16:05:24.564589 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: WARNING py.warnings [None req-30543ee1-f475-4097-8a7e-5b56563e9b2d admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_projects": "(role:reader and system_scope:all) or (role:reader and domain_id:%(target.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 22 16:05:24.564589 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: warnings.warn(msg) Aug 22 16:05:24.564589 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: Aug 22 16:05:24.565825 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30543ee1-f475-4097-8a7e-5b56563e9b2d admin admin] RBAC: Authorization granted {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 16:05:24.585612 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 729/1478] 10.222.0.15 () {56 vars in 1108 bytes} [Mon Aug 22 16:05:24 2022] GET /identity/v3/projects => generated 2553 bytes in 262 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 22 16:05:26.591033 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e34a4da4-9f3f-4bb5-8516-1a47e0b895f9 None None] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:05:26.591654 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e34a4da4-9f3f-4bb5-8516-1a47e0b895f9 None None] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:05:26.591654 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e34a4da4-9f3f-4bb5-8516-1a47e0b895f9 None None] PATH_INFO: `/` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:05:26.592980 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 750/1479] 10.222.0.15 () {54 vars in 927 bytes} [Mon Aug 22 16:05:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 16:05:26.598588 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22a88275-7518-4d8c-99ed-6f8cc9f71574 None None] REQUEST_METHOD: `POST` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:05:26.599028 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22a88275-7518-4d8c-99ed-6f8cc9f71574 None None] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:05:26.599144 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22a88275-7518-4d8c-99ed-6f8cc9f71574 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:05:26.669218 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.auth.core [None req-22a88275-7518-4d8c-99ed-6f8cc9f71574 None None] MFA Rules not processed for user `6b2199ae0fb0480aa917ada391f5ec54`. Rule list: `[]` (Enabled: `True`). {{(pid=81767) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 16:05:26.670450 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.fernet_utils [None req-22a88275-7518-4d8c-99ed-6f8cc9f71574 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81767) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 16:05:26.774553 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 730/1480] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Aug 22 16:05:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 16:05:27.072271 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e34a4da4-9f3f-4bb5-8516-1a47e0b895f9 None None] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 16:05:27.073881 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-e34a4da4-9f3f-4bb5-8516-1a47e0b895f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 16:05:27.084459 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=100 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 16:05:27.238134 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e34a4da4-9f3f-4bb5-8516-1a47e0b895f9 None None] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 16:05:27.328322 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=100 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 16:05:27.363916 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13d0b27a-a6a0-4008-97a0-43c7689e26ea service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4ef569e41fb46d0b721bb1c7663616f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4ee0413f9ef4d4fb1f95a660babe090', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 16:05:27.364529 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13d0b27a-a6a0-4008-97a0-43c7689e26ea service nova] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:05:27.364804 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13d0b27a-a6a0-4008-97a0-43c7689e26ea service nova] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:05:27.365074 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13d0b27a-a6a0-4008-97a0-43c7689e26ea service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:05:27.366739 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-13d0b27a-a6a0-4008-97a0-43c7689e26ea service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 16:05:27.487343 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13d0b27a-a6a0-4008-97a0-43c7689e26ea service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 16:05:27.489039 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13d0b27a-a6a0-4008-97a0-43c7689e26ea service nova] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 16:05:27.507893 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 751/1481] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Aug 22 16:05:27 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 436 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 22 16:05:29.707496 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bef9f997-3b5f-4567-8f09-63a41a186bbd None None] REQUEST_METHOD: `GET` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:05:29.708017 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bef9f997-3b5f-4567-8f09-63a41a186bbd None None] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:05:29.708017 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bef9f997-3b5f-4567-8f09-63a41a186bbd None None] PATH_INFO: `/` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:05:29.709397 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 731/1482] 10.222.0.15 () {54 vars in 927 bytes} [Mon Aug 22 16:05:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 16:05:29.715554 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7be2d00-602b-4dfc-98c5-a8e8a7af9715 None None] REQUEST_METHOD: `POST` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:05:29.716191 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7be2d00-602b-4dfc-98c5-a8e8a7af9715 None None] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:05:29.716191 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7be2d00-602b-4dfc-98c5-a8e8a7af9715 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:05:29.790009 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.auth.core [None req-b7be2d00-602b-4dfc-98c5-a8e8a7af9715 None None] MFA Rules not processed for user `6b2199ae0fb0480aa917ada391f5ec54`. Rule list: `[]` (Enabled: `True`). {{(pid=81768) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 16:05:29.791233 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-b7be2d00-602b-4dfc-98c5-a8e8a7af9715 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 16:05:29.839338 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=100 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 16:05:29.891626 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 752/1483] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Aug 22 16:05:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 16:05:30.199219 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bef9f997-3b5f-4567-8f09-63a41a186bbd None None] Authenticating user token {{(pid=81767) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 16:05:30.200920 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.fernet_utils [None req-bef9f997-3b5f-4567-8f09-63a41a186bbd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81767) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 16:05:30.228735 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG dbcounter [-] [81767] Writing DB stats keystone:SELECT=100 {{(pid=81767) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 16:05:30.322068 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bef9f997-3b5f-4567-8f09-63a41a186bbd None None] Validating token access rules against request {{(pid=81767) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 16:05:30.405055 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dff7b92-95c6-4639-8622-d805304557cb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '234687adad594f1aa45775a565ffbdc7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4ee0413f9ef4d4fb1f95a660babe090', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81767) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 16:05:30.406415 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dff7b92-95c6-4639-8622-d805304557cb service neutron] REQUEST_METHOD: `GET` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:05:30.406692 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dff7b92-95c6-4639-8622-d805304557cb service neutron] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:05:30.406960 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dff7b92-95c6-4639-8622-d805304557cb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:05:30.408657 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.fernet_utils [None req-1dff7b92-95c6-4639-8622-d805304557cb service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81767) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 16:05:30.490504 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG dbcounter [-] [81767] Writing DB stats keystone:SELECT=100 {{(pid=81767) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 16:05:30.529041 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dff7b92-95c6-4639-8622-d805304557cb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 16:05:30.531068 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dff7b92-95c6-4639-8622-d805304557cb service neutron] RBAC: Authorization granted {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 16:05:30.550472 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 732/1484] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Aug 22 16:05:30 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 352 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 16:05:32.548608 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2091f5d-9010-4bca-be48-8091a0a4eab0 None None] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:05:32.549199 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2091f5d-9010-4bca-be48-8091a0a4eab0 None None] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:05:32.549199 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2091f5d-9010-4bca-be48-8091a0a4eab0 None None] PATH_INFO: `/` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:05:32.550480 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 753/1485] 10.222.0.15 () {54 vars in 927 bytes} [Mon Aug 22 16:05:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 16:05:32.555900 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-782d9c4e-1ab7-4b6d-9dcc-db50ca571378 None None] REQUEST_METHOD: `POST` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:05:32.556366 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-782d9c4e-1ab7-4b6d-9dcc-db50ca571378 None None] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:05:32.556469 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-782d9c4e-1ab7-4b6d-9dcc-db50ca571378 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:05:32.626402 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.auth.core [None req-782d9c4e-1ab7-4b6d-9dcc-db50ca571378 None None] MFA Rules not processed for user `6b2199ae0fb0480aa917ada391f5ec54`. Rule list: `[]` (Enabled: `True`). {{(pid=81767) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 16:05:32.627569 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.fernet_utils [None req-782d9c4e-1ab7-4b6d-9dcc-db50ca571378 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81767) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 16:05:32.732509 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 733/1486] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Aug 22 16:05:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 22 16:05:33.048158 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2091f5d-9010-4bca-be48-8091a0a4eab0 None None] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 16:05:33.049733 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-c2091f5d-9010-4bca-be48-8091a0a4eab0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 16:05:33.159917 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2091f5d-9010-4bca-be48-8091a0a4eab0 None None] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 16:05:33.239423 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=100 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 16:05:33.241178 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faac7b23-9f33-41d6-81b8-77ae52cb24c0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '234687adad594f1aa45775a565ffbdc7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4ee0413f9ef4d4fb1f95a660babe090', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 16:05:33.241886 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faac7b23-9f33-41d6-81b8-77ae52cb24c0 service neutron] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:05:33.242312 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faac7b23-9f33-41d6-81b8-77ae52cb24c0 service neutron] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:05:33.242591 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faac7b23-9f33-41d6-81b8-77ae52cb24c0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:05:33.244452 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-faac7b23-9f33-41d6-81b8-77ae52cb24c0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 16:05:33.365412 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faac7b23-9f33-41d6-81b8-77ae52cb24c0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 16:05:33.366879 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faac7b23-9f33-41d6-81b8-77ae52cb24c0 service neutron] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 16:05:33.386549 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 754/1487] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Aug 22 16:05:33 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 339 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 16:05:35.429976 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d3b53e1-16da-48f4-932c-a4a59f61fb49 None None] REQUEST_METHOD: `GET` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:05:35.430534 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d3b53e1-16da-48f4-932c-a4a59f61fb49 None None] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:05:35.430534 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d3b53e1-16da-48f4-932c-a4a59f61fb49 None None] PATH_INFO: `/` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:05:35.431888 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 734/1488] 10.222.0.15 () {54 vars in 927 bytes} [Mon Aug 22 16:05:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 16:05:35.437141 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e41688a9-ab0d-4e51-814d-4ae33e4997a4 None None] REQUEST_METHOD: `POST` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:05:35.437588 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e41688a9-ab0d-4e51-814d-4ae33e4997a4 None None] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:05:35.437693 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e41688a9-ab0d-4e51-814d-4ae33e4997a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:05:35.504312 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.auth.core [None req-e41688a9-ab0d-4e51-814d-4ae33e4997a4 None None] MFA Rules not processed for user `6b2199ae0fb0480aa917ada391f5ec54`. Rule list: `[]` (Enabled: `True`). {{(pid=81768) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 16:05:35.505515 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-e41688a9-ab0d-4e51-814d-4ae33e4997a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 16:05:35.559519 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=100 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 16:05:35.605828 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 755/1489] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Aug 22 16:05:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 16:05:35.899421 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d3b53e1-16da-48f4-932c-a4a59f61fb49 None None] Authenticating user token {{(pid=81767) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 16:05:35.900385 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.fernet_utils [None req-7d3b53e1-16da-48f4-932c-a4a59f61fb49 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81767) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 16:05:35.937957 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG dbcounter [-] [81767] Writing DB stats keystone:SELECT=100 {{(pid=81767) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 16:05:36.006839 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d3b53e1-16da-48f4-932c-a4a59f61fb49 None None] Validating token access rules against request {{(pid=81767) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 16:05:36.081543 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baa22696-ffd3-4acd-b04e-9d56b655152d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '234687adad594f1aa45775a565ffbdc7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4ee0413f9ef4d4fb1f95a660babe090', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81767) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 16:05:36.089347 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baa22696-ffd3-4acd-b04e-9d56b655152d service neutron] REQUEST_METHOD: `GET` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:05:36.089347 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baa22696-ffd3-4acd-b04e-9d56b655152d service neutron] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:05:36.089347 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baa22696-ffd3-4acd-b04e-9d56b655152d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:05:36.089347 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.fernet_utils [None req-baa22696-ffd3-4acd-b04e-9d56b655152d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81767) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 16:05:36.231921 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baa22696-ffd3-4acd-b04e-9d56b655152d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 16:05:36.231921 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baa22696-ffd3-4acd-b04e-9d56b655152d service neutron] RBAC: Authorization granted {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 16:05:36.231921 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 735/1490] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Aug 22 16:05:35 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 330 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 16:05:36.489863 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG dbcounter [-] [81767] Writing DB stats keystone:SELECT=100 {{(pid=81767) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 16:05:38.909165 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4714d69-5fd2-473a-a55f-9f0e41187143 None None] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:05:38.909757 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4714d69-5fd2-473a-a55f-9f0e41187143 None None] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:05:38.909757 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4714d69-5fd2-473a-a55f-9f0e41187143 None None] PATH_INFO: `/` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:05:38.911135 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 756/1491] 10.222.0.15 () {54 vars in 927 bytes} [Mon Aug 22 16:05:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 16:05:38.916582 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a46f5fa3-8488-4b9c-add6-d40b177e4d09 None None] REQUEST_METHOD: `POST` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:05:38.917145 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a46f5fa3-8488-4b9c-add6-d40b177e4d09 None None] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:05:38.917145 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a46f5fa3-8488-4b9c-add6-d40b177e4d09 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:05:38.993313 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.auth.core [None req-a46f5fa3-8488-4b9c-add6-d40b177e4d09 None None] MFA Rules not processed for user `6b2199ae0fb0480aa917ada391f5ec54`. Rule list: `[]` (Enabled: `True`). {{(pid=81767) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 16:05:38.994555 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.fernet_utils [None req-a46f5fa3-8488-4b9c-add6-d40b177e4d09 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81767) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 16:05:39.098987 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 736/1492] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Aug 22 16:05:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 16:05:39.390525 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4714d69-5fd2-473a-a55f-9f0e41187143 None None] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 16:05:39.392318 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-f4714d69-5fd2-473a-a55f-9f0e41187143 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 16:05:39.502456 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4714d69-5fd2-473a-a55f-9f0e41187143 None None] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 16:05:39.577678 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b1e0e40-85d4-4c23-982d-2d9cd512b4fa service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '234687adad594f1aa45775a565ffbdc7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4ee0413f9ef4d4fb1f95a660babe090', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 16:05:39.578352 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b1e0e40-85d4-4c23-982d-2d9cd512b4fa service neutron] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:05:39.578552 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b1e0e40-85d4-4c23-982d-2d9cd512b4fa service neutron] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:05:39.578824 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b1e0e40-85d4-4c23-982d-2d9cd512b4fa service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:05:39.580842 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-3b1e0e40-85d4-4c23-982d-2d9cd512b4fa service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 16:05:39.599641 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=100 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 16:05:39.703254 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b1e0e40-85d4-4c23-982d-2d9cd512b4fa service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 16:05:39.704720 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b1e0e40-85d4-4c23-982d-2d9cd512b4fa service neutron] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 16:05:39.725408 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 757/1493] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Aug 22 16:05:39 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 336 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 22 16:05:49.097066 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG dbcounter [-] [81767] Writing DB stats keystone:SELECT=82 {{(pid=81767) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 16:05:49.723017 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=50 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 16:07:01.769377 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3564220-7dfd-4f1e-9d91-404bd73f4aac None None] REQUEST_METHOD: `POST` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:07:01.769977 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3564220-7dfd-4f1e-9d91-404bd73f4aac None None] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:07:01.769977 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3564220-7dfd-4f1e-9d91-404bd73f4aac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:07:01.860449 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.auth.core [None req-a3564220-7dfd-4f1e-9d91-404bd73f4aac None None] MFA Rules not processed for user `6b2199ae0fb0480aa917ada391f5ec54`. Rule list: `[]` (Enabled: `True`). {{(pid=81767) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 16:07:01.861702 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.fernet_utils [None req-a3564220-7dfd-4f1e-9d91-404bd73f4aac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81767) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 16:07:01.986950 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 737/1494] 10.222.0.15 () {58 vars in 958 bytes} [Mon Aug 22 16:07:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 219 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 16:07:01.995544 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3422a473-4c3a-43d8-bb04-476a8c75e0ae None None] REQUEST_METHOD: `POST` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:07:01.996047 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3422a473-4c3a-43d8-bb04-476a8c75e0ae None None] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:07:01.996155 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3422a473-4c3a-43d8-bb04-476a8c75e0ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:07:02.017628 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.auth.core [None req-3422a473-4c3a-43d8-bb04-476a8c75e0ae None None] MFA Rules not processed for user `6b2199ae0fb0480aa917ada391f5ec54`. Rule list: `[]` (Enabled: `True`). {{(pid=81768) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 16:07:02.018846 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-3422a473-4c3a-43d8-bb04-476a8c75e0ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 16:07:02.027450 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 758/1495] 10.222.0.15 () {58 vars in 958 bytes} [Mon Aug 22 16:07:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 16:07:02.032606 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3564220-7dfd-4f1e-9d91-404bd73f4aac None None] Authenticating user token {{(pid=81767) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 16:07:02.042227 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3564220-7dfd-4f1e-9d91-404bd73f4aac None None] Validating token access rules against request {{(pid=81767) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 16:07:02.046818 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da091b27-1f58-47ef-bca3-84581b960f83 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a98b1e4431940ab8d1fc95df2858a6c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81767) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 16:07:02.047446 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da091b27-1f58-47ef-bca3-84581b960f83 admin admin] REQUEST_METHOD: `GET` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:07:02.047725 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da091b27-1f58-47ef-bca3-84581b960f83 admin admin] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:07:02.047994 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da091b27-1f58-47ef-bca3-84581b960f83 admin admin] PATH_INFO: `/v3/domains` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:07:02.048636 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da091b27-1f58-47ef-bca3-84581b960f83 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81767) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 16:07:02.048931 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da091b27-1f58-47ef-bca3-84581b960f83 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 16:07:02.049932 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: WARNING py.warnings [None req-da091b27-1f58-47ef-bca3-84581b960f83 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Aug 22 16:07:02.049932 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: warnings.warn(msg) Aug 22 16:07:02.049932 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: Aug 22 16:07:02.050812 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da091b27-1f58-47ef-bca3-84581b960f83 admin admin] RBAC: Authorization granted {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 16:07:02.062215 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 738/1496] 10.222.0.15 () {58 vars in 1153 bytes} [Mon Aug 22 16:07:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 22 16:07:02.070712 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3422a473-4c3a-43d8-bb04-476a8c75e0ae None None] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 16:07:02.080652 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3422a473-4c3a-43d8-bb04-476a8c75e0ae None None] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 16:07:02.085766 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7325d09-8103-4c2a-b1d3-62cebf857fae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a98b1e4431940ab8d1fc95df2858a6c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 16:07:02.086363 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7325d09-8103-4c2a-b1d3-62cebf857fae admin admin] REQUEST_METHOD: `POST` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:07:02.086647 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7325d09-8103-4c2a-b1d3-62cebf857fae admin admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:07:02.086919 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7325d09-8103-4c2a-b1d3-62cebf857fae admin admin] PATH_INFO: `/v3/projects` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:07:02.087591 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7325d09-8103-4c2a-b1d3-62cebf857fae admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 16:07:02.089468 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: WARNING py.warnings [None req-e7325d09-8103-4c2a-b1d3-62cebf857fae admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 22 16:07:02.089468 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: warnings.warn(msg) Aug 22 16:07:02.089468 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: Aug 22 16:07:02.090679 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7325d09-8103-4c2a-b1d3-62cebf857fae admin admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 16:07:02.176087 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 759/1497] 10.222.0.15 () {60 vars in 1153 bytes} [Mon Aug 22 16:07:02 2022] POST /identity/v3/projects => generated 379 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 22 16:07:02.180398 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da091b27-1f58-47ef-bca3-84581b960f83 admin admin] Authenticating user token {{(pid=81767) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 16:07:02.191133 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da091b27-1f58-47ef-bca3-84581b960f83 admin admin] Validating token access rules against request {{(pid=81767) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 16:07:02.241611 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG dbcounter [-] [81767] Writing DB stats keystone:SELECT=100 {{(pid=81767) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 16:07:02.249042 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1daf1362-32a9-40f2-b529-fb9d71fe1306 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a98b1e4431940ab8d1fc95df2858a6c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81767) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 16:07:02.251222 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1daf1362-32a9-40f2-b529-fb9d71fe1306 admin admin] REQUEST_METHOD: `POST` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:07:02.251222 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1daf1362-32a9-40f2-b529-fb9d71fe1306 admin admin] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:07:02.251222 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1daf1362-32a9-40f2-b529-fb9d71fe1306 admin admin] PATH_INFO: `/v3/users` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:07:02.251222 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1daf1362-32a9-40f2-b529-fb9d71fe1306 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 16:07:02.251891 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: WARNING py.warnings [None req-1daf1362-32a9-40f2-b529-fb9d71fe1306 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 22 16:07:02.251891 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: warnings.warn(msg) Aug 22 16:07:02.251891 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: Aug 22 16:07:02.253090 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1daf1362-32a9-40f2-b529-fb9d71fe1306 admin admin] RBAC: Authorization granted {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 16:07:02.318059 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 739/1498] 10.222.0.15 () {60 vars in 1144 bytes} [Mon Aug 22 16:07:02 2022] POST /identity/v3/users => generated 290 bytes in 138 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 22 16:07:02.322824 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7325d09-8103-4c2a-b1d3-62cebf857fae admin admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 16:07:02.334492 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7325d09-8103-4c2a-b1d3-62cebf857fae admin admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 16:07:02.341167 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d962e593-e02b-4f82-9c9f-aa85b9c54ab2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a98b1e4431940ab8d1fc95df2858a6c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 16:07:02.341775 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d962e593-e02b-4f82-9c9f-aa85b9c54ab2 admin admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:07:02.342054 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d962e593-e02b-4f82-9c9f-aa85b9c54ab2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:07:02.342327 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d962e593-e02b-4f82-9c9f-aa85b9c54ab2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:07:02.342974 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d962e593-e02b-4f82-9c9f-aa85b9c54ab2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 16:07:02.344130 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: WARNING py.warnings [None req-d962e593-e02b-4f82-9c9f-aa85b9c54ab2 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Aug 22 16:07:02.344130 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: warnings.warn(msg) Aug 22 16:07:02.344130 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: Aug 22 16:07:02.345021 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d962e593-e02b-4f82-9c9f-aa85b9c54ab2 admin admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 16:07:02.359074 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 760/1499] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Aug 22 16:07:02 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 22 16:07:02.363098 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1daf1362-32a9-40f2-b529-fb9d71fe1306 admin admin] Authenticating user token {{(pid=81767) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 16:07:02.373810 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1daf1362-32a9-40f2-b529-fb9d71fe1306 admin admin] Validating token access rules against request {{(pid=81767) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 16:07:02.378421 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd1e0962-ed58-4830-8788-10b17713c4d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a98b1e4431940ab8d1fc95df2858a6c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81767) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 16:07:02.379013 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd1e0962-ed58-4830-8788-10b17713c4d3 admin admin] REQUEST_METHOD: `GET` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:07:02.379316 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd1e0962-ed58-4830-8788-10b17713c4d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:07:02.379592 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd1e0962-ed58-4830-8788-10b17713c4d3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:07:02.380239 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd1e0962-ed58-4830-8788-10b17713c4d3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 16:07:02.381238 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: WARNING py.warnings [None req-fd1e0962-ed58-4830-8788-10b17713c4d3 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Aug 22 16:07:02.381238 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: warnings.warn(msg) Aug 22 16:07:02.381238 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: Aug 22 16:07:02.382114 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd1e0962-ed58-4830-8788-10b17713c4d3 admin admin] RBAC: Authorization granted {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 16:07:02.396130 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 740/1500] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Aug 22 16:07:02 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 22 16:07:02.400294 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d962e593-e02b-4f82-9c9f-aa85b9c54ab2 admin admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 16:07:02.410775 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d962e593-e02b-4f82-9c9f-aa85b9c54ab2 admin admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 16:07:02.415667 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89be0f5a-1e64-4a98-a93f-cc11b9ac6a63 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a98b1e4431940ab8d1fc95df2858a6c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 16:07:02.416333 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89be0f5a-1e64-4a98-a93f-cc11b9ac6a63 admin admin] REQUEST_METHOD: `PUT` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:07:02.416600 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89be0f5a-1e64-4a98-a93f-cc11b9ac6a63 admin admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:07:02.416872 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89be0f5a-1e64-4a98-a93f-cc11b9ac6a63 admin admin] PATH_INFO: `/v3/projects/adff9a865660477c8906597c6270843a/users/41e8bbd1ca81439797745462ed60d47e/roles/e6fc1ee3fb974125ba9b4fe55b12b45e` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:07:02.432563 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89be0f5a-1e64-4a98-a93f-cc11b9ac6a63 admin admin] RBAC: Authorizing `identity:create_grant(project_id=adff9a865660477c8906597c6270843a, user_id=41e8bbd1ca81439797745462ed60d47e, role_id=e6fc1ee3fb974125ba9b4fe55b12b45e)` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 16:07:02.433601 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: WARNING py.warnings [None req-89be0f5a-1e64-4a98-a93f-cc11b9ac6a63 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 22 16:07:02.433601 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: warnings.warn(msg) Aug 22 16:07:02.433601 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: Aug 22 16:07:02.436082 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89be0f5a-1e64-4a98-a93f-cc11b9ac6a63 admin admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 16:07:02.534246 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 761/1501] 10.222.0.15 () {60 vars in 1483 bytes} [Mon Aug 22 16:07:02 2022] PUT /identity/v3/projects/adff9a865660477c8906597c6270843a/users/41e8bbd1ca81439797745462ed60d47e/roles/e6fc1ee3fb974125ba9b4fe55b12b45e => generated 0 bytes in 134 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 16:07:02.547161 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f0010d4-6a08-407b-bd94-5cc6fcdee710 None None] REQUEST_METHOD: `POST` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:07:02.547478 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f0010d4-6a08-407b-bd94-5cc6fcdee710 None None] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:07:02.547758 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f0010d4-6a08-407b-bd94-5cc6fcdee710 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:07:02.577524 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.auth.core [None req-5f0010d4-6a08-407b-bd94-5cc6fcdee710 None None] MFA Rules not processed for user `41e8bbd1ca81439797745462ed60d47e`. Rule list: `[]` (Enabled: `True`). {{(pid=81767) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 16:07:02.578767 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.fernet_utils [None req-5f0010d4-6a08-407b-bd94-5cc6fcdee710 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81767) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 16:07:02.645099 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 741/1502] 10.222.0.15 () {58 vars in 958 bytes} [Mon Aug 22 16:07:02 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 16:07:02.658422 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e98d520f-5431-4172-aa14-64a984ac0e61 None None] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:07:02.658703 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e98d520f-5431-4172-aa14-64a984ac0e61 None None] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:07:02.658973 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e98d520f-5431-4172-aa14-64a984ac0e61 None None] PATH_INFO: `/` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:07:02.660333 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 762/1503] 10.222.0.15 () {54 vars in 964 bytes} [Mon Aug 22 16:07:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 16:07:02.665163 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79ab404b-7f45-4141-a44c-86273f33a298 None None] REQUEST_METHOD: `POST` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:07:02.665457 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79ab404b-7f45-4141-a44c-86273f33a298 None None] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:07:02.665730 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79ab404b-7f45-4141-a44c-86273f33a298 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:07:02.729389 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.auth.core [None req-79ab404b-7f45-4141-a44c-86273f33a298 None None] MFA Rules not processed for user `79706d480f864998b9f8594be407ceab`. Rule list: `[]` (Enabled: `True`). {{(pid=81767) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 16:07:02.730641 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.fernet_utils [None req-79ab404b-7f45-4141-a44c-86273f33a298 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81767) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 16:07:02.813038 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 742/1504] 10.222.0.15 () {58 vars in 1061 bytes} [Mon Aug 22 16:07:02 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 16:07:02.820817 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e98d520f-5431-4172-aa14-64a984ac0e61 None None] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 16:07:02.832642 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e98d520f-5431-4172-aa14-64a984ac0e61 None None] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 16:07:02.838289 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf003b1f-42e4-41ea-aa86-18ff39099851 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79706d480f864998b9f8594be407ceab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4ee0413f9ef4d4fb1f95a660babe090', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 16:07:02.838905 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf003b1f-42e4-41ea-aa86-18ff39099851 service cinder] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:07:02.839181 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf003b1f-42e4-41ea-aa86-18ff39099851 service cinder] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:07:02.839486 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf003b1f-42e4-41ea-aa86-18ff39099851 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:07:02.848579 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf003b1f-42e4-41ea-aa86-18ff39099851 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 16:07:02.850083 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf003b1f-42e4-41ea-aa86-18ff39099851 service cinder] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 16:07:02.859788 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 763/1505] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Aug 22 16:07:02 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 16:07:02.906673 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1ac0f6a-fcbf-4a2f-afa3-c8e22b05beb8 None None] REQUEST_METHOD: `GET` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:07:02.906958 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1ac0f6a-fcbf-4a2f-afa3-c8e22b05beb8 None None] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:07:02.907230 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1ac0f6a-fcbf-4a2f-afa3-c8e22b05beb8 None None] PATH_INFO: `/` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:07:02.908553 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 743/1506] 10.222.0.15 () {52 vars in 830 bytes} [Mon Aug 22 16:07:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 16:07:02.912324 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf003b1f-42e4-41ea-aa86-18ff39099851 service cinder] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 16:07:02.980916 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf003b1f-42e4-41ea-aa86-18ff39099851 service cinder] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 16:07:02.985780 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad12d482-c988-49b3-9497-d301cdfddfb4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a98b1e4431940ab8d1fc95df2858a6c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 16:07:02.986421 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad12d482-c988-49b3-9497-d301cdfddfb4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:07:02.986701 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad12d482-c988-49b3-9497-d301cdfddfb4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:07:02.986976 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad12d482-c988-49b3-9497-d301cdfddfb4 admin admin] PATH_INFO: `/v3/users/41e8bbd1ca81439797745462ed60d47e` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:07:02.988237 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad12d482-c988-49b3-9497-d301cdfddfb4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=41e8bbd1ca81439797745462ed60d47e)` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 16:07:02.989179 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: WARNING py.warnings [None req-ad12d482-c988-49b3-9497-d301cdfddfb4 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 22 16:07:02.989179 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: warnings.warn(msg) Aug 22 16:07:02.989179 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: Aug 22 16:07:02.990381 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad12d482-c988-49b3-9497-d301cdfddfb4 admin admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 16:07:03.167493 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-ad12d482-c988-49b3-9497-d301cdfddfb4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '41e8bbd1ca81439797745462ed60d47e'} {{(pid=81768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 16:07:03.168174 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-ad12d482-c988-49b3-9497-d301cdfddfb4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '41e8bbd1ca81439797745462ed60d47e'} {{(pid=81768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 16:07:03.168788 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-ad12d482-c988-49b3-9497-d301cdfddfb4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '41e8bbd1ca81439797745462ed60d47e'} {{(pid=81768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 16:07:03.185495 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-ad12d482-c988-49b3-9497-d301cdfddfb4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '41e8bbd1ca81439797745462ed60d47e'} {{(pid=81768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 16:07:03.316745 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-ad12d482-c988-49b3-9497-d301cdfddfb4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '41e8bbd1ca81439797745462ed60d47e'} {{(pid=81768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 16:07:03.340462 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=88,keystone:INSERT=3,keystone:DELETE=9 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 16:07:03.349482 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 764/1507] 10.222.0.15 () {58 vars in 1224 bytes} [Mon Aug 22 16:07:02 2022] DELETE /identity/v3/users/41e8bbd1ca81439797745462ed60d47e => generated 0 bytes in 437 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 16:07:03.353734 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e188304-f04b-44ed-8a96-e885f5ddd49e None None] REQUEST_METHOD: `POST` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:07:03.354020 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e188304-f04b-44ed-8a96-e885f5ddd49e None None] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:07:03.354295 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e188304-f04b-44ed-8a96-e885f5ddd49e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:07:03.374034 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.auth.core [None req-2e188304-f04b-44ed-8a96-e885f5ddd49e None None] MFA Rules not processed for user `6b2199ae0fb0480aa917ada391f5ec54`. Rule list: `[]` (Enabled: `True`). {{(pid=81767) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 16:07:03.375315 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.fernet_utils [None req-2e188304-f04b-44ed-8a96-e885f5ddd49e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81767) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 16:07:03.385037 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG dbcounter [-] [81767] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81767) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 16:07:03.435528 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 744/1508] 10.222.0.15 () {58 vars in 958 bytes} [Mon Aug 22 16:07:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 16:07:03.445186 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad12d482-c988-49b3-9497-d301cdfddfb4 admin admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 16:07:03.446596 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-ad12d482-c988-49b3-9497-d301cdfddfb4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 16:07:03.527068 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad12d482-c988-49b3-9497-d301cdfddfb4 admin admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 16:07:03.531514 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87eeb802-a300-4763-a954-6d7e4a867ae8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '234687adad594f1aa45775a565ffbdc7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4ee0413f9ef4d4fb1f95a660babe090', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 16:07:03.532121 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87eeb802-a300-4763-a954-6d7e4a867ae8 service neutron] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:07:03.532430 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87eeb802-a300-4763-a954-6d7e4a867ae8 service neutron] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:07:03.532695 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87eeb802-a300-4763-a954-6d7e4a867ae8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:07:03.541873 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87eeb802-a300-4763-a954-6d7e4a867ae8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 16:07:03.543325 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87eeb802-a300-4763-a954-6d7e4a867ae8 service neutron] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 16:07:03.552668 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 765/1509] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Aug 22 16:07:03 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 16:07:04.048993 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e188304-f04b-44ed-8a96-e885f5ddd49e None None] Authenticating user token {{(pid=81767) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 16:07:04.050503 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.fernet_utils [None req-2e188304-f04b-44ed-8a96-e885f5ddd49e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81767) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 16:07:04.062886 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e188304-f04b-44ed-8a96-e885f5ddd49e None None] Validating token access rules against request {{(pid=81767) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 16:07:04.068394 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84253d38-eed4-46ce-acb4-6a73f7a0b9b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a98b1e4431940ab8d1fc95df2858a6c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81767) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 16:07:04.069126 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84253d38-eed4-46ce-acb4-6a73f7a0b9b9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:07:04.070047 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84253d38-eed4-46ce-acb4-6a73f7a0b9b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:07:04.070047 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84253d38-eed4-46ce-acb4-6a73f7a0b9b9 admin admin] PATH_INFO: `/v3/projects/adff9a865660477c8906597c6270843a` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:07:04.071068 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84253d38-eed4-46ce-acb4-6a73f7a0b9b9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=adff9a865660477c8906597c6270843a)` {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 16:07:04.072132 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: WARNING py.warnings [None req-84253d38-eed4-46ce-acb4-6a73f7a0b9b9 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 22 16:07:04.072132 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: warnings.warn(msg) Aug 22 16:07:04.072132 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: Aug 22 16:07:04.073595 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84253d38-eed4-46ce-acb4-6a73f7a0b9b9 admin admin] RBAC: Authorization granted {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 16:07:04.243757 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.notifications [None req-84253d38-eed4-46ce-acb4-6a73f7a0b9b9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'adff9a865660477c8906597c6270843a', 'request_id': 'req-84253d38-eed4-46ce-acb4-6a73f7a0b9b9'} {{(pid=81767) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 16:07:04.249939 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.notifications [None req-84253d38-eed4-46ce-acb4-6a73f7a0b9b9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'adff9a865660477c8906597c6270843a', 'request_id': 'req-84253d38-eed4-46ce-acb4-6a73f7a0b9b9'} {{(pid=81767) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 16:07:04.322303 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.notifications [None req-84253d38-eed4-46ce-acb4-6a73f7a0b9b9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'adff9a865660477c8906597c6270843a', 'request_id': 'req-84253d38-eed4-46ce-acb4-6a73f7a0b9b9'} {{(pid=81767) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 16:07:04.343461 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.notifications [None req-84253d38-eed4-46ce-acb4-6a73f7a0b9b9 admin admin] The token cache is being invalidate because project adff9a865660477c8906597c6270843a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81767) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 22 16:07:04.343747 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.notifications [None req-84253d38-eed4-46ce-acb4-6a73f7a0b9b9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81767) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 16:07:04.344436 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.notifications [None req-84253d38-eed4-46ce-acb4-6a73f7a0b9b9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81767) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 16:07:04.346005 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 745/1510] 10.222.0.15 () {58 vars in 1233 bytes} [Mon Aug 22 16:07:04 2022] DELETE /identity/v3/projects/adff9a865660477c8906597c6270843a => generated 0 bytes in 298 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 16:07:13.548514 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:DELETE=1,keystone:SELECT=37 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 16:07:14.339486 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG dbcounter [-] [81767] Writing DB stats keystone:SELECT=47,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81767) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 16:08:39.373502 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cc15619-f13a-494a-9cd8-8ce0427366cc None None] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:08:39.374068 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cc15619-f13a-494a-9cd8-8ce0427366cc None None] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:08:39.374068 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cc15619-f13a-494a-9cd8-8ce0427366cc None None] PATH_INFO: `/` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:08:39.375417 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 766/1511] 127.0.0.1 () {54 vars in 901 bytes} [Mon Aug 22 16:08:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 16:08:39.380513 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-913ca5bb-f67f-4c95-8743-62522cf39bd8 None None] REQUEST_METHOD: `POST` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:08:39.380945 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-913ca5bb-f67f-4c95-8743-62522cf39bd8 None None] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:08:39.381047 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-913ca5bb-f67f-4c95-8743-62522cf39bd8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:08:39.401028 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.auth.core [None req-913ca5bb-f67f-4c95-8743-62522cf39bd8 None None] MFA Rules not processed for user `6b2199ae0fb0480aa917ada391f5ec54`. Rule list: `[]` (Enabled: `True`). {{(pid=81767) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 16:08:39.402795 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.fernet_utils [None req-913ca5bb-f67f-4c95-8743-62522cf39bd8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81767) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 16:08:39.464771 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 746/1512] 127.0.0.1 () {58 vars in 998 bytes} [Mon Aug 22 16:08:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 16:08:39.624248 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cc15619-f13a-494a-9cd8-8ce0427366cc None None] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 16:08:39.632973 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-3cc15619-f13a-494a-9cd8-8ce0427366cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 16:08:39.730820 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cc15619-f13a-494a-9cd8-8ce0427366cc None None] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 16:08:39.736398 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce1c1f77-806f-47fe-ab6b-87bde060260a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4ef569e41fb46d0b721bb1c7663616f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4ee0413f9ef4d4fb1f95a660babe090', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 16:08:39.737013 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce1c1f77-806f-47fe-ab6b-87bde060260a service nova] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:08:39.737368 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce1c1f77-806f-47fe-ab6b-87bde060260a service nova] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:08:39.737568 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce1c1f77-806f-47fe-ab6b-87bde060260a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:08:39.746494 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce1c1f77-806f-47fe-ab6b-87bde060260a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 16:08:39.748138 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce1c1f77-806f-47fe-ab6b-87bde060260a service nova] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 16:08:39.757610 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 767/1513] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Aug 22 16:08:39 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 134 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 16:08:40.607337 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4cc74fa-c2f8-40e0-8660-9515963698b8 None None] REQUEST_METHOD: `GET` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:08:40.607980 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4cc74fa-c2f8-40e0-8660-9515963698b8 None None] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:08:40.607980 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4cc74fa-c2f8-40e0-8660-9515963698b8 None None] PATH_INFO: `/` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:08:40.609119 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 747/1514] 127.0.0.1 () {54 vars in 901 bytes} [Mon Aug 22 16:08:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 22 16:08:40.614144 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1e087a5-011e-41b0-acad-10c8b651b61b None None] REQUEST_METHOD: `POST` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:08:40.614561 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1e087a5-011e-41b0-acad-10c8b651b61b None None] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:08:40.614672 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1e087a5-011e-41b0-acad-10c8b651b61b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:08:40.634207 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.auth.core [None req-d1e087a5-011e-41b0-acad-10c8b651b61b None None] MFA Rules not processed for user `6b2199ae0fb0480aa917ada391f5ec54`. Rule list: `[]` (Enabled: `True`). {{(pid=81768) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 16:08:40.635445 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-d1e087a5-011e-41b0-acad-10c8b651b61b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 16:08:40.643958 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 768/1515] 127.0.0.1 () {58 vars in 998 bytes} [Mon Aug 22 16:08:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 16:08:40.797333 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4cc74fa-c2f8-40e0-8660-9515963698b8 None None] Authenticating user token {{(pid=81767) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 16:08:40.812606 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4cc74fa-c2f8-40e0-8660-9515963698b8 None None] Validating token access rules against request {{(pid=81767) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 16:08:40.818289 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-831138e4-ec0c-427f-879f-412d9fbcdda2 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4ef569e41fb46d0b721bb1c7663616f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4ee0413f9ef4d4fb1f95a660babe090', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81767) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 16:08:40.818902 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-831138e4-ec0c-427f-879f-412d9fbcdda2 service nova] REQUEST_METHOD: `GET` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:08:40.819179 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-831138e4-ec0c-427f-879f-412d9fbcdda2 service nova] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:08:40.819480 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-831138e4-ec0c-427f-879f-412d9fbcdda2 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:08:40.828429 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-831138e4-ec0c-427f-879f-412d9fbcdda2 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 16:08:40.830008 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-831138e4-ec0c-427f-879f-412d9fbcdda2 service nova] RBAC: Authorization granted {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 16:08:40.840866 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 748/1516] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Aug 22 16:08:40 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 16:08:42.285336 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb165f04-7097-4246-9007-2b3ec931e9dc None None] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:08:42.285900 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb165f04-7097-4246-9007-2b3ec931e9dc None None] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:08:42.285900 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb165f04-7097-4246-9007-2b3ec931e9dc None None] PATH_INFO: `/v3` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:08:42.287155 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 769/1517] 10.222.0.73 () {54 vars in 925 bytes} [Mon Aug 22 16:08:42 2022] GET /identity/v3 => generated 255 bytes in 3 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 16:08:42.293966 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1f8cd0c-6daf-4774-ab8f-b842c50f984c None None] REQUEST_METHOD: `POST` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:08:42.294379 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1f8cd0c-6daf-4774-ab8f-b842c50f984c None None] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:08:42.294500 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1f8cd0c-6daf-4774-ab8f-b842c50f984c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:08:42.346001 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.auth.core [None req-d1f8cd0c-6daf-4774-ab8f-b842c50f984c None None] MFA Rules not processed for user `08159bf8edda4246bb463f7b11b0894b`. Rule list: `[]` (Enabled: `True`). {{(pid=81767) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 16:08:42.347220 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.fernet_utils [None req-d1f8cd0c-6daf-4774-ab8f-b842c50f984c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81767) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 16:08:42.418640 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 749/1518] 10.222.0.73 () {58 vars in 1015 bytes} [Mon Aug 22 16:08:42 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 16:08:46.581129 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df4bb42-d7d6-448d-b541-a4a19884671f None None] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:08:46.581703 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df4bb42-d7d6-448d-b541-a4a19884671f None None] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:08:46.581703 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df4bb42-d7d6-448d-b541-a4a19884671f None None] PATH_INFO: `/` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:08:46.583242 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 770/1519] 10.222.0.15 () {54 vars in 961 bytes} [Mon Aug 22 16:08:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 16:08:46.589121 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ac2431b-0030-4bf9-b9b1-dbdff1f724f4 None None] REQUEST_METHOD: `POST` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:08:46.589583 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ac2431b-0030-4bf9-b9b1-dbdff1f724f4 None None] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:08:46.589739 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ac2431b-0030-4bf9-b9b1-dbdff1f724f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:08:46.609247 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.auth.core [None req-7ac2431b-0030-4bf9-b9b1-dbdff1f724f4 None None] MFA Rules not processed for user `08159bf8edda4246bb463f7b11b0894b`. Rule list: `[]` (Enabled: `True`). {{(pid=81767) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 16:08:46.610442 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.fernet_utils [None req-7ac2431b-0030-4bf9-b9b1-dbdff1f724f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81767) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 16:08:46.618831 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 750/1520] 10.222.0.15 () {58 vars in 1058 bytes} [Mon Aug 22 16:08:46 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 22 16:08:46.626423 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2df4bb42-d7d6-448d-b541-a4a19884671f None None] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 16:08:46.640236 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2df4bb42-d7d6-448d-b541-a4a19884671f None None] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 16:08:46.645024 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7134a22-a3e1-46d2-88d3-1d175b7f0ec8 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '08159bf8edda4246bb463f7b11b0894b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4ee0413f9ef4d4fb1f95a660babe090', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 16:08:46.645605 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7134a22-a3e1-46d2-88d3-1d175b7f0ec8 service placement] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:08:46.645877 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7134a22-a3e1-46d2-88d3-1d175b7f0ec8 service placement] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:08:46.646144 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7134a22-a3e1-46d2-88d3-1d175b7f0ec8 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:08:46.655772 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7134a22-a3e1-46d2-88d3-1d175b7f0ec8 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 16:08:46.657361 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7134a22-a3e1-46d2-88d3-1d175b7f0ec8 service placement] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 16:08:46.667403 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 771/1521] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Aug 22 16:08:46 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 16:08:52.069597 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53a8033f-be2b-480e-956a-1dcdbc9f433d None None] REQUEST_METHOD: `GET` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:08:52.070141 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53a8033f-be2b-480e-956a-1dcdbc9f433d None None] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:08:52.070258 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53a8033f-be2b-480e-956a-1dcdbc9f433d None None] PATH_INFO: `/` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:08:52.071621 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 751/1522] 127.0.0.1 () {54 vars in 901 bytes} [Mon Aug 22 16:08:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 16:08:52.077031 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cfe8659-dcbd-408a-90f2-bf0820cdb492 None None] REQUEST_METHOD: `POST` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:08:52.077507 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cfe8659-dcbd-408a-90f2-bf0820cdb492 None None] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:08:52.077634 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cfe8659-dcbd-408a-90f2-bf0820cdb492 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:08:52.097318 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.auth.core [None req-8cfe8659-dcbd-408a-90f2-bf0820cdb492 None None] MFA Rules not processed for user `6b2199ae0fb0480aa917ada391f5ec54`. Rule list: `[]` (Enabled: `True`). {{(pid=81768) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 16:08:52.098508 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-8cfe8659-dcbd-408a-90f2-bf0820cdb492 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 16:08:52.107508 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 772/1523] 127.0.0.1 () {58 vars in 998 bytes} [Mon Aug 22 16:08:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 16:08:52.262629 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53a8033f-be2b-480e-956a-1dcdbc9f433d None None] Authenticating user token {{(pid=81767) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 16:08:52.277636 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53a8033f-be2b-480e-956a-1dcdbc9f433d None None] Validating token access rules against request {{(pid=81767) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 16:08:52.283097 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe5ac1ba-d51a-4974-a0ea-e51da5ac973f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4ef569e41fb46d0b721bb1c7663616f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4ee0413f9ef4d4fb1f95a660babe090', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81767) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 16:08:52.283727 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe5ac1ba-d51a-4974-a0ea-e51da5ac973f service nova] REQUEST_METHOD: `GET` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:08:52.284020 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe5ac1ba-d51a-4974-a0ea-e51da5ac973f service nova] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:08:52.284276 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe5ac1ba-d51a-4974-a0ea-e51da5ac973f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:08:52.292580 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe5ac1ba-d51a-4974-a0ea-e51da5ac973f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 16:08:52.294477 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe5ac1ba-d51a-4974-a0ea-e51da5ac973f service nova] RBAC: Authorization granted {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 16:08:52.304866 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 752/1524] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Aug 22 16:08:52 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 16:08:53.131962 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b079786d-1130-4997-9504-94184ab5310e None None] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:08:53.132553 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b079786d-1130-4997-9504-94184ab5310e None None] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:08:53.132553 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b079786d-1130-4997-9504-94184ab5310e None None] PATH_INFO: `/` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:08:53.134022 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 773/1525] 127.0.0.1 () {54 vars in 901 bytes} [Mon Aug 22 16:08:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 16:08:53.139442 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb5ab1d-4913-4a64-bf40-22a8e74a72eb None None] REQUEST_METHOD: `POST` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:08:53.139903 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb5ab1d-4913-4a64-bf40-22a8e74a72eb None None] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:08:53.140003 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb5ab1d-4913-4a64-bf40-22a8e74a72eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:08:53.160017 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.auth.core [None req-7bb5ab1d-4913-4a64-bf40-22a8e74a72eb None None] MFA Rules not processed for user `6b2199ae0fb0480aa917ada391f5ec54`. Rule list: `[]` (Enabled: `True`). {{(pid=81767) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 16:08:53.161238 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.fernet_utils [None req-7bb5ab1d-4913-4a64-bf40-22a8e74a72eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81767) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 16:08:53.169067 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 753/1526] 127.0.0.1 () {58 vars in 998 bytes} [Mon Aug 22 16:08:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 16:08:53.320323 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b079786d-1130-4997-9504-94184ab5310e None None] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 16:08:53.335227 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b079786d-1130-4997-9504-94184ab5310e None None] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 16:08:53.341393 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a7e533e-03c3-45f4-a051-845daeb94064 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4ef569e41fb46d0b721bb1c7663616f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4ee0413f9ef4d4fb1f95a660babe090', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 16:08:53.341989 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a7e533e-03c3-45f4-a051-845daeb94064 service nova] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:08:53.342263 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a7e533e-03c3-45f4-a051-845daeb94064 service nova] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:08:53.342531 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a7e533e-03c3-45f4-a051-845daeb94064 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:08:53.350933 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a7e533e-03c3-45f4-a051-845daeb94064 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 16:08:53.352545 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a7e533e-03c3-45f4-a051-845daeb94064 service nova] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 16:08:53.362217 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 774/1527] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Aug 22 16:08:53 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 16:08:57.217219 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d530f9da-d79f-4f0f-bc76-4c10963b35f0 None None] REQUEST_METHOD: `GET` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:08:57.217814 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d530f9da-d79f-4f0f-bc76-4c10963b35f0 None None] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:08:57.217814 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d530f9da-d79f-4f0f-bc76-4c10963b35f0 None None] PATH_INFO: `/` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:08:57.219085 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 754/1528] 127.0.0.1 () {54 vars in 917 bytes} [Mon Aug 22 16:08:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 16:08:57.224542 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42bf9520-ea81-42ee-b09a-e25268118567 None None] REQUEST_METHOD: `POST` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:08:57.225007 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42bf9520-ea81-42ee-b09a-e25268118567 None None] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:08:57.225104 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42bf9520-ea81-42ee-b09a-e25268118567 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:08:57.245595 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.auth.core [None req-42bf9520-ea81-42ee-b09a-e25268118567 None None] MFA Rules not processed for user `6b2199ae0fb0480aa917ada391f5ec54`. Rule list: `[]` (Enabled: `True`). {{(pid=81768) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 16:08:57.246833 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-42bf9520-ea81-42ee-b09a-e25268118567 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 16:08:57.254732 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 775/1529] 127.0.0.1 () {58 vars in 1014 bytes} [Mon Aug 22 16:08:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 16:08:57.541111 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d530f9da-d79f-4f0f-bc76-4c10963b35f0 None None] Authenticating user token {{(pid=81767) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 16:08:57.542843 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.fernet_utils [None req-d530f9da-d79f-4f0f-bc76-4c10963b35f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81767) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 16:08:57.572190 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG dbcounter [-] [81767] Writing DB stats keystone:SELECT=100 {{(pid=81767) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 16:08:57.602513 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d530f9da-d79f-4f0f-bc76-4c10963b35f0 None None] Validating token access rules against request {{(pid=81767) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 16:08:57.606654 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-458feb7d-c2a4-4c95-9b66-38aa085db60d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '234687adad594f1aa45775a565ffbdc7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4ee0413f9ef4d4fb1f95a660babe090', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81767) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 16:08:57.607263 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-458feb7d-c2a4-4c95-9b66-38aa085db60d service neutron] REQUEST_METHOD: `GET` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:08:57.607575 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-458feb7d-c2a4-4c95-9b66-38aa085db60d service neutron] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:08:57.607850 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-458feb7d-c2a4-4c95-9b66-38aa085db60d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:08:57.617059 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-458feb7d-c2a4-4c95-9b66-38aa085db60d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 16:08:57.618482 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-458feb7d-c2a4-4c95-9b66-38aa085db60d service neutron] RBAC: Authorization granted {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 16:08:57.628616 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 755/1530] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Aug 22 16:08:57 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 16:08:59.554832 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-509a91a6-18f5-486e-942c-2db601504576 None None] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:08:59.555403 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-509a91a6-18f5-486e-942c-2db601504576 None None] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:08:59.555403 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-509a91a6-18f5-486e-942c-2db601504576 None None] PATH_INFO: `/` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:08:59.556921 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 776/1531] 127.0.0.1 () {54 vars in 917 bytes} [Mon Aug 22 16:08:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 16:08:59.562734 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77fc15f8-3411-45fe-afb5-8ce2988ae717 None None] REQUEST_METHOD: `POST` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:08:59.563250 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77fc15f8-3411-45fe-afb5-8ce2988ae717 None None] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:08:59.563381 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77fc15f8-3411-45fe-afb5-8ce2988ae717 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:08:59.585803 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.auth.core [None req-77fc15f8-3411-45fe-afb5-8ce2988ae717 None None] MFA Rules not processed for user `6b2199ae0fb0480aa917ada391f5ec54`. Rule list: `[]` (Enabled: `True`). {{(pid=81767) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 16:08:59.590697 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.fernet_utils [None req-77fc15f8-3411-45fe-afb5-8ce2988ae717 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81767) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 16:08:59.595654 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 756/1532] 127.0.0.1 () {58 vars in 1014 bytes} [Mon Aug 22 16:08:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 16:08:59.880364 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-509a91a6-18f5-486e-942c-2db601504576 None None] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 16:08:59.892055 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-509a91a6-18f5-486e-942c-2db601504576 None None] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 16:08:59.896054 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2473a478-45e7-46e8-a299-fb731ba3324e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '234687adad594f1aa45775a565ffbdc7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4ee0413f9ef4d4fb1f95a660babe090', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 16:08:59.896655 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2473a478-45e7-46e8-a299-fb731ba3324e service neutron] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:08:59.896950 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2473a478-45e7-46e8-a299-fb731ba3324e service neutron] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:08:59.897201 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2473a478-45e7-46e8-a299-fb731ba3324e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:08:59.906268 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2473a478-45e7-46e8-a299-fb731ba3324e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 16:08:59.907739 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2473a478-45e7-46e8-a299-fb731ba3324e service neutron] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 16:08:59.917589 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 777/1533] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Aug 22 16:08:59 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 16:09:03.455943 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e0305e7-fc75-478a-b976-446a4393cacc None None] REQUEST_METHOD: `GET` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:09:03.456491 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e0305e7-fc75-478a-b976-446a4393cacc None None] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:09:03.456491 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e0305e7-fc75-478a-b976-446a4393cacc None None] PATH_INFO: `/` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:09:03.458086 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 757/1534] 127.0.0.1 () {54 vars in 917 bytes} [Mon Aug 22 16:09:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 16:09:03.463508 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea262517-99c0-48f2-a116-cb1dd206b538 None None] REQUEST_METHOD: `POST` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:09:03.463926 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea262517-99c0-48f2-a116-cb1dd206b538 None None] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:09:03.464052 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea262517-99c0-48f2-a116-cb1dd206b538 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:09:03.483469 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.auth.core [None req-ea262517-99c0-48f2-a116-cb1dd206b538 None None] MFA Rules not processed for user `6b2199ae0fb0480aa917ada391f5ec54`. Rule list: `[]` (Enabled: `True`). {{(pid=81768) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 16:09:03.484642 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-ea262517-99c0-48f2-a116-cb1dd206b538 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 16:09:03.492308 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 778/1535] 127.0.0.1 () {58 vars in 1014 bytes} [Mon Aug 22 16:09:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 16:09:04.053041 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e0305e7-fc75-478a-b976-446a4393cacc None None] Authenticating user token {{(pid=81767) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 16:09:04.054851 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.fernet_utils [None req-5e0305e7-fc75-478a-b976-446a4393cacc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81767) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 16:09:04.116353 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e0305e7-fc75-478a-b976-446a4393cacc None None] Validating token access rules against request {{(pid=81767) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 16:09:04.120238 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1fc4d0d-bba8-4f2d-af00-71cb8a648fdd service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79706d480f864998b9f8594be407ceab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4ee0413f9ef4d4fb1f95a660babe090', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81767) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 16:09:04.120888 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1fc4d0d-bba8-4f2d-af00-71cb8a648fdd service cinder] REQUEST_METHOD: `GET` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:09:04.121182 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1fc4d0d-bba8-4f2d-af00-71cb8a648fdd service cinder] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:09:04.121469 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1fc4d0d-bba8-4f2d-af00-71cb8a648fdd service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:09:04.130963 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1fc4d0d-bba8-4f2d-af00-71cb8a648fdd service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 16:09:04.135854 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1fc4d0d-bba8-4f2d-af00-71cb8a648fdd service cinder] RBAC: Authorization granted {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 16:09:04.142690 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 758/1536] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Aug 22 16:09:04 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 16:09:06.110757 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0764955d-0cc2-4b2b-8721-f44e95c8f637 None None] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:09:06.111356 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0764955d-0cc2-4b2b-8721-f44e95c8f637 None None] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:09:06.111356 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0764955d-0cc2-4b2b-8721-f44e95c8f637 None None] PATH_INFO: `/` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:09:06.112707 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 779/1537] 127.0.0.1 () {54 vars in 917 bytes} [Mon Aug 22 16:09:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 16:09:06.118252 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4df61882-06b4-419b-a4d3-547291db0595 None None] REQUEST_METHOD: `POST` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:09:06.119319 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4df61882-06b4-419b-a4d3-547291db0595 None None] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:09:06.119479 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4df61882-06b4-419b-a4d3-547291db0595 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:09:06.138840 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.auth.core [None req-4df61882-06b4-419b-a4d3-547291db0595 None None] MFA Rules not processed for user `6b2199ae0fb0480aa917ada391f5ec54`. Rule list: `[]` (Enabled: `True`). {{(pid=81767) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 16:09:06.140082 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.fernet_utils [None req-4df61882-06b4-419b-a4d3-547291db0595 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81767) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 16:09:06.148043 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 759/1538] 127.0.0.1 () {58 vars in 1014 bytes} [Mon Aug 22 16:09:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 16:09:06.710591 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0764955d-0cc2-4b2b-8721-f44e95c8f637 None None] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 16:09:06.712290 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-0764955d-0cc2-4b2b-8721-f44e95c8f637 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 16:09:06.720064 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=77 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 16:09:06.725599 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0764955d-0cc2-4b2b-8721-f44e95c8f637 None None] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 16:09:06.729713 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63e8adc7-6bcc-4827-ae0d-14fbe41e3e64 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79706d480f864998b9f8594be407ceab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4ee0413f9ef4d4fb1f95a660babe090', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 16:09:06.736523 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63e8adc7-6bcc-4827-ae0d-14fbe41e3e64 service cinder] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:09:06.736523 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63e8adc7-6bcc-4827-ae0d-14fbe41e3e64 service cinder] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:09:06.736523 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63e8adc7-6bcc-4827-ae0d-14fbe41e3e64 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:09:06.739812 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63e8adc7-6bcc-4827-ae0d-14fbe41e3e64 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 16:09:06.741231 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63e8adc7-6bcc-4827-ae0d-14fbe41e3e64 service cinder] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 16:09:06.751265 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 780/1539] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Aug 22 16:09:06 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 16:09:16.133429 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG dbcounter [-] [81767] Writing DB stats keystone:SELECT=47 {{(pid=81767) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 16:09:16.754847 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=5 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 16:10:31.158263 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-313c3f45-d866-4032-b375-1f90edcef631 None None] REQUEST_METHOD: `POST` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:10:31.158994 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-313c3f45-d866-4032-b375-1f90edcef631 None None] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:10:31.158994 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-313c3f45-d866-4032-b375-1f90edcef631 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:10:31.184432 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.auth.core [None req-313c3f45-d866-4032-b375-1f90edcef631 None None] MFA Rules not processed for user `6b2199ae0fb0480aa917ada391f5ec54`. Rule list: `[]` (Enabled: `True`). {{(pid=81767) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 16:10:31.186674 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.fernet_utils [None req-313c3f45-d866-4032-b375-1f90edcef631 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81767) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 16:10:31.200242 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 760/1540] 10.222.0.15 () {58 vars in 958 bytes} [Mon Aug 22 16:10:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 16:10:31.209022 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1fc09bc-0164-47f5-abb7-2fced21fac6a None None] REQUEST_METHOD: `POST` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:10:31.210106 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1fc09bc-0164-47f5-abb7-2fced21fac6a None None] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:10:31.210106 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1fc09bc-0164-47f5-abb7-2fced21fac6a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:10:31.236309 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.auth.core [None req-b1fc09bc-0164-47f5-abb7-2fced21fac6a None None] MFA Rules not processed for user `6b2199ae0fb0480aa917ada391f5ec54`. Rule list: `[]` (Enabled: `True`). {{(pid=81768) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 16:10:31.237954 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-b1fc09bc-0164-47f5-abb7-2fced21fac6a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 16:10:31.249820 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 781/1541] 10.222.0.15 () {58 vars in 958 bytes} [Mon Aug 22 16:10:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 16:10:31.254104 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-313c3f45-d866-4032-b375-1f90edcef631 None None] Authenticating user token {{(pid=81767) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 16:10:31.283459 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-313c3f45-d866-4032-b375-1f90edcef631 None None] Validating token access rules against request {{(pid=81767) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 16:10:31.296125 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e97ccf5b-69d6-4fd4-8f7c-51b6a46deb76 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a98b1e4431940ab8d1fc95df2858a6c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81767) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 16:10:31.296125 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e97ccf5b-69d6-4fd4-8f7c-51b6a46deb76 admin admin] REQUEST_METHOD: `GET` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:10:31.296125 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e97ccf5b-69d6-4fd4-8f7c-51b6a46deb76 admin admin] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:10:31.296125 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e97ccf5b-69d6-4fd4-8f7c-51b6a46deb76 admin admin] PATH_INFO: `/v3/domains` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:10:31.296532 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e97ccf5b-69d6-4fd4-8f7c-51b6a46deb76 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81767) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 16:10:31.301446 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e97ccf5b-69d6-4fd4-8f7c-51b6a46deb76 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 16:10:31.303286 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e97ccf5b-69d6-4fd4-8f7c-51b6a46deb76 admin admin] RBAC: Authorization granted {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 16:10:31.318733 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 761/1542] 10.222.0.15 () {58 vars in 1153 bytes} [Mon Aug 22 16:10:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 65 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 16:10:31.327074 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1fc09bc-0164-47f5-abb7-2fced21fac6a None None] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 16:10:31.340257 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1fc09bc-0164-47f5-abb7-2fced21fac6a None None] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 16:10:31.352949 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e3b582e-ced0-4f0c-8734-a4eb2982581e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a98b1e4431940ab8d1fc95df2858a6c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 16:10:31.353498 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e3b582e-ced0-4f0c-8734-a4eb2982581e admin admin] REQUEST_METHOD: `POST` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:10:31.353784 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e3b582e-ced0-4f0c-8734-a4eb2982581e admin admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:10:31.354060 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e3b582e-ced0-4f0c-8734-a4eb2982581e admin admin] PATH_INFO: `/v3/projects` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:10:31.354703 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e3b582e-ced0-4f0c-8734-a4eb2982581e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 16:10:31.357000 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e3b582e-ced0-4f0c-8734-a4eb2982581e admin admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 16:10:31.460624 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 782/1543] 10.222.0.15 () {60 vars in 1153 bytes} [Mon Aug 22 16:10:31 2022] POST /identity/v3/projects => generated 379 bytes in 134 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 22 16:10:31.466762 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e97ccf5b-69d6-4fd4-8f7c-51b6a46deb76 admin admin] Authenticating user token {{(pid=81767) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 16:10:31.494306 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e97ccf5b-69d6-4fd4-8f7c-51b6a46deb76 admin admin] Validating token access rules against request {{(pid=81767) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 16:10:31.594806 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43a5757c-afb4-41f5-8599-ec76228873b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a98b1e4431940ab8d1fc95df2858a6c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81767) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 16:10:31.595461 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43a5757c-afb4-41f5-8599-ec76228873b5 admin admin] REQUEST_METHOD: `POST` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:10:31.595742 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43a5757c-afb4-41f5-8599-ec76228873b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:10:31.596012 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43a5757c-afb4-41f5-8599-ec76228873b5 admin admin] PATH_INFO: `/v3/users` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:10:31.596644 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43a5757c-afb4-41f5-8599-ec76228873b5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 16:10:31.598603 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43a5757c-afb4-41f5-8599-ec76228873b5 admin admin] RBAC: Authorization granted {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 16:10:31.618266 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-694146ba-8f79-4063-9a0a-efceb441c94c None None] REQUEST_METHOD: `POST` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:10:31.618484 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-694146ba-8f79-4063-9a0a-efceb441c94c None None] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:10:31.618751 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-694146ba-8f79-4063-9a0a-efceb441c94c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:10:31.640326 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.auth.core [None req-694146ba-8f79-4063-9a0a-efceb441c94c None None] MFA Rules not processed for user `6b2199ae0fb0480aa917ada391f5ec54`. Rule list: `[]` (Enabled: `True`). {{(pid=81768) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 16:10:31.641636 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-694146ba-8f79-4063-9a0a-efceb441c94c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 16:10:31.651545 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 783/1544] 10.222.0.15 () {58 vars in 958 bytes} [Mon Aug 22 16:10:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 16:10:31.669367 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b1b1d02-f6ee-4f8a-bc59-81e8b1cee6a6 None None] REQUEST_METHOD: `POST` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:10:31.669747 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b1b1d02-f6ee-4f8a-bc59-81e8b1cee6a6 None None] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:10:31.669962 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b1b1d02-f6ee-4f8a-bc59-81e8b1cee6a6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:10:31.701933 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.auth.core [None req-3b1b1d02-f6ee-4f8a-bc59-81e8b1cee6a6 None None] MFA Rules not processed for user `6b2199ae0fb0480aa917ada391f5ec54`. Rule list: `[]` (Enabled: `True`). {{(pid=81768) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 16:10:31.703173 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-3b1b1d02-f6ee-4f8a-bc59-81e8b1cee6a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 16:10:31.715597 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 784/1545] 10.222.0.15 () {58 vars in 958 bytes} [Mon Aug 22 16:10:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 47 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 22 16:10:31.720641 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b1b1d02-f6ee-4f8a-bc59-81e8b1cee6a6 None None] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 16:10:31.731659 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b1b1d02-f6ee-4f8a-bc59-81e8b1cee6a6 None None] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 16:10:31.739521 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06372afb-eb70-4c0f-9fd1-def71aa5cf63 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a98b1e4431940ab8d1fc95df2858a6c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 16:10:31.740824 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06372afb-eb70-4c0f-9fd1-def71aa5cf63 admin admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:10:31.741117 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06372afb-eb70-4c0f-9fd1-def71aa5cf63 admin admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:10:31.743734 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06372afb-eb70-4c0f-9fd1-def71aa5cf63 admin admin] PATH_INFO: `/v3/domains` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:10:31.744462 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06372afb-eb70-4c0f-9fd1-def71aa5cf63 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81768) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 16:10:31.744779 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06372afb-eb70-4c0f-9fd1-def71aa5cf63 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 16:10:31.745852 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: WARNING py.warnings [None req-06372afb-eb70-4c0f-9fd1-def71aa5cf63 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Aug 22 16:10:31.745852 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: warnings.warn(msg) Aug 22 16:10:31.745852 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: Aug 22 16:10:31.746761 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06372afb-eb70-4c0f-9fd1-def71aa5cf63 admin admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 16:10:31.762963 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 785/1546] 10.222.0.15 () {58 vars in 1153 bytes} [Mon Aug 22 16:10:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 22 16:10:31.766194 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 762/1547] 10.222.0.15 () {60 vars in 1144 bytes} [Mon Aug 22 16:10:31 2022] POST /identity/v3/users => generated 290 bytes in 300 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 22 16:10:31.770526 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06372afb-eb70-4c0f-9fd1-def71aa5cf63 admin admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 16:10:31.771003 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee1b7fe-9f21-4994-8168-402707fb3af2 None None] REQUEST_METHOD: `POST` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:10:31.771305 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee1b7fe-9f21-4994-8168-402707fb3af2 None None] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:10:31.771618 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee1b7fe-9f21-4994-8168-402707fb3af2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:10:31.782186 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06372afb-eb70-4c0f-9fd1-def71aa5cf63 admin admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 16:10:31.791884 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25288957-bba9-49d6-926e-00f28c23faec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a98b1e4431940ab8d1fc95df2858a6c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 16:10:31.792507 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25288957-bba9-49d6-926e-00f28c23faec admin admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:10:31.792784 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25288957-bba9-49d6-926e-00f28c23faec admin admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:10:31.793059 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25288957-bba9-49d6-926e-00f28c23faec admin admin] PATH_INFO: `/v3/roles` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:10:31.793714 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25288957-bba9-49d6-926e-00f28c23faec admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 16:10:31.795722 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25288957-bba9-49d6-926e-00f28c23faec admin admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 16:10:31.819558 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.auth.core [None req-fee1b7fe-9f21-4994-8168-402707fb3af2 None None] MFA Rules not processed for user `6b2199ae0fb0480aa917ada391f5ec54`. Rule list: `[]` (Enabled: `True`). {{(pid=81767) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 16:10:31.820804 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.fernet_utils [None req-fee1b7fe-9f21-4994-8168-402707fb3af2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81767) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 16:10:31.840065 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 763/1548] 10.222.0.15 () {58 vars in 958 bytes} [Mon Aug 22 16:10:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 70 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 16:10:31.840959 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fee1b7fe-9f21-4994-8168-402707fb3af2 None None] Authenticating user token {{(pid=81767) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 16:10:31.856650 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fee1b7fe-9f21-4994-8168-402707fb3af2 None None] Validating token access rules against request {{(pid=81767) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 16:10:31.875674 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e8253ed-71b9-4d03-ac91-104412cc0dba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a98b1e4431940ab8d1fc95df2858a6c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81767) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 16:10:31.876232 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e8253ed-71b9-4d03-ac91-104412cc0dba admin admin] REQUEST_METHOD: `POST` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:10:31.876510 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e8253ed-71b9-4d03-ac91-104412cc0dba admin admin] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:10:31.876781 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e8253ed-71b9-4d03-ac91-104412cc0dba admin admin] PATH_INFO: `/v3/projects` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:10:31.877432 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e8253ed-71b9-4d03-ac91-104412cc0dba admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 16:10:31.879114 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: WARNING py.warnings [None req-3e8253ed-71b9-4d03-ac91-104412cc0dba admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 22 16:10:31.879114 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: warnings.warn(msg) Aug 22 16:10:31.879114 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: Aug 22 16:10:31.880368 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e8253ed-71b9-4d03-ac91-104412cc0dba admin admin] RBAC: Authorization granted {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 16:10:31.959042 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 786/1549] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Aug 22 16:10:31 2022] GET /identity/v3/roles => generated 1420 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 22 16:10:31.961261 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb212abf-2d7d-4264-9d55-3b1ad1df2ac3 None None] REQUEST_METHOD: `POST` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:10:31.961540 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb212abf-2d7d-4264-9d55-3b1ad1df2ac3 None None] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:10:31.961809 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb212abf-2d7d-4264-9d55-3b1ad1df2ac3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:10:31.983751 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.auth.core [None req-bb212abf-2d7d-4264-9d55-3b1ad1df2ac3 None None] MFA Rules not processed for user `6b2199ae0fb0480aa917ada391f5ec54`. Rule list: `[]` (Enabled: `True`). {{(pid=81768) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 16:10:31.984870 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-bb212abf-2d7d-4264-9d55-3b1ad1df2ac3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 16:10:31.992914 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 787/1550] 10.222.0.15 () {58 vars in 958 bytes} [Mon Aug 22 16:10:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 16:10:31.994201 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e94c037e-b151-4b4e-987b-85b95bbce914 None None] REQUEST_METHOD: `POST` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:10:31.994481 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e94c037e-b151-4b4e-987b-85b95bbce914 None None] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:10:31.994748 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e94c037e-b151-4b4e-987b-85b95bbce914 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:10:32.014604 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.auth.core [None req-e94c037e-b151-4b4e-987b-85b95bbce914 None None] MFA Rules not processed for user `6b2199ae0fb0480aa917ada391f5ec54`. Rule list: `[]` (Enabled: `True`). {{(pid=81768) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 16:10:32.015856 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-e94c037e-b151-4b4e-987b-85b95bbce914 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 16:10:32.025446 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 788/1551] 10.222.0.15 () {58 vars in 958 bytes} [Mon Aug 22 16:10:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 16:10:32.026642 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0078c8b9-8dc1-4e57-be29-059e33c73ff3 None None] REQUEST_METHOD: `POST` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:10:32.027094 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0078c8b9-8dc1-4e57-be29-059e33c73ff3 None None] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:10:32.027349 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0078c8b9-8dc1-4e57-be29-059e33c73ff3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:10:32.035994 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 764/1552] 10.222.0.15 () {60 vars in 1153 bytes} [Mon Aug 22 16:10:31 2022] POST /identity/v3/projects => generated 375 bytes in 195 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 22 16:10:32.037222 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7faf6cc6-48bc-42e9-9756-457e6393d58c None None] REQUEST_METHOD: `POST` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:10:32.037514 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7faf6cc6-48bc-42e9-9756-457e6393d58c None None] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:10:32.038346 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7faf6cc6-48bc-42e9-9756-457e6393d58c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:10:32.047573 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.auth.core [None req-0078c8b9-8dc1-4e57-be29-059e33c73ff3 None None] MFA Rules not processed for user `6b2199ae0fb0480aa917ada391f5ec54`. Rule list: `[]` (Enabled: `True`). {{(pid=81768) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 16:10:32.048746 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-0078c8b9-8dc1-4e57-be29-059e33c73ff3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 16:10:32.058260 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.auth.core [None req-7faf6cc6-48bc-42e9-9756-457e6393d58c None None] MFA Rules not processed for user `6b2199ae0fb0480aa917ada391f5ec54`. Rule list: `[]` (Enabled: `True`). {{(pid=81767) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 16:10:32.059491 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.fernet_utils [None req-7faf6cc6-48bc-42e9-9756-457e6393d58c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81767) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 16:10:32.112129 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 789/1553] 10.222.0.15 () {58 vars in 958 bytes} [Mon Aug 22 16:10:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 16:10:32.113338 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0078c8b9-8dc1-4e57-be29-059e33c73ff3 None None] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 16:10:32.122594 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0078c8b9-8dc1-4e57-be29-059e33c73ff3 None None] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 16:10:32.125452 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 765/1554] 10.222.0.15 () {58 vars in 958 bytes} [Mon Aug 22 16:10:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 16:10:32.126793 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d762d10d-43c8-408a-892f-3db2572a6fef None None] REQUEST_METHOD: `POST` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:10:32.127140 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d762d10d-43c8-408a-892f-3db2572a6fef None None] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:10:32.127259 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebb018bf-e22f-4576-aa2c-6c8687d99fa2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a98b1e4431940ab8d1fc95df2858a6c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 16:10:32.127518 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d762d10d-43c8-408a-892f-3db2572a6fef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:10:32.127795 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebb018bf-e22f-4576-aa2c-6c8687d99fa2 admin admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:10:32.128059 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebb018bf-e22f-4576-aa2c-6c8687d99fa2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:10:32.128331 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebb018bf-e22f-4576-aa2c-6c8687d99fa2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:10:32.128994 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebb018bf-e22f-4576-aa2c-6c8687d99fa2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 16:10:32.130575 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebb018bf-e22f-4576-aa2c-6c8687d99fa2 admin admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 16:10:32.142547 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 790/1555] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Aug 22 16:10:32 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 22 16:10:32.143606 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f212a79f-e572-414b-9005-9287f2186d4e None None] REQUEST_METHOD: `POST` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:10:32.143917 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f212a79f-e572-414b-9005-9287f2186d4e None None] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:10:32.144187 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f212a79f-e572-414b-9005-9287f2186d4e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:10:32.148800 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.auth.core [None req-d762d10d-43c8-408a-892f-3db2572a6fef None None] MFA Rules not processed for user `6b2199ae0fb0480aa917ada391f5ec54`. Rule list: `[]` (Enabled: `True`). {{(pid=81767) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 16:10:32.150015 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.fernet_utils [None req-d762d10d-43c8-408a-892f-3db2572a6fef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81767) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 16:10:32.158325 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 766/1556] 10.222.0.15 () {58 vars in 958 bytes} [Mon Aug 22 16:10:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 16:10:32.159591 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81efadb1-55a8-4660-82e5-c4e8b52f5b72 None None] REQUEST_METHOD: `POST` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:10:32.159873 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81efadb1-55a8-4660-82e5-c4e8b52f5b72 None None] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:10:32.160184 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81efadb1-55a8-4660-82e5-c4e8b52f5b72 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:10:32.166895 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.auth.core [None req-f212a79f-e572-414b-9005-9287f2186d4e None None] MFA Rules not processed for user `6b2199ae0fb0480aa917ada391f5ec54`. Rule list: `[]` (Enabled: `True`). {{(pid=81768) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 16:10:32.168358 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-f212a79f-e572-414b-9005-9287f2186d4e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 16:10:32.176615 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 791/1557] 10.222.0.15 () {58 vars in 958 bytes} [Mon Aug 22 16:10:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 16:10:32.177793 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdd99f3c-c00a-4858-9cbf-cc55996a7f4d None None] REQUEST_METHOD: `POST` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:10:32.178069 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdd99f3c-c00a-4858-9cbf-cc55996a7f4d None None] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:10:32.178447 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdd99f3c-c00a-4858-9cbf-cc55996a7f4d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:10:32.179997 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.auth.core [None req-81efadb1-55a8-4660-82e5-c4e8b52f5b72 None None] MFA Rules not processed for user `6b2199ae0fb0480aa917ada391f5ec54`. Rule list: `[]` (Enabled: `True`). {{(pid=81767) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 16:10:32.181232 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.fernet_utils [None req-81efadb1-55a8-4660-82e5-c4e8b52f5b72 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81767) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 16:10:32.189425 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 767/1558] 10.222.0.15 () {58 vars in 958 bytes} [Mon Aug 22 16:10:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 16:10:32.190286 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81efadb1-55a8-4660-82e5-c4e8b52f5b72 None None] Authenticating user token {{(pid=81767) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 16:10:32.202690 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81efadb1-55a8-4660-82e5-c4e8b52f5b72 None None] Validating token access rules against request {{(pid=81767) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 16:10:32.204021 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.auth.core [None req-fdd99f3c-c00a-4858-9cbf-cc55996a7f4d None None] MFA Rules not processed for user `6b2199ae0fb0480aa917ada391f5ec54`. Rule list: `[]` (Enabled: `True`). {{(pid=81768) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 16:10:32.205237 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-fdd99f3c-c00a-4858-9cbf-cc55996a7f4d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 16:10:32.207577 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-976a77ea-e4e0-4196-b9fe-bd3ecf7d7039 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a98b1e4431940ab8d1fc95df2858a6c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81767) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 16:10:32.208430 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-976a77ea-e4e0-4196-b9fe-bd3ecf7d7039 admin admin] REQUEST_METHOD: `POST` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:10:32.208718 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-976a77ea-e4e0-4196-b9fe-bd3ecf7d7039 admin admin] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:10:32.208990 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-976a77ea-e4e0-4196-b9fe-bd3ecf7d7039 admin admin] PATH_INFO: `/v3/users` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:10:32.209716 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-976a77ea-e4e0-4196-b9fe-bd3ecf7d7039 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 16:10:32.211649 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-976a77ea-e4e0-4196-b9fe-bd3ecf7d7039 admin admin] RBAC: Authorization granted {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 16:10:32.213540 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 792/1559] 10.222.0.15 () {58 vars in 958 bytes} [Mon Aug 22 16:10:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 16:10:32.214820 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdd99f3c-c00a-4858-9cbf-cc55996a7f4d None None] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 16:10:32.224425 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdd99f3c-c00a-4858-9cbf-cc55996a7f4d None None] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 16:10:32.229574 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfa81056-f0f7-4b3b-a593-441ddb224afe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a98b1e4431940ab8d1fc95df2858a6c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 16:10:32.234482 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfa81056-f0f7-4b3b-a593-441ddb224afe admin admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:10:32.234482 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfa81056-f0f7-4b3b-a593-441ddb224afe admin admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:10:32.234482 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfa81056-f0f7-4b3b-a593-441ddb224afe admin admin] PATH_INFO: `/v3/domains` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:10:32.234482 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfa81056-f0f7-4b3b-a593-441ddb224afe admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81768) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 16:10:32.234482 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfa81056-f0f7-4b3b-a593-441ddb224afe admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 16:10:32.234482 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfa81056-f0f7-4b3b-a593-441ddb224afe admin admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 16:10:32.245632 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 793/1560] 10.222.0.15 () {58 vars in 1153 bytes} [Mon Aug 22 16:10:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 16:10:32.246808 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-054a9dfd-5f14-4f5c-a7a5-db40740c2c29 None None] REQUEST_METHOD: `POST` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:10:32.247093 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-054a9dfd-5f14-4f5c-a7a5-db40740c2c29 None None] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:10:32.247395 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-054a9dfd-5f14-4f5c-a7a5-db40740c2c29 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:10:32.266652 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.auth.core [None req-054a9dfd-5f14-4f5c-a7a5-db40740c2c29 None None] MFA Rules not processed for user `6b2199ae0fb0480aa917ada391f5ec54`. Rule list: `[]` (Enabled: `True`). {{(pid=81768) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 16:10:32.267878 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-054a9dfd-5f14-4f5c-a7a5-db40740c2c29 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 16:10:32.275768 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 794/1561] 10.222.0.15 () {58 vars in 958 bytes} [Mon Aug 22 16:10:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 16:10:32.276526 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-054a9dfd-5f14-4f5c-a7a5-db40740c2c29 None None] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 16:10:32.285780 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-054a9dfd-5f14-4f5c-a7a5-db40740c2c29 None None] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 16:10:32.290954 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-347d28f8-6eef-4918-b98d-ab74f64f45d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a98b1e4431940ab8d1fc95df2858a6c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 16:10:32.293161 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-347d28f8-6eef-4918-b98d-ab74f64f45d7 admin admin] REQUEST_METHOD: `PUT` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:10:32.293161 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-347d28f8-6eef-4918-b98d-ab74f64f45d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:10:32.293161 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-347d28f8-6eef-4918-b98d-ab74f64f45d7 admin admin] PATH_INFO: `/v3/projects/ceb2b51fac23410cba12504825341c28/users/4763c7fefe5a4fcdbead877717f760b8/roles/e6fc1ee3fb974125ba9b4fe55b12b45e` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:10:32.306580 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-347d28f8-6eef-4918-b98d-ab74f64f45d7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ceb2b51fac23410cba12504825341c28, user_id=4763c7fefe5a4fcdbead877717f760b8, role_id=e6fc1ee3fb974125ba9b4fe55b12b45e)` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 16:10:32.309250 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-347d28f8-6eef-4918-b98d-ab74f64f45d7 admin admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 16:10:32.318204 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 768/1562] 10.222.0.15 () {60 vars in 1144 bytes} [Mon Aug 22 16:10:32 2022] POST /identity/v3/users => generated 288 bytes in 128 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 22 16:10:32.321742 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4d1b360-65e1-4dd5-8a7f-50e3773ae028 None None] REQUEST_METHOD: `POST` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:10:32.321742 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4d1b360-65e1-4dd5-8a7f-50e3773ae028 None None] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:10:32.321742 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4d1b360-65e1-4dd5-8a7f-50e3773ae028 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:10:32.340451 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.auth.core [None req-a4d1b360-65e1-4dd5-8a7f-50e3773ae028 None None] MFA Rules not processed for user `6b2199ae0fb0480aa917ada391f5ec54`. Rule list: `[]` (Enabled: `True`). {{(pid=81767) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 16:10:32.341639 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.fernet_utils [None req-a4d1b360-65e1-4dd5-8a7f-50e3773ae028 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81767) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 16:10:32.349511 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 769/1563] 10.222.0.15 () {58 vars in 958 bytes} [Mon Aug 22 16:10:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 16:10:32.350420 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4d1b360-65e1-4dd5-8a7f-50e3773ae028 None None] Authenticating user token {{(pid=81767) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 16:10:32.356372 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG dbcounter [-] [81767] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=81767) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 16:10:32.360177 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4d1b360-65e1-4dd5-8a7f-50e3773ae028 None None] Validating token access rules against request {{(pid=81767) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 16:10:32.364806 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e3b88c2-a59e-4aa3-a9a5-388e977071fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a98b1e4431940ab8d1fc95df2858a6c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81767) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 16:10:32.365411 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e3b88c2-a59e-4aa3-a9a5-388e977071fc admin admin] REQUEST_METHOD: `GET` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:10:32.365733 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e3b88c2-a59e-4aa3-a9a5-388e977071fc admin admin] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:10:32.366045 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e3b88c2-a59e-4aa3-a9a5-388e977071fc admin admin] PATH_INFO: `/v3/domains` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:10:32.367117 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e3b88c2-a59e-4aa3-a9a5-388e977071fc admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81767) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 16:10:32.367479 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e3b88c2-a59e-4aa3-a9a5-388e977071fc admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 16:10:32.369583 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e3b88c2-a59e-4aa3-a9a5-388e977071fc admin admin] RBAC: Authorization granted {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 16:10:32.382078 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 770/1564] 10.222.0.15 () {58 vars in 1153 bytes} [Mon Aug 22 16:10:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 16:10:32.383174 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc90dd36-b67a-4e71-81e5-51c45b765f61 None None] REQUEST_METHOD: `POST` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:10:32.383492 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc90dd36-b67a-4e71-81e5-51c45b765f61 None None] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:10:32.383767 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc90dd36-b67a-4e71-81e5-51c45b765f61 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:10:32.403977 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.auth.core [None req-bc90dd36-b67a-4e71-81e5-51c45b765f61 None None] MFA Rules not processed for user `6b2199ae0fb0480aa917ada391f5ec54`. Rule list: `[]` (Enabled: `True`). {{(pid=81767) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 16:10:32.405273 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.fernet_utils [None req-bc90dd36-b67a-4e71-81e5-51c45b765f61 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81767) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 16:10:32.413252 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 771/1565] 10.222.0.15 () {58 vars in 958 bytes} [Mon Aug 22 16:10:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 16:10:32.414301 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc90dd36-b67a-4e71-81e5-51c45b765f61 None None] Authenticating user token {{(pid=81767) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 16:10:32.423476 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc90dd36-b67a-4e71-81e5-51c45b765f61 None None] Validating token access rules against request {{(pid=81767) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 16:10:32.423677 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 795/1566] 10.222.0.15 () {60 vars in 1483 bytes} [Mon Aug 22 16:10:32 2022] PUT /identity/v3/projects/ceb2b51fac23410cba12504825341c28/users/4763c7fefe5a4fcdbead877717f760b8/roles/e6fc1ee3fb974125ba9b4fe55b12b45e => generated 0 bytes in 147 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 16:10:32.428042 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-347d28f8-6eef-4918-b98d-ab74f64f45d7 admin admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 16:10:32.442718 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 16:10:32.485036 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c75b6459-46a8-4554-bf63-e2295ebf39ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a98b1e4431940ab8d1fc95df2858a6c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81767) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 16:10:32.509223 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c75b6459-46a8-4554-bf63-e2295ebf39ef admin admin] REQUEST_METHOD: `GET` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:10:32.509223 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c75b6459-46a8-4554-bf63-e2295ebf39ef admin admin] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:10:32.509223 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c75b6459-46a8-4554-bf63-e2295ebf39ef admin admin] PATH_INFO: `/v3/domains` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:10:32.509223 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c75b6459-46a8-4554-bf63-e2295ebf39ef admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81767) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 16:10:32.509223 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c75b6459-46a8-4554-bf63-e2295ebf39ef admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 16:10:32.509223 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c75b6459-46a8-4554-bf63-e2295ebf39ef admin admin] RBAC: Authorization granted {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 16:10:32.509761 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-347d28f8-6eef-4918-b98d-ab74f64f45d7 admin admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 16:10:32.509857 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 772/1567] 10.222.0.15 () {58 vars in 1153 bytes} [Mon Aug 22 16:10:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 16:10:32.509857 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c75b6459-46a8-4554-bf63-e2295ebf39ef admin admin] Authenticating user token {{(pid=81767) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 16:10:32.509996 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4620b5a-4285-424a-ae0b-0c2bea587c47 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a98b1e4431940ab8d1fc95df2858a6c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 16:10:32.509996 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4620b5a-4285-424a-ae0b-0c2bea587c47 admin admin] REQUEST_METHOD: `POST` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:10:32.509996 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4620b5a-4285-424a-ae0b-0c2bea587c47 admin admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:10:32.509996 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4620b5a-4285-424a-ae0b-0c2bea587c47 admin admin] PATH_INFO: `/v3/projects` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:10:32.510301 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4620b5a-4285-424a-ae0b-0c2bea587c47 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 16:10:32.511421 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4620b5a-4285-424a-ae0b-0c2bea587c47 admin admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 16:10:32.520622 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c75b6459-46a8-4554-bf63-e2295ebf39ef admin admin] Validating token access rules against request {{(pid=81767) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 16:10:32.525827 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11907165-f8e0-40a4-a287-4fd9a26e86db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a98b1e4431940ab8d1fc95df2858a6c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81767) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 16:10:32.526427 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11907165-f8e0-40a4-a287-4fd9a26e86db admin admin] REQUEST_METHOD: `GET` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:10:32.526699 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11907165-f8e0-40a4-a287-4fd9a26e86db admin admin] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:10:32.526967 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11907165-f8e0-40a4-a287-4fd9a26e86db admin admin] PATH_INFO: `/v3/domains` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:10:32.527639 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11907165-f8e0-40a4-a287-4fd9a26e86db admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81767) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 16:10:32.527941 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11907165-f8e0-40a4-a287-4fd9a26e86db admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 16:10:32.529559 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11907165-f8e0-40a4-a287-4fd9a26e86db admin admin] RBAC: Authorization granted {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 16:10:32.543210 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 773/1568] 10.222.0.15 () {58 vars in 1153 bytes} [Mon Aug 22 16:10:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 16:10:32.544029 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11907165-f8e0-40a4-a287-4fd9a26e86db admin admin] Authenticating user token {{(pid=81767) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 16:10:32.555588 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11907165-f8e0-40a4-a287-4fd9a26e86db admin admin] Validating token access rules against request {{(pid=81767) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 16:10:32.560202 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dab5e333-a035-481a-bb3b-da862d0e0650 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a98b1e4431940ab8d1fc95df2858a6c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81767) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 16:10:32.560775 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab5e333-a035-481a-bb3b-da862d0e0650 admin admin] REQUEST_METHOD: `GET` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:10:32.561052 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab5e333-a035-481a-bb3b-da862d0e0650 admin admin] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:10:32.561315 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab5e333-a035-481a-bb3b-da862d0e0650 admin admin] PATH_INFO: `/v3/roles` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:10:32.561935 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dab5e333-a035-481a-bb3b-da862d0e0650 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 16:10:32.563523 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dab5e333-a035-481a-bb3b-da862d0e0650 admin admin] RBAC: Authorization granted {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 16:10:32.576745 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 774/1569] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Aug 22 16:10:32 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 22 16:10:32.576745 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dab5e333-a035-481a-bb3b-da862d0e0650 admin admin] Authenticating user token {{(pid=81767) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 16:10:32.584355 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dab5e333-a035-481a-bb3b-da862d0e0650 admin admin] Validating token access rules against request {{(pid=81767) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 16:10:32.589700 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17edf1af-2a81-4763-a7e3-9c63a67ceb99 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a98b1e4431940ab8d1fc95df2858a6c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81767) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 16:10:32.590273 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17edf1af-2a81-4763-a7e3-9c63a67ceb99 admin admin] REQUEST_METHOD: `GET` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:10:32.590547 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17edf1af-2a81-4763-a7e3-9c63a67ceb99 admin admin] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:10:32.591017 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17edf1af-2a81-4763-a7e3-9c63a67ceb99 admin admin] PATH_INFO: `/v3/domains` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:10:32.591691 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17edf1af-2a81-4763-a7e3-9c63a67ceb99 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81767) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 16:10:32.591988 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17edf1af-2a81-4763-a7e3-9c63a67ceb99 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 16:10:32.593565 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17edf1af-2a81-4763-a7e3-9c63a67ceb99 admin admin] RBAC: Authorization granted {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 16:10:32.595298 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 796/1570] 10.222.0.15 () {60 vars in 1153 bytes} [Mon Aug 22 16:10:32 2022] POST /identity/v3/projects => generated 393 bytes in 171 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 22 16:10:32.600047 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4620b5a-4285-424a-ae0b-0c2bea587c47 admin admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 16:10:32.607919 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 775/1571] 10.222.0.15 () {58 vars in 1153 bytes} [Mon Aug 22 16:10:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 16:10:32.608868 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17edf1af-2a81-4763-a7e3-9c63a67ceb99 admin admin] Authenticating user token {{(pid=81767) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 16:10:32.618544 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17edf1af-2a81-4763-a7e3-9c63a67ceb99 admin admin] Validating token access rules against request {{(pid=81767) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 16:10:32.670881 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4620b5a-4285-424a-ae0b-0c2bea587c47 admin admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 16:10:32.675447 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b10f87dc-f526-4281-aba1-2551a0d9ce7d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a98b1e4431940ab8d1fc95df2858a6c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 16:10:32.676050 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b10f87dc-f526-4281-aba1-2551a0d9ce7d admin admin] REQUEST_METHOD: `POST` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:10:32.676437 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b10f87dc-f526-4281-aba1-2551a0d9ce7d admin admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:10:32.676812 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b10f87dc-f526-4281-aba1-2551a0d9ce7d admin admin] PATH_INFO: `/v3/projects` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:10:32.677436 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b10f87dc-f526-4281-aba1-2551a0d9ce7d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 16:10:32.677785 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f189654-8193-47fe-8154-ced3d3ca0cd3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a98b1e4431940ab8d1fc95df2858a6c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81767) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 16:10:32.678416 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f189654-8193-47fe-8154-ced3d3ca0cd3 admin admin] REQUEST_METHOD: `GET` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:10:32.678690 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f189654-8193-47fe-8154-ced3d3ca0cd3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:10:32.678959 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f189654-8193-47fe-8154-ced3d3ca0cd3 admin admin] PATH_INFO: `/v3/domains` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:10:32.679376 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b10f87dc-f526-4281-aba1-2551a0d9ce7d admin admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 16:10:32.679631 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f189654-8193-47fe-8154-ced3d3ca0cd3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81767) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 16:10:32.679943 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f189654-8193-47fe-8154-ced3d3ca0cd3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 16:10:32.681484 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f189654-8193-47fe-8154-ced3d3ca0cd3 admin admin] RBAC: Authorization granted {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 16:10:32.692921 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 776/1572] 10.222.0.15 () {58 vars in 1153 bytes} [Mon Aug 22 16:10:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 16:10:32.694094 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05088459-e461-4263-b8a8-f226c9723993 None None] REQUEST_METHOD: `POST` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:10:32.694375 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05088459-e461-4263-b8a8-f226c9723993 None None] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:10:32.694642 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05088459-e461-4263-b8a8-f226c9723993 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:10:32.714459 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.auth.core [None req-05088459-e461-4263-b8a8-f226c9723993 None None] MFA Rules not processed for user `6b2199ae0fb0480aa917ada391f5ec54`. Rule list: `[]` (Enabled: `True`). {{(pid=81767) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 16:10:32.715811 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.fernet_utils [None req-05088459-e461-4263-b8a8-f226c9723993 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81767) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 16:10:32.723882 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 777/1573] 10.222.0.15 () {58 vars in 958 bytes} [Mon Aug 22 16:10:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 16:10:32.724637 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05088459-e461-4263-b8a8-f226c9723993 None None] Authenticating user token {{(pid=81767) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 16:10:32.734684 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05088459-e461-4263-b8a8-f226c9723993 None None] Validating token access rules against request {{(pid=81767) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 16:10:32.739498 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b353a9ec-0ea0-452d-b62c-21bb7551ee78 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a98b1e4431940ab8d1fc95df2858a6c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81767) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 16:10:32.740116 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b353a9ec-0ea0-452d-b62c-21bb7551ee78 admin admin] REQUEST_METHOD: `POST` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:10:32.740385 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b353a9ec-0ea0-452d-b62c-21bb7551ee78 admin admin] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:10:32.740645 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b353a9ec-0ea0-452d-b62c-21bb7551ee78 admin admin] PATH_INFO: `/v3/projects` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:10:32.741257 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b353a9ec-0ea0-452d-b62c-21bb7551ee78 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 16:10:32.743162 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b353a9ec-0ea0-452d-b62c-21bb7551ee78 admin admin] RBAC: Authorization granted {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 16:10:32.793886 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 797/1574] 10.222.0.15 () {60 vars in 1153 bytes} [Mon Aug 22 16:10:32 2022] POST /identity/v3/projects => generated 373 bytes in 198 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 22 16:10:32.794734 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b10f87dc-f526-4281-aba1-2551a0d9ce7d admin admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 16:10:32.857107 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 778/1575] 10.222.0.15 () {60 vars in 1153 bytes} [Mon Aug 22 16:10:32 2022] POST /identity/v3/projects => generated 403 bytes in 133 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 22 16:10:32.857945 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b353a9ec-0ea0-452d-b62c-21bb7551ee78 admin admin] Authenticating user token {{(pid=81767) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 16:10:32.858183 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b10f87dc-f526-4281-aba1-2551a0d9ce7d admin admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 16:10:32.867103 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG dbcounter [-] [81767] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81767) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 16:10:32.890306 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 16:10:32.919974 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97979138-bf9d-4651-bc14-891f63956c00 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a98b1e4431940ab8d1fc95df2858a6c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 16:10:32.920591 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97979138-bf9d-4651-bc14-891f63956c00 admin admin] REQUEST_METHOD: `POST` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:10:32.920871 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97979138-bf9d-4651-bc14-891f63956c00 admin admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:10:32.921207 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97979138-bf9d-4651-bc14-891f63956c00 admin admin] PATH_INFO: `/v3/projects` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:10:32.921766 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97979138-bf9d-4651-bc14-891f63956c00 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 16:10:32.922318 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b353a9ec-0ea0-452d-b62c-21bb7551ee78 admin admin] Validating token access rules against request {{(pid=81767) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 16:10:32.923935 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97979138-bf9d-4651-bc14-891f63956c00 admin admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 16:10:32.927322 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7ba16bc-adab-44cb-b7ad-537817e30e5f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a98b1e4431940ab8d1fc95df2858a6c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81767) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 16:10:32.927931 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7ba16bc-adab-44cb-b7ad-537817e30e5f admin admin] REQUEST_METHOD: `GET` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:10:32.928229 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7ba16bc-adab-44cb-b7ad-537817e30e5f admin admin] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:10:32.928497 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7ba16bc-adab-44cb-b7ad-537817e30e5f admin admin] PATH_INFO: `/v3/roles` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:10:32.929129 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7ba16bc-adab-44cb-b7ad-537817e30e5f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 16:10:32.930751 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7ba16bc-adab-44cb-b7ad-537817e30e5f admin admin] RBAC: Authorization granted {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 16:10:32.942152 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 779/1576] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Aug 22 16:10:32 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 22 16:10:32.943013 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7ba16bc-adab-44cb-b7ad-537817e30e5f admin admin] Authenticating user token {{(pid=81767) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 16:10:32.956509 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7ba16bc-adab-44cb-b7ad-537817e30e5f admin admin] Validating token access rules against request {{(pid=81767) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 16:10:32.962345 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63828413-a23b-4ffa-bdbb-b7ee19986747 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a98b1e4431940ab8d1fc95df2858a6c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81767) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 16:10:32.962888 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63828413-a23b-4ffa-bdbb-b7ee19986747 admin admin] REQUEST_METHOD: `POST` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:10:32.963172 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63828413-a23b-4ffa-bdbb-b7ee19986747 admin admin] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:10:32.963465 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63828413-a23b-4ffa-bdbb-b7ee19986747 admin admin] PATH_INFO: `/v3/users` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:10:32.964091 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63828413-a23b-4ffa-bdbb-b7ee19986747 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 16:10:32.966002 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63828413-a23b-4ffa-bdbb-b7ee19986747 admin admin] RBAC: Authorization granted {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 16:10:33.027689 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 798/1577] 10.222.0.15 () {60 vars in 1153 bytes} [Mon Aug 22 16:10:32 2022] POST /identity/v3/projects => generated 371 bytes in 230 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 22 16:10:33.027689 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97979138-bf9d-4651-bc14-891f63956c00 admin admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 16:10:33.084216 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97979138-bf9d-4651-bc14-891f63956c00 admin admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 16:10:33.086377 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 780/1578] 10.222.0.15 () {60 vars in 1144 bytes} [Mon Aug 22 16:10:32 2022] POST /identity/v3/users => generated 297 bytes in 144 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 22 16:10:33.089688 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e54bd1f-6402-4f67-8ecb-ad609c283a91 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a98b1e4431940ab8d1fc95df2858a6c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 16:10:33.090345 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63828413-a23b-4ffa-bdbb-b7ee19986747 admin admin] Authenticating user token {{(pid=81767) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 16:10:33.091107 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e54bd1f-6402-4f67-8ecb-ad609c283a91 admin admin] REQUEST_METHOD: `POST` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:10:33.091107 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e54bd1f-6402-4f67-8ecb-ad609c283a91 admin admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:10:33.091107 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e54bd1f-6402-4f67-8ecb-ad609c283a91 admin admin] PATH_INFO: `/v3/projects` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:10:33.091774 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e54bd1f-6402-4f67-8ecb-ad609c283a91 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 16:10:33.093687 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e54bd1f-6402-4f67-8ecb-ad609c283a91 admin admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 16:10:33.101019 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63828413-a23b-4ffa-bdbb-b7ee19986747 admin admin] Validating token access rules against request {{(pid=81767) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 16:10:33.105678 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-390a68ac-5d48-4990-8572-e1ff1bbc8463 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a98b1e4431940ab8d1fc95df2858a6c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81767) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 16:10:33.106368 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-390a68ac-5d48-4990-8572-e1ff1bbc8463 admin admin] REQUEST_METHOD: `POST` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:10:33.106656 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-390a68ac-5d48-4990-8572-e1ff1bbc8463 admin admin] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:10:33.107006 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-390a68ac-5d48-4990-8572-e1ff1bbc8463 admin admin] PATH_INFO: `/v3/projects` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:10:33.107735 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-390a68ac-5d48-4990-8572-e1ff1bbc8463 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 16:10:33.109620 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-390a68ac-5d48-4990-8572-e1ff1bbc8463 admin admin] RBAC: Authorization granted {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 16:10:33.201611 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 799/1579] 10.222.0.15 () {60 vars in 1153 bytes} [Mon Aug 22 16:10:33 2022] POST /identity/v3/projects => generated 383 bytes in 176 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 22 16:10:33.202373 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e54bd1f-6402-4f67-8ecb-ad609c283a91 admin admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 16:10:33.238762 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 781/1580] 10.222.0.15 () {60 vars in 1153 bytes} [Mon Aug 22 16:10:33 2022] POST /identity/v3/projects => generated 387 bytes in 145 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 22 16:10:33.238762 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-390a68ac-5d48-4990-8572-e1ff1bbc8463 admin admin] Authenticating user token {{(pid=81767) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 16:10:33.260940 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e54bd1f-6402-4f67-8ecb-ad609c283a91 admin admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 16:10:33.300454 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-390a68ac-5d48-4990-8572-e1ff1bbc8463 admin admin] Validating token access rules against request {{(pid=81767) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 16:10:33.305779 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bf0faaf-a919-4455-a6b0-e09c78533415 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a98b1e4431940ab8d1fc95df2858a6c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81767) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 16:10:33.306387 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf0faaf-a919-4455-a6b0-e09c78533415 admin admin] REQUEST_METHOD: `POST` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:10:33.306661 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf0faaf-a919-4455-a6b0-e09c78533415 admin admin] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:10:33.306932 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf0faaf-a919-4455-a6b0-e09c78533415 admin admin] PATH_INFO: `/v3/users` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:10:33.307585 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bf0faaf-a919-4455-a6b0-e09c78533415 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 16:10:33.309534 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bf0faaf-a919-4455-a6b0-e09c78533415 admin admin] RBAC: Authorization granted {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 16:10:33.312264 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fe9d5d7-008b-47dd-b6e1-88f9fc0d8c52 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '234687adad594f1aa45775a565ffbdc7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4ee0413f9ef4d4fb1f95a660babe090', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 16:10:33.312871 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fe9d5d7-008b-47dd-b6e1-88f9fc0d8c52 service neutron] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:10:33.313149 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fe9d5d7-008b-47dd-b6e1-88f9fc0d8c52 service neutron] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:10:33.313419 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fe9d5d7-008b-47dd-b6e1-88f9fc0d8c52 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:10:33.323299 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fe9d5d7-008b-47dd-b6e1-88f9fc0d8c52 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 16:10:33.324726 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fe9d5d7-008b-47dd-b6e1-88f9fc0d8c52 service neutron] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 16:10:33.329739 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 16:10:33.335475 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 800/1581] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Aug 22 16:10:33 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 133 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 16:10:33.336397 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fe9d5d7-008b-47dd-b6e1-88f9fc0d8c52 service neutron] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 16:10:33.345948 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fe9d5d7-008b-47dd-b6e1-88f9fc0d8c52 service neutron] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 16:10:33.351269 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfe35f65-8e26-46e7-a706-8848416addd2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a98b1e4431940ab8d1fc95df2858a6c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 16:10:33.351892 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfe35f65-8e26-46e7-a706-8848416addd2 admin admin] REQUEST_METHOD: `POST` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:10:33.352167 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfe35f65-8e26-46e7-a706-8848416addd2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:10:33.352477 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfe35f65-8e26-46e7-a706-8848416addd2 admin admin] PATH_INFO: `/v3/users` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:10:33.353225 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfe35f65-8e26-46e7-a706-8848416addd2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 16:10:33.354247 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: WARNING py.warnings [None req-dfe35f65-8e26-46e7-a706-8848416addd2 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 22 16:10:33.354247 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: warnings.warn(msg) Aug 22 16:10:33.354247 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: Aug 22 16:10:33.355469 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfe35f65-8e26-46e7-a706-8848416addd2 admin admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 16:10:33.417684 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 782/1582] 10.222.0.15 () {60 vars in 1144 bytes} [Mon Aug 22 16:10:33 2022] POST /identity/v3/users => generated 287 bytes in 182 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 22 16:10:33.418488 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bf0faaf-a919-4455-a6b0-e09c78533415 admin admin] Authenticating user token {{(pid=81767) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 16:10:33.428188 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bf0faaf-a919-4455-a6b0-e09c78533415 admin admin] Validating token access rules against request {{(pid=81767) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 16:10:33.432818 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38d04791-d786-40f9-bc64-66c4a3419346 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a98b1e4431940ab8d1fc95df2858a6c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81767) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 16:10:33.433448 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38d04791-d786-40f9-bc64-66c4a3419346 admin admin] REQUEST_METHOD: `PUT` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:10:33.433727 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38d04791-d786-40f9-bc64-66c4a3419346 admin admin] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:10:33.433994 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38d04791-d786-40f9-bc64-66c4a3419346 admin admin] PATH_INFO: `/v3/projects/efef11be3fe24f61975e2b30f1c31a94/users/6ec6b84e494c481abe9c9772a10037c9/roles/e6fc1ee3fb974125ba9b4fe55b12b45e` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:10:33.447906 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 801/1583] 10.222.0.15 () {60 vars in 1144 bytes} [Mon Aug 22 16:10:33 2022] POST /identity/v3/users => generated 302 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 22 16:10:33.447906 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfe35f65-8e26-46e7-a706-8848416addd2 admin admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 16:10:33.453275 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38d04791-d786-40f9-bc64-66c4a3419346 admin admin] RBAC: Authorizing `identity:create_grant(project_id=efef11be3fe24f61975e2b30f1c31a94, user_id=6ec6b84e494c481abe9c9772a10037c9, role_id=e6fc1ee3fb974125ba9b4fe55b12b45e)` {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 16:10:33.454273 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: WARNING py.warnings [None req-38d04791-d786-40f9-bc64-66c4a3419346 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 22 16:10:33.454273 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: warnings.warn(msg) Aug 22 16:10:33.454273 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: Aug 22 16:10:33.456301 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38d04791-d786-40f9-bc64-66c4a3419346 admin admin] RBAC: Authorization granted {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 16:10:33.457799 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfe35f65-8e26-46e7-a706-8848416addd2 admin admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 16:10:33.463257 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef445d97-17e4-447f-a471-264a207ba0cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a98b1e4431940ab8d1fc95df2858a6c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 16:10:33.463895 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef445d97-17e4-447f-a471-264a207ba0cf admin admin] REQUEST_METHOD: `POST` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:10:33.464168 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef445d97-17e4-447f-a471-264a207ba0cf admin admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:10:33.464435 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef445d97-17e4-447f-a471-264a207ba0cf admin admin] PATH_INFO: `/v3/users` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:10:33.465202 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef445d97-17e4-447f-a471-264a207ba0cf admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 16:10:33.467109 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef445d97-17e4-447f-a471-264a207ba0cf admin admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 16:10:33.560323 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 783/1584] 10.222.0.15 () {60 vars in 1483 bytes} [Mon Aug 22 16:10:33 2022] PUT /identity/v3/projects/efef11be3fe24f61975e2b30f1c31a94/users/6ec6b84e494c481abe9c9772a10037c9/roles/e6fc1ee3fb974125ba9b4fe55b12b45e => generated 0 bytes in 142 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 16:10:33.563814 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 802/1585] 10.222.0.15 () {60 vars in 1144 bytes} [Mon Aug 22 16:10:33 2022] POST /identity/v3/users => generated 286 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 22 16:10:33.567014 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38d04791-d786-40f9-bc64-66c4a3419346 admin admin] Authenticating user token {{(pid=81767) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 16:10:33.567161 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef445d97-17e4-447f-a471-264a207ba0cf admin admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 16:10:33.573257 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef445d97-17e4-447f-a471-264a207ba0cf admin admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 16:10:33.598502 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG dbcounter [-] [81767] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=81767) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 16:10:33.635409 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38d04791-d786-40f9-bc64-66c4a3419346 admin admin] Validating token access rules against request {{(pid=81767) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 16:10:33.638198 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12a147c4-9eae-4220-ad6c-00268cdaf2bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a98b1e4431940ab8d1fc95df2858a6c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 16:10:33.638822 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12a147c4-9eae-4220-ad6c-00268cdaf2bf admin admin] REQUEST_METHOD: `POST` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:10:33.639105 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12a147c4-9eae-4220-ad6c-00268cdaf2bf admin admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:10:33.639410 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12a147c4-9eae-4220-ad6c-00268cdaf2bf admin admin] PATH_INFO: `/v3/users` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:10:33.640038 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12a147c4-9eae-4220-ad6c-00268cdaf2bf admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 16:10:33.641576 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f7d11b2-f2cc-4dd0-a9a1-19507b01e23a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a98b1e4431940ab8d1fc95df2858a6c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81767) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 16:10:33.641730 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f7d11b2-f2cc-4dd0-a9a1-19507b01e23a admin admin] REQUEST_METHOD: `GET` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:10:33.641978 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f7d11b2-f2cc-4dd0-a9a1-19507b01e23a admin admin] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:10:33.642100 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12a147c4-9eae-4220-ad6c-00268cdaf2bf admin admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 16:10:33.642248 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f7d11b2-f2cc-4dd0-a9a1-19507b01e23a admin admin] PATH_INFO: `/v3/roles` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:10:33.642894 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f7d11b2-f2cc-4dd0-a9a1-19507b01e23a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 16:10:33.644614 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f7d11b2-f2cc-4dd0-a9a1-19507b01e23a admin admin] RBAC: Authorization granted {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 16:10:33.659545 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 784/1586] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Aug 22 16:10:33 2022] GET /identity/v3/roles => generated 1420 bytes in 96 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 22 16:10:33.659545 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f7d11b2-f2cc-4dd0-a9a1-19507b01e23a admin admin] Authenticating user token {{(pid=81767) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 16:10:33.668605 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f7d11b2-f2cc-4dd0-a9a1-19507b01e23a admin admin] Validating token access rules against request {{(pid=81767) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 16:10:33.673360 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb057260-9c3e-40a3-baae-49af26cd2d38 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a98b1e4431940ab8d1fc95df2858a6c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81767) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 16:10:33.673979 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb057260-9c3e-40a3-baae-49af26cd2d38 admin admin] REQUEST_METHOD: `POST` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:10:33.674262 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb057260-9c3e-40a3-baae-49af26cd2d38 admin admin] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:10:33.674535 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb057260-9c3e-40a3-baae-49af26cd2d38 admin admin] PATH_INFO: `/v3/users` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:10:33.675160 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb057260-9c3e-40a3-baae-49af26cd2d38 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 16:10:33.677121 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb057260-9c3e-40a3-baae-49af26cd2d38 admin admin] RBAC: Authorization granted {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 16:10:33.765467 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 803/1587] 10.222.0.15 () {60 vars in 1144 bytes} [Mon Aug 22 16:10:33 2022] POST /identity/v3/users => generated 292 bytes in 204 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 22 16:10:33.766321 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12a147c4-9eae-4220-ad6c-00268cdaf2bf admin admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 16:10:33.776602 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12a147c4-9eae-4220-ad6c-00268cdaf2bf admin admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 16:10:33.781127 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9be16800-26d9-4585-8ad0-a3cd0edbf20a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a98b1e4431940ab8d1fc95df2858a6c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 16:10:33.781705 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be16800-26d9-4585-8ad0-a3cd0edbf20a admin admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:10:33.781981 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be16800-26d9-4585-8ad0-a3cd0edbf20a admin admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:10:33.782262 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be16800-26d9-4585-8ad0-a3cd0edbf20a admin admin] PATH_INFO: `/v3/roles` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:10:33.782888 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9be16800-26d9-4585-8ad0-a3cd0edbf20a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 16:10:33.784524 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9be16800-26d9-4585-8ad0-a3cd0edbf20a admin admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 16:10:33.794921 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 804/1588] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Aug 22 16:10:33 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 22 16:10:33.795734 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9be16800-26d9-4585-8ad0-a3cd0edbf20a admin admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 16:10:33.805733 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9be16800-26d9-4585-8ad0-a3cd0edbf20a admin admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 16:10:33.810551 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b546d6f8-f984-4c4d-88b3-ae2a15c09068 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a98b1e4431940ab8d1fc95df2858a6c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 16:10:33.811128 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b546d6f8-f984-4c4d-88b3-ae2a15c09068 admin admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:10:33.811452 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b546d6f8-f984-4c4d-88b3-ae2a15c09068 admin admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:10:33.811726 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b546d6f8-f984-4c4d-88b3-ae2a15c09068 admin admin] PATH_INFO: `/v3/roles` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:10:33.812353 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b546d6f8-f984-4c4d-88b3-ae2a15c09068 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 16:10:33.814027 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b546d6f8-f984-4c4d-88b3-ae2a15c09068 admin admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 16:10:33.824643 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 805/1589] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Aug 22 16:10:33 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 22 16:10:33.825660 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b546d6f8-f984-4c4d-88b3-ae2a15c09068 admin admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 16:10:33.828067 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 785/1590] 10.222.0.15 () {60 vars in 1144 bytes} [Mon Aug 22 16:10:33 2022] POST /identity/v3/users => generated 294 bytes in 171 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 22 16:10:33.829436 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ecae2b8-28e2-4c26-8667-08f08bafa646 None None] REQUEST_METHOD: `POST` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:10:33.837093 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ecae2b8-28e2-4c26-8667-08f08bafa646 None None] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:10:33.837093 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ecae2b8-28e2-4c26-8667-08f08bafa646 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:10:33.837331 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b546d6f8-f984-4c4d-88b3-ae2a15c09068 admin admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 16:10:33.841546 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9997dbb-fc94-4e51-ad84-0b0f93d39c33 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a98b1e4431940ab8d1fc95df2858a6c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 16:10:33.842195 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9997dbb-fc94-4e51-ad84-0b0f93d39c33 admin admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:10:33.842469 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9997dbb-fc94-4e51-ad84-0b0f93d39c33 admin admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:10:33.842834 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9997dbb-fc94-4e51-ad84-0b0f93d39c33 admin admin] PATH_INFO: `/v3/roles` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:10:33.843502 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9997dbb-fc94-4e51-ad84-0b0f93d39c33 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 16:10:33.845088 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9997dbb-fc94-4e51-ad84-0b0f93d39c33 admin admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 16:10:33.852915 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.auth.core [None req-6ecae2b8-28e2-4c26-8667-08f08bafa646 None None] MFA Rules not processed for user `6b2199ae0fb0480aa917ada391f5ec54`. Rule list: `[]` (Enabled: `True`). {{(pid=81767) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 16:10:33.854142 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.fernet_utils [None req-6ecae2b8-28e2-4c26-8667-08f08bafa646 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81767) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 16:10:33.855413 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 806/1591] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Aug 22 16:10:33 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 22 16:10:33.860468 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9997dbb-fc94-4e51-ad84-0b0f93d39c33 admin admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 16:10:33.868909 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 786/1592] 10.222.0.15 () {58 vars in 958 bytes} [Mon Aug 22 16:10:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 16:10:33.868909 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ecae2b8-28e2-4c26-8667-08f08bafa646 None None] Authenticating user token {{(pid=81767) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 16:10:33.871959 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9997dbb-fc94-4e51-ad84-0b0f93d39c33 admin admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 16:10:33.877182 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ecae2b8-28e2-4c26-8667-08f08bafa646 None None] Validating token access rules against request {{(pid=81767) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 16:10:33.877450 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0684f2c0-99b3-4075-abf9-27fa387d3307 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a98b1e4431940ab8d1fc95df2858a6c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 16:10:33.878066 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0684f2c0-99b3-4075-abf9-27fa387d3307 admin admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:10:33.878425 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0684f2c0-99b3-4075-abf9-27fa387d3307 admin admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:10:33.878705 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0684f2c0-99b3-4075-abf9-27fa387d3307 admin admin] PATH_INFO: `/v3/roles` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:10:33.879412 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0684f2c0-99b3-4075-abf9-27fa387d3307 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 16:10:33.881016 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0684f2c0-99b3-4075-abf9-27fa387d3307 admin admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 16:10:33.883692 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f32848be-b94d-4c94-91d5-f44b594b6164 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a98b1e4431940ab8d1fc95df2858a6c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81767) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 16:10:33.884588 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f32848be-b94d-4c94-91d5-f44b594b6164 admin admin] REQUEST_METHOD: `GET` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:10:33.885142 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f32848be-b94d-4c94-91d5-f44b594b6164 admin admin] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:10:33.885621 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f32848be-b94d-4c94-91d5-f44b594b6164 admin admin] PATH_INFO: `/v3/roles` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:10:33.886453 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f32848be-b94d-4c94-91d5-f44b594b6164 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 16:10:33.888232 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f32848be-b94d-4c94-91d5-f44b594b6164 admin admin] RBAC: Authorization granted {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 16:10:33.892544 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 807/1593] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Aug 22 16:10:33 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 22 16:10:33.893369 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0684f2c0-99b3-4075-abf9-27fa387d3307 admin admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 16:10:33.901785 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 787/1594] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Aug 22 16:10:33 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 22 16:10:33.902631 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f32848be-b94d-4c94-91d5-f44b594b6164 admin admin] Authenticating user token {{(pid=81767) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 16:10:33.907774 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0684f2c0-99b3-4075-abf9-27fa387d3307 admin admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 16:10:33.913582 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38ba949f-d55e-4d76-9b78-3423ecd489d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a98b1e4431940ab8d1fc95df2858a6c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 16:10:33.917534 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38ba949f-d55e-4d76-9b78-3423ecd489d7 admin admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:10:33.917534 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38ba949f-d55e-4d76-9b78-3423ecd489d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:10:33.917534 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38ba949f-d55e-4d76-9b78-3423ecd489d7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:10:33.917831 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f32848be-b94d-4c94-91d5-f44b594b6164 admin admin] Validating token access rules against request {{(pid=81767) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 16:10:33.917916 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38ba949f-d55e-4d76-9b78-3423ecd489d7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 16:10:33.917916 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38ba949f-d55e-4d76-9b78-3423ecd489d7 admin admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 16:10:33.921092 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f463d6ca-634b-4119-809f-4c720346ae35 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a98b1e4431940ab8d1fc95df2858a6c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81767) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 16:10:33.921689 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f463d6ca-634b-4119-809f-4c720346ae35 admin admin] REQUEST_METHOD: `GET` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:10:33.921959 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f463d6ca-634b-4119-809f-4c720346ae35 admin admin] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:10:33.922221 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f463d6ca-634b-4119-809f-4c720346ae35 admin admin] PATH_INFO: `/v3/roles` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:10:33.922844 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f463d6ca-634b-4119-809f-4c720346ae35 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 16:10:33.924972 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f463d6ca-634b-4119-809f-4c720346ae35 admin admin] RBAC: Authorization granted {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 16:10:33.928322 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 808/1595] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Aug 22 16:10:33 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 22 16:10:33.929355 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38ba949f-d55e-4d76-9b78-3423ecd489d7 admin admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 16:10:33.943063 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38ba949f-d55e-4d76-9b78-3423ecd489d7 admin admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 16:10:33.946763 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 788/1596] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Aug 22 16:10:33 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 22 16:10:33.947746 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f463d6ca-634b-4119-809f-4c720346ae35 admin admin] Authenticating user token {{(pid=81767) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 16:10:33.951452 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c771d31f-9b81-45a9-a563-5ca3f7f5be93 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a98b1e4431940ab8d1fc95df2858a6c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 16:10:33.952179 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c771d31f-9b81-45a9-a563-5ca3f7f5be93 admin admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:10:33.952457 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c771d31f-9b81-45a9-a563-5ca3f7f5be93 admin admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:10:33.952746 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c771d31f-9b81-45a9-a563-5ca3f7f5be93 admin admin] PATH_INFO: `/v3/roles` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:10:33.953375 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c771d31f-9b81-45a9-a563-5ca3f7f5be93 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 16:10:33.955081 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c771d31f-9b81-45a9-a563-5ca3f7f5be93 admin admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 16:10:33.960981 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f463d6ca-634b-4119-809f-4c720346ae35 admin admin] Validating token access rules against request {{(pid=81767) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 16:10:33.967176 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f9436de-bdaf-4ab8-bdf6-80b387711648 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a98b1e4431940ab8d1fc95df2858a6c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81767) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 16:10:33.968555 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f9436de-bdaf-4ab8-bdf6-80b387711648 admin admin] REQUEST_METHOD: `GET` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:10:33.968903 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f9436de-bdaf-4ab8-bdf6-80b387711648 admin admin] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:10:33.969252 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f9436de-bdaf-4ab8-bdf6-80b387711648 admin admin] PATH_INFO: `/v3/roles` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:10:33.970110 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 809/1597] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Aug 22 16:10:33 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 22 16:10:33.970441 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f9436de-bdaf-4ab8-bdf6-80b387711648 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 16:10:33.971155 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c771d31f-9b81-45a9-a563-5ca3f7f5be93 admin admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 16:10:33.972306 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f9436de-bdaf-4ab8-bdf6-80b387711648 admin admin] RBAC: Authorization granted {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 16:10:33.973471 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-c771d31f-9b81-45a9-a563-5ca3f7f5be93 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 16:10:33.984969 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 789/1598] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Aug 22 16:10:33 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 22 16:10:33.989479 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f9436de-bdaf-4ab8-bdf6-80b387711648 admin admin] Authenticating user token {{(pid=81767) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 16:10:34.002407 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f9436de-bdaf-4ab8-bdf6-80b387711648 admin admin] Validating token access rules against request {{(pid=81767) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 16:10:34.007934 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4809dfe-c309-41e8-bd63-f750a8a40362 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a98b1e4431940ab8d1fc95df2858a6c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81767) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 16:10:34.008600 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4809dfe-c309-41e8-bd63-f750a8a40362 admin admin] REQUEST_METHOD: `PUT` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:10:34.008876 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4809dfe-c309-41e8-bd63-f750a8a40362 admin admin] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:10:34.009179 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4809dfe-c309-41e8-bd63-f750a8a40362 admin admin] PATH_INFO: `/v3/projects/102cf78148744c2aae7ba0c1aff3b3c7/users/a377dda3bddf4346b9b85a2d99228ae1/roles/e6fc1ee3fb974125ba9b4fe55b12b45e` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:10:34.029162 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4809dfe-c309-41e8-bd63-f750a8a40362 admin admin] RBAC: Authorizing `identity:create_grant(project_id=102cf78148744c2aae7ba0c1aff3b3c7, user_id=a377dda3bddf4346b9b85a2d99228ae1, role_id=e6fc1ee3fb974125ba9b4fe55b12b45e)` {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 16:10:34.032807 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4809dfe-c309-41e8-bd63-f750a8a40362 admin admin] RBAC: Authorization granted {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 16:10:34.048812 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=91,keystone:INSERT=9 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 16:10:34.053031 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c771d31f-9b81-45a9-a563-5ca3f7f5be93 admin admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 16:10:34.058502 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a6ad121-42a9-4a7b-b69e-aed0e8bbe688 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '234687adad594f1aa45775a565ffbdc7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4ee0413f9ef4d4fb1f95a660babe090', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 16:10:34.058689 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a6ad121-42a9-4a7b-b69e-aed0e8bbe688 service neutron] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:10:34.058893 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a6ad121-42a9-4a7b-b69e-aed0e8bbe688 service neutron] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:10:34.059168 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a6ad121-42a9-4a7b-b69e-aed0e8bbe688 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:10:34.071407 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a6ad121-42a9-4a7b-b69e-aed0e8bbe688 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 16:10:34.073173 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a6ad121-42a9-4a7b-b69e-aed0e8bbe688 service neutron] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 16:10:34.084291 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 810/1599] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Aug 22 16:10:33 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 16:10:34.085161 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a6ad121-42a9-4a7b-b69e-aed0e8bbe688 service neutron] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 16:10:34.095233 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a6ad121-42a9-4a7b-b69e-aed0e8bbe688 service neutron] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 16:10:34.100277 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0498ac03-a1f0-42d0-b2fd-ad075c51ceef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a98b1e4431940ab8d1fc95df2858a6c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 16:10:34.100962 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0498ac03-a1f0-42d0-b2fd-ad075c51ceef admin admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:10:34.101247 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0498ac03-a1f0-42d0-b2fd-ad075c51ceef admin admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:10:34.101698 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0498ac03-a1f0-42d0-b2fd-ad075c51ceef admin admin] PATH_INFO: `/v3/roles` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:10:34.102360 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0498ac03-a1f0-42d0-b2fd-ad075c51ceef admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 16:10:34.104029 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0498ac03-a1f0-42d0-b2fd-ad075c51ceef admin admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 16:10:34.114832 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 811/1600] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Aug 22 16:10:34 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 22 16:10:34.116618 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0498ac03-a1f0-42d0-b2fd-ad075c51ceef admin admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 16:10:34.128717 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0498ac03-a1f0-42d0-b2fd-ad075c51ceef admin admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 16:10:34.140237 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-972ca28a-a024-4b31-9b08-6e5a2a165c6d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a98b1e4431940ab8d1fc95df2858a6c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 16:10:34.141337 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-972ca28a-a024-4b31-9b08-6e5a2a165c6d admin admin] REQUEST_METHOD: `PUT` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:10:34.141706 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-972ca28a-a024-4b31-9b08-6e5a2a165c6d admin admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:10:34.142054 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-972ca28a-a024-4b31-9b08-6e5a2a165c6d admin admin] PATH_INFO: `/v3/projects/8f6e1f848a244d968c7d172cf065c240/users/7e18979107144f63b3c6157fee263aed/roles/e6fc1ee3fb974125ba9b4fe55b12b45e` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:10:34.160644 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 790/1601] 10.222.0.15 () {60 vars in 1483 bytes} [Mon Aug 22 16:10:33 2022] PUT /identity/v3/projects/102cf78148744c2aae7ba0c1aff3b3c7/users/a377dda3bddf4346b9b85a2d99228ae1/roles/e6fc1ee3fb974125ba9b4fe55b12b45e => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 16:10:34.161950 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4809dfe-c309-41e8-bd63-f750a8a40362 admin admin] Authenticating user token {{(pid=81767) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 16:10:34.173447 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-972ca28a-a024-4b31-9b08-6e5a2a165c6d admin admin] RBAC: Authorizing `identity:create_grant(project_id=8f6e1f848a244d968c7d172cf065c240, user_id=7e18979107144f63b3c6157fee263aed, role_id=e6fc1ee3fb974125ba9b4fe55b12b45e)` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 16:10:34.176886 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-972ca28a-a024-4b31-9b08-6e5a2a165c6d admin admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 16:10:34.245761 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4809dfe-c309-41e8-bd63-f750a8a40362 admin admin] Validating token access rules against request {{(pid=81767) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 16:10:34.250470 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d851bec-dc21-4c48-86cb-07abb06458e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a98b1e4431940ab8d1fc95df2858a6c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81767) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 16:10:34.251133 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d851bec-dc21-4c48-86cb-07abb06458e8 admin admin] REQUEST_METHOD: `PUT` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:10:34.251446 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d851bec-dc21-4c48-86cb-07abb06458e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:10:34.251705 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d851bec-dc21-4c48-86cb-07abb06458e8 admin admin] PATH_INFO: `/v3/projects/aa723f69df45423b88bbed0c958be4b6/users/844f37c1f52e47c692ddabb07d807f39/roles/e6fc1ee3fb974125ba9b4fe55b12b45e` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:10:34.268120 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d851bec-dc21-4c48-86cb-07abb06458e8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=aa723f69df45423b88bbed0c958be4b6, user_id=844f37c1f52e47c692ddabb07d807f39, role_id=e6fc1ee3fb974125ba9b4fe55b12b45e)` {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 16:10:34.270760 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d851bec-dc21-4c48-86cb-07abb06458e8 admin admin] RBAC: Authorization granted {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 16:10:34.483776 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 812/1602] 10.222.0.15 () {60 vars in 1483 bytes} [Mon Aug 22 16:10:34 2022] PUT /identity/v3/projects/8f6e1f848a244d968c7d172cf065c240/users/7e18979107144f63b3c6157fee263aed/roles/e6fc1ee3fb974125ba9b4fe55b12b45e => generated 0 bytes in 303 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 16:10:34.483776 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-972ca28a-a024-4b31-9b08-6e5a2a165c6d admin admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 16:10:34.483776 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-972ca28a-a024-4b31-9b08-6e5a2a165c6d admin admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 16:10:34.488309 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-771394d1-4a0e-4a96-b3cf-548ad21e00b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a98b1e4431940ab8d1fc95df2858a6c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 16:10:34.488947 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-771394d1-4a0e-4a96-b3cf-548ad21e00b5 admin admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:10:34.489224 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-771394d1-4a0e-4a96-b3cf-548ad21e00b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:10:34.489545 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-771394d1-4a0e-4a96-b3cf-548ad21e00b5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:10:34.490191 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-771394d1-4a0e-4a96-b3cf-548ad21e00b5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 16:10:34.491844 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-771394d1-4a0e-4a96-b3cf-548ad21e00b5 admin admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 16:10:34.501816 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 813/1603] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Aug 22 16:10:34 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 22 16:10:34.502673 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-771394d1-4a0e-4a96-b3cf-548ad21e00b5 admin admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 16:10:34.516670 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 791/1604] 10.222.0.15 () {60 vars in 1483 bytes} [Mon Aug 22 16:10:34 2022] PUT /identity/v3/projects/aa723f69df45423b88bbed0c958be4b6/users/844f37c1f52e47c692ddabb07d807f39/roles/e6fc1ee3fb974125ba9b4fe55b12b45e => generated 0 bytes in 355 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 16:10:34.519024 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d851bec-dc21-4c48-86cb-07abb06458e8 admin admin] Authenticating user token {{(pid=81767) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 16:10:34.526655 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-771394d1-4a0e-4a96-b3cf-548ad21e00b5 admin admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 16:10:34.575623 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG dbcounter [-] [81767] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81767) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 16:10:34.614411 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2027224-e465-4e33-8592-665f834b560e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a98b1e4431940ab8d1fc95df2858a6c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 16:10:34.616500 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2027224-e465-4e33-8592-665f834b560e admin admin] REQUEST_METHOD: `PUT` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:10:34.616500 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2027224-e465-4e33-8592-665f834b560e admin admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:10:34.616500 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2027224-e465-4e33-8592-665f834b560e admin admin] PATH_INFO: `/v3/projects/c4b14e0de34d42cba3ac8629f8f21549/users/4a6290a807e44071b1f98e81b83f6a7e/roles/e6fc1ee3fb974125ba9b4fe55b12b45e` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:10:34.624922 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d851bec-dc21-4c48-86cb-07abb06458e8 admin admin] Validating token access rules against request {{(pid=81767) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 16:10:34.629660 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e426f840-4e80-4cc3-962c-1ee0351a61be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a98b1e4431940ab8d1fc95df2858a6c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81767) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 16:10:34.630330 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e426f840-4e80-4cc3-962c-1ee0351a61be admin admin] REQUEST_METHOD: `PUT` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:10:34.630612 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e426f840-4e80-4cc3-962c-1ee0351a61be admin admin] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:10:34.630922 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e426f840-4e80-4cc3-962c-1ee0351a61be admin admin] PATH_INFO: `/v3/projects/847871389fa24926ba5b1608ec48b7d2/users/a293f8ba66b74eb5bd616ce7674e177f/roles/e6fc1ee3fb974125ba9b4fe55b12b45e` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:10:34.633579 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2027224-e465-4e33-8592-665f834b560e admin admin] RBAC: Authorizing `identity:create_grant(project_id=c4b14e0de34d42cba3ac8629f8f21549, user_id=4a6290a807e44071b1f98e81b83f6a7e, role_id=e6fc1ee3fb974125ba9b4fe55b12b45e)` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 16:10:34.636355 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2027224-e465-4e33-8592-665f834b560e admin admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 16:10:34.648055 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e426f840-4e80-4cc3-962c-1ee0351a61be admin admin] RBAC: Authorizing `identity:create_grant(project_id=847871389fa24926ba5b1608ec48b7d2, user_id=a293f8ba66b74eb5bd616ce7674e177f, role_id=e6fc1ee3fb974125ba9b4fe55b12b45e)` {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 16:10:34.650655 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e426f840-4e80-4cc3-962c-1ee0351a61be admin admin] RBAC: Authorization granted {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 16:10:34.741116 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 814/1605] 10.222.0.15 () {60 vars in 1483 bytes} [Mon Aug 22 16:10:34 2022] PUT /identity/v3/projects/c4b14e0de34d42cba3ac8629f8f21549/users/4a6290a807e44071b1f98e81b83f6a7e/roles/e6fc1ee3fb974125ba9b4fe55b12b45e => generated 0 bytes in 238 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 16:10:34.742344 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 792/1606] 10.222.0.15 () {60 vars in 1483 bytes} [Mon Aug 22 16:10:34 2022] PUT /identity/v3/projects/847871389fa24926ba5b1608ec48b7d2/users/a293f8ba66b74eb5bd616ce7674e177f/roles/e6fc1ee3fb974125ba9b4fe55b12b45e => generated 0 bytes in 225 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 16:10:34.742993 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec0aad29-99da-4809-b0f3-15c800e7e411 None None] REQUEST_METHOD: `POST` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:10:34.743472 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec0aad29-99da-4809-b0f3-15c800e7e411 None None] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:10:34.748420 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec0aad29-99da-4809-b0f3-15c800e7e411 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:10:34.748570 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-343d3918-0d5a-4a54-9781-1d7ee3daf332 None None] REQUEST_METHOD: `POST` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:10:34.748570 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-343d3918-0d5a-4a54-9781-1d7ee3daf332 None None] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:10:34.748570 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-343d3918-0d5a-4a54-9781-1d7ee3daf332 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:10:34.768058 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.auth.core [None req-ec0aad29-99da-4809-b0f3-15c800e7e411 None None] MFA Rules not processed for user `6b2199ae0fb0480aa917ada391f5ec54`. Rule list: `[]` (Enabled: `True`). {{(pid=81768) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 16:10:34.769313 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-ec0aad29-99da-4809-b0f3-15c800e7e411 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 16:10:34.777928 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.auth.core [None req-343d3918-0d5a-4a54-9781-1d7ee3daf332 None None] MFA Rules not processed for user `7e18979107144f63b3c6157fee263aed`. Rule list: `[]` (Enabled: `True`). {{(pid=81767) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 16:10:34.779161 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.fernet_utils [None req-343d3918-0d5a-4a54-9781-1d7ee3daf332 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81767) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 16:10:34.821110 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 16:10:34.855780 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 815/1607] 10.222.0.15 () {58 vars in 958 bytes} [Mon Aug 22 16:10:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 16:10:34.857227 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec0aad29-99da-4809-b0f3-15c800e7e411 None None] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 16:10:34.861906 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 793/1608] 10.222.0.15 () {58 vars in 958 bytes} [Mon Aug 22 16:10:34 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 16:10:34.863511 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b377de7-ca6d-480d-8c5b-4f17e19dafa7 None None] REQUEST_METHOD: `POST` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:10:34.864211 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b377de7-ca6d-480d-8c5b-4f17e19dafa7 None None] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:10:34.864708 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b377de7-ca6d-480d-8c5b-4f17e19dafa7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:10:34.884637 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec0aad29-99da-4809-b0f3-15c800e7e411 None None] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 16:10:34.893565 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b717d8bb-88ef-4dc2-9052-9b969f704200 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a98b1e4431940ab8d1fc95df2858a6c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 16:10:34.894602 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b717d8bb-88ef-4dc2-9052-9b969f704200 admin admin] REQUEST_METHOD: `PUT` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:10:34.894888 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b717d8bb-88ef-4dc2-9052-9b969f704200 admin admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:10:34.895163 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b717d8bb-88ef-4dc2-9052-9b969f704200 admin admin] PATH_INFO: `/v3/projects/efecd6320bd24fdf97f753c0230d3b5c/users/dbb8092b0c6d410e93a3f397e282157f/roles/e6fc1ee3fb974125ba9b4fe55b12b45e` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:10:34.908960 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.auth.core [None req-1b377de7-ca6d-480d-8c5b-4f17e19dafa7 None None] MFA Rules not processed for user `6b2199ae0fb0480aa917ada391f5ec54`. Rule list: `[]` (Enabled: `True`). {{(pid=81767) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 16:10:34.910604 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.fernet_utils [None req-1b377de7-ca6d-480d-8c5b-4f17e19dafa7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81767) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 16:10:34.917286 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b717d8bb-88ef-4dc2-9052-9b969f704200 admin admin] RBAC: Authorizing `identity:create_grant(project_id=efecd6320bd24fdf97f753c0230d3b5c, user_id=dbb8092b0c6d410e93a3f397e282157f, role_id=e6fc1ee3fb974125ba9b4fe55b12b45e)` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 16:10:34.919494 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 794/1609] 10.222.0.15 () {58 vars in 958 bytes} [Mon Aug 22 16:10:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 57 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 16:10:34.920114 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b717d8bb-88ef-4dc2-9052-9b969f704200 admin admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 16:10:34.921259 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28301853-870a-4f68-8b61-07f054c1c49e None None] REQUEST_METHOD: `POST` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:10:34.921767 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28301853-870a-4f68-8b61-07f054c1c49e None None] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:10:34.922263 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28301853-870a-4f68-8b61-07f054c1c49e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:10:34.953517 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.auth.core [None req-28301853-870a-4f68-8b61-07f054c1c49e None None] MFA Rules not processed for user `a293f8ba66b74eb5bd616ce7674e177f`. Rule list: `[]` (Enabled: `True`). {{(pid=81767) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 16:10:34.955056 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.fernet_utils [None req-28301853-870a-4f68-8b61-07f054c1c49e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81767) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 16:10:34.984324 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 816/1610] 10.222.0.15 () {60 vars in 1483 bytes} [Mon Aug 22 16:10:34 2022] PUT /identity/v3/projects/efecd6320bd24fdf97f753c0230d3b5c/users/dbb8092b0c6d410e93a3f397e282157f/roles/e6fc1ee3fb974125ba9b4fe55b12b45e => generated 0 bytes in 128 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 16:10:34.986012 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c2d68bc-a7dc-4339-bff3-ea6905a67e1a None None] REQUEST_METHOD: `POST` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:10:34.986468 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c2d68bc-a7dc-4339-bff3-ea6905a67e1a None None] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:10:34.986793 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c2d68bc-a7dc-4339-bff3-ea6905a67e1a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:10:35.026314 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.auth.core [None req-2c2d68bc-a7dc-4339-bff3-ea6905a67e1a None None] MFA Rules not processed for user `4a6290a807e44071b1f98e81b83f6a7e`. Rule list: `[]` (Enabled: `True`). {{(pid=81768) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 16:10:35.028612 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-2c2d68bc-a7dc-4339-bff3-ea6905a67e1a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 16:10:35.066426 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 795/1611] 10.222.0.15 () {58 vars in 958 bytes} [Mon Aug 22 16:10:34 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 16:10:35.067612 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28301853-870a-4f68-8b61-07f054c1c49e None None] Authenticating user token {{(pid=81767) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 16:10:35.110590 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG dbcounter [-] [81767] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81767) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 16:10:35.129348 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 817/1612] 10.222.0.15 () {58 vars in 958 bytes} [Mon Aug 22 16:10:34 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 16:10:35.130940 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-788d16dc-8dc0-4099-997d-90c2beb05e51 None None] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:10:35.131411 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-788d16dc-8dc0-4099-997d-90c2beb05e51 None None] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:10:35.132066 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-788d16dc-8dc0-4099-997d-90c2beb05e51 None None] PATH_INFO: `/` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:10:35.142790 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 818/1613] 10.222.0.15 () {54 vars in 962 bytes} [Mon Aug 22 16:10:35 2022] GET /identity => generated 270 bytes in 13 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 16:10:35.144458 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-788d16dc-8dc0-4099-997d-90c2beb05e51 None None] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 16:10:35.187819 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28301853-870a-4f68-8b61-07f054c1c49e None None] Validating token access rules against request {{(pid=81767) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 16:10:35.231214 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-788d16dc-8dc0-4099-997d-90c2beb05e51 None None] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 16:10:35.236895 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5680df59-13a6-4b1f-af83-81c2a33a1aa2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '234687adad594f1aa45775a565ffbdc7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4ee0413f9ef4d4fb1f95a660babe090', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 16:10:35.237538 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5680df59-13a6-4b1f-af83-81c2a33a1aa2 service neutron] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:10:35.239519 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5680df59-13a6-4b1f-af83-81c2a33a1aa2 service neutron] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:10:35.239826 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5680df59-13a6-4b1f-af83-81c2a33a1aa2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:10:35.254363 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a0e2bbb-1407-43b1-ae04-f349e777c263 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '234687adad594f1aa45775a565ffbdc7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4ee0413f9ef4d4fb1f95a660babe090', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81767) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 16:10:35.255003 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a0e2bbb-1407-43b1-ae04-f349e777c263 service neutron] REQUEST_METHOD: `GET` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:10:35.255351 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a0e2bbb-1407-43b1-ae04-f349e777c263 service neutron] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:10:35.255800 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a0e2bbb-1407-43b1-ae04-f349e777c263 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:10:35.339160 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5680df59-13a6-4b1f-af83-81c2a33a1aa2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 16:10:35.341676 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5680df59-13a6-4b1f-af83-81c2a33a1aa2 service neutron] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 16:10:35.342905 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a0e2bbb-1407-43b1-ae04-f349e777c263 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 16:10:35.344599 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a0e2bbb-1407-43b1-ae04-f349e777c263 service neutron] RBAC: Authorization granted {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 16:10:35.358509 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 796/1614] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Aug 22 16:10:35 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 292 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 16:10:35.360405 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2093563-0984-4f25-ac07-0e70e4add463 None None] REQUEST_METHOD: `POST` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:10:35.360790 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2093563-0984-4f25-ac07-0e70e4add463 None None] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:10:35.361148 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2093563-0984-4f25-ac07-0e70e4add463 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:10:35.362275 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 819/1615] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Aug 22 16:10:35 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 219 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 16:10:35.363673 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26e4a5fe-d223-48d2-bd3d-f32fcfb3a980 None None] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:10:35.363964 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26e4a5fe-d223-48d2-bd3d-f32fcfb3a980 None None] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:10:35.364247 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26e4a5fe-d223-48d2-bd3d-f32fcfb3a980 None None] PATH_INFO: `/` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:10:35.373975 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 820/1616] 10.222.0.15 () {54 vars in 962 bytes} [Mon Aug 22 16:10:35 2022] GET /identity => generated 270 bytes in 11 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 16:10:35.375965 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b0889e8-27e7-4d56-97ea-b4988663d442 None None] REQUEST_METHOD: `POST` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:10:35.376253 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b0889e8-27e7-4d56-97ea-b4988663d442 None None] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:10:35.376530 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b0889e8-27e7-4d56-97ea-b4988663d442 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:10:35.394786 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.auth.core [None req-d2093563-0984-4f25-ac07-0e70e4add463 None None] MFA Rules not processed for user `6b2199ae0fb0480aa917ada391f5ec54`. Rule list: `[]` (Enabled: `True`). {{(pid=81767) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 16:10:35.396137 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.fernet_utils [None req-d2093563-0984-4f25-ac07-0e70e4add463 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81767) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 16:10:35.422139 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 797/1617] 10.222.0.15 () {58 vars in 958 bytes} [Mon Aug 22 16:10:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 63 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 16:10:35.423309 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-920f8af8-171a-4e28-ae0c-3e93768d300e None None] REQUEST_METHOD: `POST` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:10:35.425710 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.auth.core [None req-0b0889e8-27e7-4d56-97ea-b4988663d442 None None] MFA Rules not processed for user `e4ef569e41fb46d0b721bb1c7663616f`. Rule list: `[]` (Enabled: `True`). {{(pid=81768) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 16:10:35.426057 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-920f8af8-171a-4e28-ae0c-3e93768d300e None None] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:10:35.426336 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-920f8af8-171a-4e28-ae0c-3e93768d300e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:10:35.428968 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-0b0889e8-27e7-4d56-97ea-b4988663d442 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 16:10:35.469828 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.auth.core [None req-920f8af8-171a-4e28-ae0c-3e93768d300e None None] MFA Rules not processed for user `e4ef569e41fb46d0b721bb1c7663616f`. Rule list: `[]` (Enabled: `True`). {{(pid=81767) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 16:10:35.471750 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.fernet_utils [None req-920f8af8-171a-4e28-ae0c-3e93768d300e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81767) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 16:10:35.532517 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 821/1618] 10.222.0.15 () {58 vars in 1059 bytes} [Mon Aug 22 16:10:35 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 16:10:35.533488 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b0889e8-27e7-4d56-97ea-b4988663d442 None None] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 16:10:35.551568 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b0889e8-27e7-4d56-97ea-b4988663d442 None None] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 16:10:35.564276 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40b29bf0-ee96-49b7-90ca-849d00850ca9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '234687adad594f1aa45775a565ffbdc7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4ee0413f9ef4d4fb1f95a660babe090', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 16:10:35.564915 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40b29bf0-ee96-49b7-90ca-849d00850ca9 service neutron] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:10:35.565193 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40b29bf0-ee96-49b7-90ca-849d00850ca9 service neutron] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:10:35.565463 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40b29bf0-ee96-49b7-90ca-849d00850ca9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:10:35.580397 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 798/1619] 10.222.0.15 () {58 vars in 1059 bytes} [Mon Aug 22 16:10:35 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 16:10:35.582357 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-920f8af8-171a-4e28-ae0c-3e93768d300e None None] Authenticating user token {{(pid=81767) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 16:10:35.588490 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40b29bf0-ee96-49b7-90ca-849d00850ca9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 16:10:35.589976 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40b29bf0-ee96-49b7-90ca-849d00850ca9 service neutron] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 16:10:35.603929 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-920f8af8-171a-4e28-ae0c-3e93768d300e None None] Validating token access rules against request {{(pid=81767) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 16:10:35.611011 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 822/1620] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Aug 22 16:10:35 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 78 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 16:10:35.616467 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40b29bf0-ee96-49b7-90ca-849d00850ca9 service neutron] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 16:10:35.616725 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97114696-b868-4ea3-b633-d3c669e3dc79 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4ef569e41fb46d0b721bb1c7663616f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4ee0413f9ef4d4fb1f95a660babe090', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81767) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 16:10:35.616725 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97114696-b868-4ea3-b633-d3c669e3dc79 service nova] REQUEST_METHOD: `GET` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:10:35.616725 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97114696-b868-4ea3-b633-d3c669e3dc79 service nova] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:10:35.616725 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97114696-b868-4ea3-b633-d3c669e3dc79 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:10:35.640143 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40b29bf0-ee96-49b7-90ca-849d00850ca9 service neutron] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 16:10:35.656602 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0558fe10-23dd-47da-b49d-7db439eafe97 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4ef569e41fb46d0b721bb1c7663616f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4ee0413f9ef4d4fb1f95a660babe090', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 16:10:35.657225 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0558fe10-23dd-47da-b49d-7db439eafe97 service nova] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:10:35.657499 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0558fe10-23dd-47da-b49d-7db439eafe97 service nova] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:10:35.657769 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0558fe10-23dd-47da-b49d-7db439eafe97 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:10:35.694002 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97114696-b868-4ea3-b633-d3c669e3dc79 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 16:10:35.696173 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97114696-b868-4ea3-b633-d3c669e3dc79 service nova] RBAC: Authorization granted {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 16:10:35.720178 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 16:10:35.720471 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 799/1621] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Aug 22 16:10:35 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 139 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 16:10:35.770412 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0558fe10-23dd-47da-b49d-7db439eafe97 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 16:10:35.772581 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0558fe10-23dd-47da-b49d-7db439eafe97 service nova] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 16:10:35.797507 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 823/1622] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Aug 22 16:10:35 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 186 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 16:10:36.055850 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97114696-b868-4ea3-b633-d3c669e3dc79 service nova] Authenticating user token {{(pid=81767) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 16:10:36.074872 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97114696-b868-4ea3-b633-d3c669e3dc79 service nova] Validating token access rules against request {{(pid=81767) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 16:10:36.086674 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfa1af0b-0bc4-4ca6-ae31-f5d0cbe1a8be service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4ef569e41fb46d0b721bb1c7663616f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4ee0413f9ef4d4fb1f95a660babe090', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81767) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 16:10:36.087685 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfa1af0b-0bc4-4ca6-ae31-f5d0cbe1a8be service nova] REQUEST_METHOD: `GET` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:10:36.087871 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfa1af0b-0bc4-4ca6-ae31-f5d0cbe1a8be service nova] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:10:36.088094 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfa1af0b-0bc4-4ca6-ae31-f5d0cbe1a8be service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:10:36.377839 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfa1af0b-0bc4-4ca6-ae31-f5d0cbe1a8be service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 16:10:36.377839 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfa1af0b-0bc4-4ca6-ae31-f5d0cbe1a8be service nova] RBAC: Authorization granted {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 16:10:36.377839 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 800/1623] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Aug 22 16:10:36 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 16:10:36.384215 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44c584ab-3aa8-446a-b68b-38335d6bbce5 None None] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:10:36.384215 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44c584ab-3aa8-446a-b68b-38335d6bbce5 None None] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:10:36.384215 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44c584ab-3aa8-446a-b68b-38335d6bbce5 None None] PATH_INFO: `/` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:10:36.384215 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 824/1624] 10.222.0.15 () {54 vars in 912 bytes} [Mon Aug 22 16:10:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 16:10:36.386701 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8970ea3-1cbc-45ba-a148-5a59c6de1407 None None] REQUEST_METHOD: `POST` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:10:36.386701 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8970ea3-1cbc-45ba-a148-5a59c6de1407 None None] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:10:36.386701 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8970ea3-1cbc-45ba-a148-5a59c6de1407 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:10:36.386701 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.auth.core [None req-c8970ea3-1cbc-45ba-a148-5a59c6de1407 None None] MFA Rules not processed for user `e4ef569e41fb46d0b721bb1c7663616f`. Rule list: `[]` (Enabled: `True`). {{(pid=81767) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 16:10:36.386701 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.fernet_utils [None req-c8970ea3-1cbc-45ba-a148-5a59c6de1407 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81767) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 16:10:36.387512 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ce18128-dc34-4701-b0e6-c1cb6bb35f24 None None] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:10:36.387512 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ce18128-dc34-4701-b0e6-c1cb6bb35f24 None None] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:10:36.387512 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ce18128-dc34-4701-b0e6-c1cb6bb35f24 None None] PATH_INFO: `/` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:10:36.387512 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 825/1625] 10.222.0.15 () {54 vars in 912 bytes} [Mon Aug 22 16:10:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 16:10:36.387512 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff4fc9d0-c54c-42db-b5ee-f25ba55d2360 None None] REQUEST_METHOD: `POST` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:10:36.387512 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff4fc9d0-c54c-42db-b5ee-f25ba55d2360 None None] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:10:36.390457 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff4fc9d0-c54c-42db-b5ee-f25ba55d2360 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:10:36.390998 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 801/1626] 10.222.0.15 () {58 vars in 1009 bytes} [Mon Aug 22 16:10:36 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 75 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 16:10:36.390998 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8970ea3-1cbc-45ba-a148-5a59c6de1407 None None] Authenticating user token {{(pid=81767) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 16:10:36.391572 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.auth.core [None req-ff4fc9d0-c54c-42db-b5ee-f25ba55d2360 None None] MFA Rules not processed for user `e4ef569e41fb46d0b721bb1c7663616f`. Rule list: `[]` (Enabled: `True`). {{(pid=81768) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 16:10:36.391572 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-ff4fc9d0-c54c-42db-b5ee-f25ba55d2360 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 16:10:36.391572 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 826/1627] 10.222.0.15 () {58 vars in 1009 bytes} [Mon Aug 22 16:10:36 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 74 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 16:10:36.391572 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff4fc9d0-c54c-42db-b5ee-f25ba55d2360 None None] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 16:10:36.392355 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8970ea3-1cbc-45ba-a148-5a59c6de1407 None None] Validating token access rules against request {{(pid=81767) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 16:10:36.392355 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dd50e07-facf-4652-8dac-f7bddedc3cb7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79706d480f864998b9f8594be407ceab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4ee0413f9ef4d4fb1f95a660babe090', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81767) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 16:10:36.392355 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dd50e07-facf-4652-8dac-f7bddedc3cb7 service cinder] REQUEST_METHOD: `GET` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:10:36.392355 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dd50e07-facf-4652-8dac-f7bddedc3cb7 service cinder] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:10:36.392700 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dd50e07-facf-4652-8dac-f7bddedc3cb7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:10:36.407794 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dd50e07-facf-4652-8dac-f7bddedc3cb7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 16:10:36.412255 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dd50e07-facf-4652-8dac-f7bddedc3cb7 service cinder] RBAC: Authorization granted {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 16:10:36.451154 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff4fc9d0-c54c-42db-b5ee-f25ba55d2360 None None] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 16:10:36.454740 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 802/1628] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Aug 22 16:10:36 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 194 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 22 16:10:36.464030 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-642a977b-11e4-44ed-8057-d4850c887eee service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79706d480f864998b9f8594be407ceab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4ee0413f9ef4d4fb1f95a660babe090', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 16:10:36.465122 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-642a977b-11e4-44ed-8057-d4850c887eee service cinder] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:10:36.465547 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-642a977b-11e4-44ed-8057-d4850c887eee service cinder] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:10:36.465948 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-642a977b-11e4-44ed-8057-d4850c887eee service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:10:36.490484 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-642a977b-11e4-44ed-8057-d4850c887eee service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 16:10:36.492572 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-642a977b-11e4-44ed-8057-d4850c887eee service cinder] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 16:10:36.519850 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 827/1629] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Aug 22 16:10:36 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 217 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 16:10:37.135502 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG dbcounter [-] [81767] Writing DB stats keystone:SELECT=100 {{(pid=81767) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 16:10:37.954817 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=100 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 16:10:38.425358 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dd50e07-facf-4652-8dac-f7bddedc3cb7 service cinder] Authenticating user token {{(pid=81767) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 16:10:38.438796 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dd50e07-facf-4652-8dac-f7bddedc3cb7 service cinder] Validating token access rules against request {{(pid=81767) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 16:10:38.446759 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb407a6c-f324-435d-a5d4-231070f15d8c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a98b1e4431940ab8d1fc95df2858a6c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81767) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 16:10:38.447389 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb407a6c-f324-435d-a5d4-231070f15d8c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:10:38.448396 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb407a6c-f324-435d-a5d4-231070f15d8c admin admin] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:10:38.448753 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb407a6c-f324-435d-a5d4-231070f15d8c admin admin] PATH_INFO: `/v3/users/7e18979107144f63b3c6157fee263aed` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:10:38.450341 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb407a6c-f324-435d-a5d4-231070f15d8c admin admin] RBAC: Authorizing `identity:delete_user(user_id=7e18979107144f63b3c6157fee263aed)` {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 16:10:38.451552 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: WARNING py.warnings [None req-fb407a6c-f324-435d-a5d4-231070f15d8c admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 22 16:10:38.451552 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: warnings.warn(msg) Aug 22 16:10:38.451552 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: Aug 22 16:10:38.453682 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb407a6c-f324-435d-a5d4-231070f15d8c admin admin] RBAC: Authorization granted {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 16:10:38.774816 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.notifications [None req-fb407a6c-f324-435d-a5d4-231070f15d8c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7e18979107144f63b3c6157fee263aed'} {{(pid=81767) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 16:10:38.776247 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.notifications [None req-fb407a6c-f324-435d-a5d4-231070f15d8c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7e18979107144f63b3c6157fee263aed'} {{(pid=81767) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 16:10:38.864581 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.notifications [None req-fb407a6c-f324-435d-a5d4-231070f15d8c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7e18979107144f63b3c6157fee263aed'} {{(pid=81767) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 16:10:38.902940 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.notifications [None req-fb407a6c-f324-435d-a5d4-231070f15d8c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7e18979107144f63b3c6157fee263aed'} {{(pid=81767) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 16:10:38.950964 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.notifications [None req-fb407a6c-f324-435d-a5d4-231070f15d8c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7e18979107144f63b3c6157fee263aed'} {{(pid=81767) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 16:10:38.961978 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 803/1630] 10.222.0.15 () {58 vars in 1224 bytes} [Mon Aug 22 16:10:38 2022] DELETE /identity/v3/users/7e18979107144f63b3c6157fee263aed => generated 0 bytes in 538 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 16:10:38.965132 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc653894-6fa2-457d-ab4b-2491708b8187 None None] REQUEST_METHOD: `POST` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:10:38.965561 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc653894-6fa2-457d-ab4b-2491708b8187 None None] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:10:38.966260 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc653894-6fa2-457d-ab4b-2491708b8187 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:10:38.992459 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.auth.core [None req-dc653894-6fa2-457d-ab4b-2491708b8187 None None] MFA Rules not processed for user `6b2199ae0fb0480aa917ada391f5ec54`. Rule list: `[]` (Enabled: `True`). {{(pid=81768) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 16:10:38.994067 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-dc653894-6fa2-457d-ab4b-2491708b8187 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 16:10:39.033694 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9edea5a2-69f6-46d4-8e3a-e182baaccb96 None None] REQUEST_METHOD: `POST` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:10:39.034386 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9edea5a2-69f6-46d4-8e3a-e182baaccb96 None None] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:10:39.034620 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9edea5a2-69f6-46d4-8e3a-e182baaccb96 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:10:39.066546 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG dbcounter [-] [81767] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81767) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 16:10:39.110879 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.auth.core [None req-9edea5a2-69f6-46d4-8e3a-e182baaccb96 None None] MFA Rules not processed for user `844f37c1f52e47c692ddabb07d807f39`. Rule list: `[]` (Enabled: `True`). {{(pid=81767) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 16:10:39.113143 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.fernet_utils [None req-9edea5a2-69f6-46d4-8e3a-e182baaccb96 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81767) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 16:10:39.209922 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 828/1631] 10.222.0.15 () {58 vars in 958 bytes} [Mon Aug 22 16:10:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 246 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 16:10:39.257287 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc653894-6fa2-457d-ab4b-2491708b8187 None None] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 16:10:39.260486 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-dc653894-6fa2-457d-ab4b-2491708b8187 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 16:10:39.290278 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 804/1632] 10.222.0.15 () {58 vars in 958 bytes} [Mon Aug 22 16:10:39 2022] POST /identity/v3/auth/tokens => generated 3280 bytes in 259 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 16:10:39.299517 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9edea5a2-69f6-46d4-8e3a-e182baaccb96 None None] Authenticating user token {{(pid=81767) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 16:10:39.301143 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.fernet_utils [None req-9edea5a2-69f6-46d4-8e3a-e182baaccb96 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81767) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 16:10:39.329951 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9edea5a2-69f6-46d4-8e3a-e182baaccb96 None None] Validating token access rules against request {{(pid=81767) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 16:10:39.341870 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4353dc0a-2a39-4a1a-9e32-23ed8354ef23 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a98b1e4431940ab8d1fc95df2858a6c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81767) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 16:10:39.343107 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4353dc0a-2a39-4a1a-9e32-23ed8354ef23 admin admin] REQUEST_METHOD: `POST` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:10:39.343431 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4353dc0a-2a39-4a1a-9e32-23ed8354ef23 admin admin] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:10:39.343750 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4353dc0a-2a39-4a1a-9e32-23ed8354ef23 admin admin] PATH_INFO: `/v3/projects` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:10:39.344816 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4353dc0a-2a39-4a1a-9e32-23ed8354ef23 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 16:10:39.348121 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4353dc0a-2a39-4a1a-9e32-23ed8354ef23 admin admin] RBAC: Authorization granted {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 16:10:39.423514 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc653894-6fa2-457d-ab4b-2491708b8187 None None] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 16:10:39.429366 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd67bb63-b3b0-40c4-bbc6-232bc21afd25 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '234687adad594f1aa45775a565ffbdc7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4ee0413f9ef4d4fb1f95a660babe090', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 16:10:39.438053 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd67bb63-b3b0-40c4-bbc6-232bc21afd25 service neutron] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:10:39.438053 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd67bb63-b3b0-40c4-bbc6-232bc21afd25 service neutron] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:10:39.438053 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd67bb63-b3b0-40c4-bbc6-232bc21afd25 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:10:39.445683 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd67bb63-b3b0-40c4-bbc6-232bc21afd25 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 16:10:39.448640 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd67bb63-b3b0-40c4-bbc6-232bc21afd25 service neutron] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 16:10:39.469248 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 805/1633] 10.222.0.15 () {60 vars in 1153 bytes} [Mon Aug 22 16:10:39 2022] POST /identity/v3/projects => generated 403 bytes in 170 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 22 16:10:39.473922 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4353dc0a-2a39-4a1a-9e32-23ed8354ef23 admin admin] Authenticating user token {{(pid=81767) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 16:10:39.486762 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 829/1634] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Aug 22 16:10:39 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 230 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 16:10:39.589984 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4353dc0a-2a39-4a1a-9e32-23ed8354ef23 admin admin] Validating token access rules against request {{(pid=81767) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 16:10:39.598978 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3fac052-4b03-4a15-bf50-dba87640fd08 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a98b1e4431940ab8d1fc95df2858a6c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81767) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 16:10:39.600480 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3fac052-4b03-4a15-bf50-dba87640fd08 admin admin] REQUEST_METHOD: `POST` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:10:39.603115 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3fac052-4b03-4a15-bf50-dba87640fd08 admin admin] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:10:39.604064 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3fac052-4b03-4a15-bf50-dba87640fd08 admin admin] PATH_INFO: `/v3/users` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:10:39.609320 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3fac052-4b03-4a15-bf50-dba87640fd08 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 16:10:39.612454 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3fac052-4b03-4a15-bf50-dba87640fd08 admin admin] RBAC: Authorization granted {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 16:10:39.756137 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 806/1635] 10.222.0.15 () {60 vars in 1144 bytes} [Mon Aug 22 16:10:39 2022] POST /identity/v3/users => generated 308 bytes in 280 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 22 16:10:39.757831 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd67bb63-b3b0-40c4-bbc6-232bc21afd25 service neutron] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 16:10:39.774098 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd67bb63-b3b0-40c4-bbc6-232bc21afd25 service neutron] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 16:10:39.794234 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40f48ff8-129b-4eda-b899-be1ebaf90a43 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a98b1e4431940ab8d1fc95df2858a6c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 16:10:39.795088 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f48ff8-129b-4eda-b899-be1ebaf90a43 admin admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:10:39.795579 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f48ff8-129b-4eda-b899-be1ebaf90a43 admin admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:10:39.796005 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f48ff8-129b-4eda-b899-be1ebaf90a43 admin admin] PATH_INFO: `/v3/roles` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:10:39.796900 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40f48ff8-129b-4eda-b899-be1ebaf90a43 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 16:10:39.798871 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40f48ff8-129b-4eda-b899-be1ebaf90a43 admin admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 16:10:39.817818 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 830/1636] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Aug 22 16:10:39 2022] GET /identity/v3/roles => generated 1420 bytes in 60 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 22 16:10:39.818919 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3fac052-4b03-4a15-bf50-dba87640fd08 admin admin] Authenticating user token {{(pid=81767) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 16:10:39.833149 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3fac052-4b03-4a15-bf50-dba87640fd08 admin admin] Validating token access rules against request {{(pid=81767) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 16:10:39.847595 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5975b10a-8ec1-4891-906d-e655654ab0ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a98b1e4431940ab8d1fc95df2858a6c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81767) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 16:10:39.848403 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5975b10a-8ec1-4891-906d-e655654ab0ba admin admin] REQUEST_METHOD: `PUT` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:10:39.848762 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5975b10a-8ec1-4891-906d-e655654ab0ba admin admin] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:10:39.849035 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5975b10a-8ec1-4891-906d-e655654ab0ba admin admin] PATH_INFO: `/v3/projects/db211d338fe64470831d2b171e5fd88e/users/c3a0c958112f44d28aa13df75d5612cc/roles/9d9848cfd22e4b089db443c350e56670` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:10:39.891550 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5975b10a-8ec1-4891-906d-e655654ab0ba admin admin] RBAC: Authorizing `identity:create_grant(project_id=db211d338fe64470831d2b171e5fd88e, user_id=c3a0c958112f44d28aa13df75d5612cc, role_id=9d9848cfd22e4b089db443c350e56670)` {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 16:10:39.898211 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5975b10a-8ec1-4891-906d-e655654ab0ba admin admin] RBAC: Authorization granted {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 16:10:40.007310 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40f48ff8-129b-4eda-b899-be1ebaf90a43 admin admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 16:10:40.017089 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 807/1637] 10.222.0.15 () {60 vars in 1483 bytes} [Mon Aug 22 16:10:39 2022] PUT /identity/v3/projects/db211d338fe64470831d2b171e5fd88e/users/c3a0c958112f44d28aa13df75d5612cc/roles/9d9848cfd22e4b089db443c350e56670 => generated 0 bytes in 185 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 22 16:10:40.029392 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40f48ff8-129b-4eda-b899-be1ebaf90a43 admin admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 16:10:40.150809 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f12e115b-5c90-47c1-8e5f-4bed846605b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a98b1e4431940ab8d1fc95df2858a6c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 16:10:40.151626 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f12e115b-5c90-47c1-8e5f-4bed846605b5 admin admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:10:40.152006 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f12e115b-5c90-47c1-8e5f-4bed846605b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:10:40.154576 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f12e115b-5c90-47c1-8e5f-4bed846605b5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:10:40.156822 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f12e115b-5c90-47c1-8e5f-4bed846605b5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 16:10:40.168364 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f12e115b-5c90-47c1-8e5f-4bed846605b5 admin admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 16:10:40.197444 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5975b10a-8ec1-4891-906d-e655654ab0ba admin admin] Authenticating user token {{(pid=81767) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 16:10:40.210628 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5975b10a-8ec1-4891-906d-e655654ab0ba admin admin] Validating token access rules against request {{(pid=81767) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 16:10:40.214958 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 831/1638] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Aug 22 16:10:40 2022] GET /identity/v3/roles => generated 1420 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 22 16:10:40.234272 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5a67cf8-d9c3-4670-b1b8-bca34de9c35a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a98b1e4431940ab8d1fc95df2858a6c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81767) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 16:10:40.235404 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5a67cf8-d9c3-4670-b1b8-bca34de9c35a admin admin] REQUEST_METHOD: `PUT` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:10:40.235831 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5a67cf8-d9c3-4670-b1b8-bca34de9c35a admin admin] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:10:40.236118 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5a67cf8-d9c3-4670-b1b8-bca34de9c35a admin admin] PATH_INFO: `/v3/projects/db211d338fe64470831d2b171e5fd88e/users/c3a0c958112f44d28aa13df75d5612cc/roles/9d9848cfd22e4b089db443c350e56670` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:10:40.244592 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5a67cf8-d9c3-4670-b1b8-bca34de9c35a admin admin] RBAC: Authorizing `identity:create_grant(project_id=db211d338fe64470831d2b171e5fd88e, user_id=c3a0c958112f44d28aa13df75d5612cc, role_id=9d9848cfd22e4b089db443c350e56670)` {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 16:10:40.263149 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5a67cf8-d9c3-4670-b1b8-bca34de9c35a admin admin] RBAC: Authorization granted {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 16:10:40.315448 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 808/1639] 10.222.0.15 () {60 vars in 1483 bytes} [Mon Aug 22 16:10:40 2022] PUT /identity/v3/projects/db211d338fe64470831d2b171e5fd88e/users/c3a0c958112f44d28aa13df75d5612cc/roles/9d9848cfd22e4b089db443c350e56670 => generated 0 bytes in 114 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 16:10:40.811062 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f12e115b-5c90-47c1-8e5f-4bed846605b5 admin admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 16:10:40.812861 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-f12e115b-5c90-47c1-8e5f-4bed846605b5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 16:10:40.833723 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f12e115b-5c90-47c1-8e5f-4bed846605b5 admin admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 16:10:40.887361 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=100 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 16:10:41.036366 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2be36b52-9c3a-44dc-8d64-56785d9c22d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a98b1e4431940ab8d1fc95df2858a6c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 16:10:41.039570 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2be36b52-9c3a-44dc-8d64-56785d9c22d3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:10:41.040511 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2be36b52-9c3a-44dc-8d64-56785d9c22d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:10:41.042642 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2be36b52-9c3a-44dc-8d64-56785d9c22d3 admin admin] PATH_INFO: `/v3/projects/8f6e1f848a244d968c7d172cf065c240` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:10:41.046725 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2be36b52-9c3a-44dc-8d64-56785d9c22d3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8f6e1f848a244d968c7d172cf065c240)` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 16:10:41.052462 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: WARNING py.warnings [None req-2be36b52-9c3a-44dc-8d64-56785d9c22d3 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 22 16:10:41.052462 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: warnings.warn(msg) Aug 22 16:10:41.052462 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: Aug 22 16:10:41.058882 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2be36b52-9c3a-44dc-8d64-56785d9c22d3 admin admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 16:10:41.368486 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-2be36b52-9c3a-44dc-8d64-56785d9c22d3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8f6e1f848a244d968c7d172cf065c240', 'request_id': 'req-2be36b52-9c3a-44dc-8d64-56785d9c22d3'} {{(pid=81768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 16:10:41.384769 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-2be36b52-9c3a-44dc-8d64-56785d9c22d3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8f6e1f848a244d968c7d172cf065c240', 'request_id': 'req-2be36b52-9c3a-44dc-8d64-56785d9c22d3'} {{(pid=81768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 16:10:41.518477 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-2be36b52-9c3a-44dc-8d64-56785d9c22d3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8f6e1f848a244d968c7d172cf065c240', 'request_id': 'req-2be36b52-9c3a-44dc-8d64-56785d9c22d3'} {{(pid=81768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 16:10:41.580927 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-2be36b52-9c3a-44dc-8d64-56785d9c22d3 admin admin] The token cache is being invalidate because project 8f6e1f848a244d968c7d172cf065c240 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81768) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 22 16:10:41.585249 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-2be36b52-9c3a-44dc-8d64-56785d9c22d3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 16:10:41.589530 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-2be36b52-9c3a-44dc-8d64-56785d9c22d3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 16:10:41.603773 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 832/1640] 10.222.0.15 () {58 vars in 1233 bytes} [Mon Aug 22 16:10:40 2022] DELETE /identity/v3/projects/8f6e1f848a244d968c7d172cf065c240 => generated 0 bytes in 792 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 16:10:41.609567 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-272e947e-0a05-4440-b678-c4146229aba9 None None] REQUEST_METHOD: `POST` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:10:41.610452 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-272e947e-0a05-4440-b678-c4146229aba9 None None] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:10:41.612177 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-272e947e-0a05-4440-b678-c4146229aba9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:10:41.679944 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.auth.core [None req-272e947e-0a05-4440-b678-c4146229aba9 None None] MFA Rules not processed for user `6b2199ae0fb0480aa917ada391f5ec54`. Rule list: `[]` (Enabled: `True`). {{(pid=81767) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 16:10:41.681450 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.fernet_utils [None req-272e947e-0a05-4440-b678-c4146229aba9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81767) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 16:10:41.733796 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG dbcounter [-] [81767] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81767) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 16:10:41.894771 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be0a2ec6-0242-4797-b831-bd74852d10b4 None None] REQUEST_METHOD: `POST` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:10:41.894771 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be0a2ec6-0242-4797-b831-bd74852d10b4 None None] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:10:41.894771 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be0a2ec6-0242-4797-b831-bd74852d10b4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:10:41.898800 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 809/1641] 10.222.0.15 () {58 vars in 958 bytes} [Mon Aug 22 16:10:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 289 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 16:10:41.932705 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.auth.core [None req-be0a2ec6-0242-4797-b831-bd74852d10b4 None None] MFA Rules not processed for user `6b2199ae0fb0480aa917ada391f5ec54`. Rule list: `[]` (Enabled: `True`). {{(pid=81768) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 16:10:41.937225 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-be0a2ec6-0242-4797-b831-bd74852d10b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 16:10:41.964273 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-272e947e-0a05-4440-b678-c4146229aba9 None None] Authenticating user token {{(pid=81767) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 16:10:41.965118 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 833/1642] 10.222.0.15 () {58 vars in 958 bytes} [Mon Aug 22 16:10:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 71 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 16:10:41.989828 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-272e947e-0a05-4440-b678-c4146229aba9 None None] Validating token access rules against request {{(pid=81767) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 16:10:41.992164 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e903f7c-3968-4425-b175-ebefb0d68cc7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a98b1e4431940ab8d1fc95df2858a6c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81767) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 16:10:41.993054 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e903f7c-3968-4425-b175-ebefb0d68cc7 admin admin] REQUEST_METHOD: `GET` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:10:41.993579 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e903f7c-3968-4425-b175-ebefb0d68cc7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:10:41.994505 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e903f7c-3968-4425-b175-ebefb0d68cc7 admin admin] PATH_INFO: `/v3/domains` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:10:41.995427 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e903f7c-3968-4425-b175-ebefb0d68cc7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81767) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 16:10:41.996554 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e903f7c-3968-4425-b175-ebefb0d68cc7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 16:10:42.002102 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e903f7c-3968-4425-b175-ebefb0d68cc7 admin admin] RBAC: Authorization granted {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 16:10:42.036418 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 810/1643] 10.222.0.15 () {58 vars in 1153 bytes} [Mon Aug 22 16:10:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 69 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 16:10:42.039194 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be0a2ec6-0242-4797-b831-bd74852d10b4 None None] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 16:10:42.056060 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be0a2ec6-0242-4797-b831-bd74852d10b4 None None] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 16:10:42.069831 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-250ffff9-62f3-4dbf-b3d7-4a5038d505a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a98b1e4431940ab8d1fc95df2858a6c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 16:10:42.070638 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-250ffff9-62f3-4dbf-b3d7-4a5038d505a9 admin admin] REQUEST_METHOD: `POST` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:10:42.071015 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-250ffff9-62f3-4dbf-b3d7-4a5038d505a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:10:42.071416 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-250ffff9-62f3-4dbf-b3d7-4a5038d505a9 admin admin] PATH_INFO: `/v3/projects` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:10:42.072159 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-250ffff9-62f3-4dbf-b3d7-4a5038d505a9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 16:10:42.074278 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-250ffff9-62f3-4dbf-b3d7-4a5038d505a9 admin admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 16:10:42.116062 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e903f7c-3968-4425-b175-ebefb0d68cc7 admin admin] Authenticating user token {{(pid=81767) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 16:10:42.118254 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.fernet_utils [None req-2e903f7c-3968-4425-b175-ebefb0d68cc7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81767) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 16:10:42.142387 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e903f7c-3968-4425-b175-ebefb0d68cc7 admin admin] Validating token access rules against request {{(pid=81767) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 16:10:42.156327 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03ea14a2-2a7b-4f5c-86d1-92764a1f6351 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a98b1e4431940ab8d1fc95df2858a6c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81767) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 16:10:42.157272 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03ea14a2-2a7b-4f5c-86d1-92764a1f6351 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:10:42.157728 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03ea14a2-2a7b-4f5c-86d1-92764a1f6351 admin admin] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:10:42.158194 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03ea14a2-2a7b-4f5c-86d1-92764a1f6351 admin admin] PATH_INFO: `/v3/users/4a6290a807e44071b1f98e81b83f6a7e` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:10:42.169426 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03ea14a2-2a7b-4f5c-86d1-92764a1f6351 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4a6290a807e44071b1f98e81b83f6a7e)` {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 16:10:42.172110 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03ea14a2-2a7b-4f5c-86d1-92764a1f6351 admin admin] RBAC: Authorization granted {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 16:10:42.228413 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 834/1644] 10.222.0.15 () {60 vars in 1153 bytes} [Mon Aug 22 16:10:42 2022] POST /identity/v3/projects => generated 387 bytes in 190 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 22 16:10:42.229647 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-250ffff9-62f3-4dbf-b3d7-4a5038d505a9 admin admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 16:10:42.325693 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-250ffff9-62f3-4dbf-b3d7-4a5038d505a9 admin admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 16:10:42.331606 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8066468a-f377-4b64-975f-183cc68b1b65 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a98b1e4431940ab8d1fc95df2858a6c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 16:10:42.332548 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8066468a-f377-4b64-975f-183cc68b1b65 admin admin] REQUEST_METHOD: `POST` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:10:42.340625 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8066468a-f377-4b64-975f-183cc68b1b65 admin admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:10:42.340921 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8066468a-f377-4b64-975f-183cc68b1b65 admin admin] PATH_INFO: `/v3/users` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:10:42.342143 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8066468a-f377-4b64-975f-183cc68b1b65 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 16:10:42.344454 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8066468a-f377-4b64-975f-183cc68b1b65 admin admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 16:10:42.437442 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 835/1645] 10.222.0.15 () {60 vars in 1144 bytes} [Mon Aug 22 16:10:42 2022] POST /identity/v3/users => generated 294 bytes in 208 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 22 16:10:42.438300 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.notifications [None req-03ea14a2-2a7b-4f5c-86d1-92764a1f6351 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4a6290a807e44071b1f98e81b83f6a7e'} {{(pid=81767) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 16:10:42.439464 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.notifications [None req-03ea14a2-2a7b-4f5c-86d1-92764a1f6351 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4a6290a807e44071b1f98e81b83f6a7e'} {{(pid=81767) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 16:10:42.441729 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8066468a-f377-4b64-975f-183cc68b1b65 admin admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 16:10:42.445533 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-8066468a-f377-4b64-975f-183cc68b1b65 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 16:10:42.503194 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.notifications [None req-03ea14a2-2a7b-4f5c-86d1-92764a1f6351 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4a6290a807e44071b1f98e81b83f6a7e'} {{(pid=81767) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 16:10:42.524028 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8066468a-f377-4b64-975f-183cc68b1b65 admin admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 16:10:42.531505 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.notifications [None req-03ea14a2-2a7b-4f5c-86d1-92764a1f6351 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4a6290a807e44071b1f98e81b83f6a7e'} {{(pid=81767) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 16:10:42.543839 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fecf9e1e-f909-45b8-9953-badd7a1d7790 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a98b1e4431940ab8d1fc95df2858a6c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 16:10:42.548400 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fecf9e1e-f909-45b8-9953-badd7a1d7790 admin admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:10:42.549319 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fecf9e1e-f909-45b8-9953-badd7a1d7790 admin admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:10:42.549966 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fecf9e1e-f909-45b8-9953-badd7a1d7790 admin admin] PATH_INFO: `/v3/roles` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:10:42.552421 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fecf9e1e-f909-45b8-9953-badd7a1d7790 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 16:10:42.556394 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fecf9e1e-f909-45b8-9953-badd7a1d7790 admin admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 16:10:42.580300 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.notifications [None req-03ea14a2-2a7b-4f5c-86d1-92764a1f6351 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4a6290a807e44071b1f98e81b83f6a7e'} {{(pid=81767) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 16:10:42.581073 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=88,keystone:DELETE=7,keystone:INSERT=5 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 16:10:42.598481 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 811/1646] 10.222.0.15 () {58 vars in 1224 bytes} [Mon Aug 22 16:10:42 2022] DELETE /identity/v3/users/4a6290a807e44071b1f98e81b83f6a7e => generated 0 bytes in 480 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 22 16:10:42.601248 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aec12d6a-9b4e-4531-bbf6-ca245196d394 None None] REQUEST_METHOD: `POST` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:10:42.601568 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aec12d6a-9b4e-4531-bbf6-ca245196d394 None None] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:10:42.601844 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aec12d6a-9b4e-4531-bbf6-ca245196d394 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:10:42.604178 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 836/1647] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Aug 22 16:10:42 2022] GET /identity/v3/roles => generated 1420 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 22 16:10:42.605271 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fecf9e1e-f909-45b8-9953-badd7a1d7790 admin admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 16:10:42.639115 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fecf9e1e-f909-45b8-9953-badd7a1d7790 admin admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 16:10:42.698103 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.auth.core [None req-aec12d6a-9b4e-4531-bbf6-ca245196d394 None None] MFA Rules not processed for user `6b2199ae0fb0480aa917ada391f5ec54`. Rule list: `[]` (Enabled: `True`). {{(pid=81767) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 16:10:42.700037 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.fernet_utils [None req-aec12d6a-9b4e-4531-bbf6-ca245196d394 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81767) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 16:10:42.879246 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-087e0dd0-b04e-45cf-b131-4b1d97a6ccbb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a98b1e4431940ab8d1fc95df2858a6c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 16:10:42.885226 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-087e0dd0-b04e-45cf-b131-4b1d97a6ccbb admin admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:10:42.885544 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-087e0dd0-b04e-45cf-b131-4b1d97a6ccbb admin admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:10:42.889572 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-087e0dd0-b04e-45cf-b131-4b1d97a6ccbb admin admin] PATH_INFO: `/v3/roles` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:10:42.891023 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-087e0dd0-b04e-45cf-b131-4b1d97a6ccbb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 16:10:42.898797 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-087e0dd0-b04e-45cf-b131-4b1d97a6ccbb admin admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 16:10:42.918721 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 812/1648] 10.222.0.15 () {58 vars in 958 bytes} [Mon Aug 22 16:10:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 319 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 16:10:42.926350 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 837/1649] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Aug 22 16:10:42 2022] GET /identity/v3/roles => generated 1420 bytes in 322 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 22 16:10:42.930904 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aec12d6a-9b4e-4531-bbf6-ca245196d394 None None] Authenticating user token {{(pid=81767) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 16:10:42.965762 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aec12d6a-9b4e-4531-bbf6-ca245196d394 None None] Validating token access rules against request {{(pid=81767) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 16:10:42.979510 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-118acb9c-2630-42f3-a59d-271d7f027ca0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a98b1e4431940ab8d1fc95df2858a6c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81767) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 16:10:42.980612 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-118acb9c-2630-42f3-a59d-271d7f027ca0 admin admin] REQUEST_METHOD: `PUT` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:10:42.981044 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-118acb9c-2630-42f3-a59d-271d7f027ca0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:10:42.981448 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-118acb9c-2630-42f3-a59d-271d7f027ca0 admin admin] PATH_INFO: `/v3/projects/0d4e62dbc6314addaa25d8bab6380b7c/users/d7d5682e9ccb437daea3a9d2526cb60e/roles/e6fc1ee3fb974125ba9b4fe55b12b45e` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:10:43.006124 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-118acb9c-2630-42f3-a59d-271d7f027ca0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0d4e62dbc6314addaa25d8bab6380b7c, user_id=d7d5682e9ccb437daea3a9d2526cb60e, role_id=e6fc1ee3fb974125ba9b4fe55b12b45e)` {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 16:10:43.009576 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-118acb9c-2630-42f3-a59d-271d7f027ca0 admin admin] RBAC: Authorization granted {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 16:10:43.064653 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-087e0dd0-b04e-45cf-b131-4b1d97a6ccbb admin admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 16:10:43.067562 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-087e0dd0-b04e-45cf-b131-4b1d97a6ccbb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 16:10:43.125853 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 813/1650] 10.222.0.15 () {60 vars in 1483 bytes} [Mon Aug 22 16:10:42 2022] PUT /identity/v3/projects/0d4e62dbc6314addaa25d8bab6380b7c/users/d7d5682e9ccb437daea3a9d2526cb60e/roles/e6fc1ee3fb974125ba9b4fe55b12b45e => generated 0 bytes in 195 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 16:10:43.137975 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ad86780-c227-455e-9b1a-7a17c5347a6e None None] REQUEST_METHOD: `POST` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:10:43.138919 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ad86780-c227-455e-9b1a-7a17c5347a6e None None] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:10:43.139422 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ad86780-c227-455e-9b1a-7a17c5347a6e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:10:43.151744 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG dbcounter [-] [81767] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=81767) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 16:10:43.195796 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.auth.core [None req-1ad86780-c227-455e-9b1a-7a17c5347a6e None None] MFA Rules not processed for user `d7d5682e9ccb437daea3a9d2526cb60e`. Rule list: `[]` (Enabled: `True`). {{(pid=81767) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 16:10:43.197125 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.fernet_utils [None req-1ad86780-c227-455e-9b1a-7a17c5347a6e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81767) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 16:10:43.201317 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-087e0dd0-b04e-45cf-b131-4b1d97a6ccbb admin admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 16:10:43.276915 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dcbab39-1436-4266-8360-2f82eb948e13 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '234687adad594f1aa45775a565ffbdc7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4ee0413f9ef4d4fb1f95a660babe090', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 16:10:43.278731 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dcbab39-1436-4266-8360-2f82eb948e13 service neutron] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:10:43.280167 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dcbab39-1436-4266-8360-2f82eb948e13 service neutron] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:10:43.280788 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dcbab39-1436-4266-8360-2f82eb948e13 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:10:43.365615 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 814/1651] 10.222.0.15 () {58 vars in 958 bytes} [Mon Aug 22 16:10:43 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 230 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 22 16:10:43.366904 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ad86780-c227-455e-9b1a-7a17c5347a6e None None] Authenticating user token {{(pid=81767) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 16:10:43.419349 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dcbab39-1436-4266-8360-2f82eb948e13 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 16:10:43.421325 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dcbab39-1436-4266-8360-2f82eb948e13 service neutron] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 16:10:43.452978 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 838/1652] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Aug 22 16:10:43 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 389 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 22 16:10:43.522187 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ad86780-c227-455e-9b1a-7a17c5347a6e None None] Validating token access rules against request {{(pid=81767) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 16:10:43.534599 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee67375c-94d6-4419-825d-386cab55c995 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a98b1e4431940ab8d1fc95df2858a6c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81767) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 16:10:43.535512 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee67375c-94d6-4419-825d-386cab55c995 admin admin] REQUEST_METHOD: `POST` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:10:43.535798 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee67375c-94d6-4419-825d-386cab55c995 admin admin] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:10:43.536127 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee67375c-94d6-4419-825d-386cab55c995 admin admin] PATH_INFO: `/v3/projects` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:10:43.536829 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee67375c-94d6-4419-825d-386cab55c995 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 16:10:43.540698 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee67375c-94d6-4419-825d-386cab55c995 admin admin] RBAC: Authorization granted {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 16:10:43.695715 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 815/1653] 10.222.0.15 () {60 vars in 1153 bytes} [Mon Aug 22 16:10:43 2022] POST /identity/v3/projects => generated 385 bytes in 329 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 22 16:10:43.700055 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dcbab39-1436-4266-8360-2f82eb948e13 service neutron] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 16:10:43.707264 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=100 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 16:10:43.716724 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dcbab39-1436-4266-8360-2f82eb948e13 service neutron] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 16:10:43.859225 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4de0bc1-6a12-47b0-8fdc-4ec0195b8577 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a98b1e4431940ab8d1fc95df2858a6c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 16:10:43.859894 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4de0bc1-6a12-47b0-8fdc-4ec0195b8577 admin admin] REQUEST_METHOD: `POST` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:10:43.860181 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4de0bc1-6a12-47b0-8fdc-4ec0195b8577 admin admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:10:43.860461 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4de0bc1-6a12-47b0-8fdc-4ec0195b8577 admin admin] PATH_INFO: `/v3/users` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:10:43.861131 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4de0bc1-6a12-47b0-8fdc-4ec0195b8577 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 16:10:43.871965 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4de0bc1-6a12-47b0-8fdc-4ec0195b8577 admin admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 16:10:43.975314 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 839/1654] 10.222.0.15 () {60 vars in 1144 bytes} [Mon Aug 22 16:10:43 2022] POST /identity/v3/users => generated 293 bytes in 275 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 22 16:10:43.979588 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee67375c-94d6-4419-825d-386cab55c995 admin admin] Authenticating user token {{(pid=81767) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 16:10:43.999257 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee67375c-94d6-4419-825d-386cab55c995 admin admin] Validating token access rules against request {{(pid=81767) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 16:10:44.008081 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2de37aeb-9db6-4901-ae71-60b464c06910 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a98b1e4431940ab8d1fc95df2858a6c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81767) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 16:10:44.008722 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2de37aeb-9db6-4901-ae71-60b464c06910 admin admin] REQUEST_METHOD: `GET` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:10:44.009002 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2de37aeb-9db6-4901-ae71-60b464c06910 admin admin] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:10:44.009277 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2de37aeb-9db6-4901-ae71-60b464c06910 admin admin] PATH_INFO: `/v3/roles` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:10:44.009935 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2de37aeb-9db6-4901-ae71-60b464c06910 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 16:10:44.013547 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2de37aeb-9db6-4901-ae71-60b464c06910 admin admin] RBAC: Authorization granted {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 16:10:44.061703 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4de0bc1-6a12-47b0-8fdc-4ec0195b8577 admin admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 16:10:44.073845 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 816/1655] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Aug 22 16:10:43 2022] GET /identity/v3/roles => generated 1420 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 22 16:10:44.075301 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4de0bc1-6a12-47b0-8fdc-4ec0195b8577 admin admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 16:10:44.081123 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d60037b-30f2-42bc-8b25-fc8142b949b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a98b1e4431940ab8d1fc95df2858a6c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 16:10:44.081968 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d60037b-30f2-42bc-8b25-fc8142b949b4 admin admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:10:44.082389 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d60037b-30f2-42bc-8b25-fc8142b949b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:10:44.082833 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d60037b-30f2-42bc-8b25-fc8142b949b4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:10:44.084126 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d60037b-30f2-42bc-8b25-fc8142b949b4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 16:10:44.101273 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d60037b-30f2-42bc-8b25-fc8142b949b4 admin admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 16:10:44.124603 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2de37aeb-9db6-4901-ae71-60b464c06910 admin admin] Authenticating user token {{(pid=81767) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 16:10:44.126735 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 840/1656] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Aug 22 16:10:44 2022] GET /identity/v3/roles => generated 1420 bytes in 62 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 22 16:10:44.139520 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2de37aeb-9db6-4901-ae71-60b464c06910 admin admin] Validating token access rules against request {{(pid=81767) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 16:10:44.147967 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea474fc9-d9a4-48d0-bb3c-48fff0960c5b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a98b1e4431940ab8d1fc95df2858a6c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81767) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 16:10:44.150603 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea474fc9-d9a4-48d0-bb3c-48fff0960c5b admin admin] REQUEST_METHOD: `PUT` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:10:44.150892 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea474fc9-d9a4-48d0-bb3c-48fff0960c5b admin admin] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:10:44.151168 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea474fc9-d9a4-48d0-bb3c-48fff0960c5b admin admin] PATH_INFO: `/v3/projects/83c9e9ba49104721ad30e46354b26669/users/6c03ee9b42d74adf842178380c59f28e/roles/e6fc1ee3fb974125ba9b4fe55b12b45e` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:10:44.197212 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea474fc9-d9a4-48d0-bb3c-48fff0960c5b admin admin] RBAC: Authorizing `identity:create_grant(project_id=83c9e9ba49104721ad30e46354b26669, user_id=6c03ee9b42d74adf842178380c59f28e, role_id=e6fc1ee3fb974125ba9b4fe55b12b45e)` {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 16:10:44.200415 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea474fc9-d9a4-48d0-bb3c-48fff0960c5b admin admin] RBAC: Authorization granted {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 16:10:44.298042 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 817/1657] 10.222.0.15 () {60 vars in 1483 bytes} [Mon Aug 22 16:10:44 2022] PUT /identity/v3/projects/83c9e9ba49104721ad30e46354b26669/users/6c03ee9b42d74adf842178380c59f28e/roles/e6fc1ee3fb974125ba9b4fe55b12b45e => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 16:10:44.310544 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e32cb0ab-66cb-46ff-a41d-da8fba8b6a73 None None] REQUEST_METHOD: `POST` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:10:44.311039 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e32cb0ab-66cb-46ff-a41d-da8fba8b6a73 None None] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:10:44.311482 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e32cb0ab-66cb-46ff-a41d-da8fba8b6a73 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:10:44.388906 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.auth.core [None req-e32cb0ab-66cb-46ff-a41d-da8fba8b6a73 None None] MFA Rules not processed for user `6c03ee9b42d74adf842178380c59f28e`. Rule list: `[]` (Enabled: `True`). {{(pid=81768) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 16:10:44.390250 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-e32cb0ab-66cb-46ff-a41d-da8fba8b6a73 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 16:10:44.530329 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 841/1658] 10.222.0.15 () {58 vars in 958 bytes} [Mon Aug 22 16:10:44 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 221 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 16:10:44.531715 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea474fc9-d9a4-48d0-bb3c-48fff0960c5b admin admin] Authenticating user token {{(pid=81767) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 16:10:44.648058 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG dbcounter [-] [81767] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81767) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 16:10:44.688197 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea474fc9-d9a4-48d0-bb3c-48fff0960c5b admin admin] Validating token access rules against request {{(pid=81767) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 16:10:44.704859 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66dbab94-fef6-446c-b700-d24b41f5fa4c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a98b1e4431940ab8d1fc95df2858a6c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81767) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 16:10:44.706296 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66dbab94-fef6-446c-b700-d24b41f5fa4c admin admin] REQUEST_METHOD: `POST` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:10:44.706826 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66dbab94-fef6-446c-b700-d24b41f5fa4c admin admin] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:10:44.707386 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66dbab94-fef6-446c-b700-d24b41f5fa4c admin admin] PATH_INFO: `/v3/projects` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:10:44.708507 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66dbab94-fef6-446c-b700-d24b41f5fa4c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 16:10:44.713670 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66dbab94-fef6-446c-b700-d24b41f5fa4c admin admin] RBAC: Authorization granted {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 16:10:44.893243 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 818/1659] 10.222.0.15 () {60 vars in 1153 bytes} [Mon Aug 22 16:10:44 2022] POST /identity/v3/projects => generated 385 bytes in 362 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 22 16:10:44.898229 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e32cb0ab-66cb-46ff-a41d-da8fba8b6a73 None None] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 16:10:44.905914 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7898a97e-9d53-4b01-8658-be8d114f0c99 None None] REQUEST_METHOD: `POST` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:10:44.906186 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7898a97e-9d53-4b01-8658-be8d114f0c99 None None] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:10:44.906465 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7898a97e-9d53-4b01-8658-be8d114f0c99 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:10:44.922362 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e32cb0ab-66cb-46ff-a41d-da8fba8b6a73 None None] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 16:10:45.011161 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.auth.core [None req-7898a97e-9d53-4b01-8658-be8d114f0c99 None None] MFA Rules not processed for user `c3a0c958112f44d28aa13df75d5612cc`. Rule list: `[]` (Enabled: `True`). {{(pid=81767) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 16:10:45.013277 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.fernet_utils [None req-7898a97e-9d53-4b01-8658-be8d114f0c99 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81767) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 16:10:45.106974 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dcfbae6-30d1-4fd7-9231-817225d12b11 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a98b1e4431940ab8d1fc95df2858a6c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 16:10:45.107641 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dcfbae6-30d1-4fd7-9231-817225d12b11 admin admin] REQUEST_METHOD: `POST` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:10:45.112333 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dcfbae6-30d1-4fd7-9231-817225d12b11 admin admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:10:45.112333 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dcfbae6-30d1-4fd7-9231-817225d12b11 admin admin] PATH_INFO: `/v3/users` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:10:45.112333 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dcfbae6-30d1-4fd7-9231-817225d12b11 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 16:10:45.112333 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dcfbae6-30d1-4fd7-9231-817225d12b11 admin admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 16:10:45.211843 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 819/1660] 10.222.0.15 () {58 vars in 958 bytes} [Mon Aug 22 16:10:44 2022] POST /identity/v3/auth/tokens => generated 3347 bytes in 301 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 16:10:45.263990 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 16:10:45.271983 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 842/1661] 10.222.0.15 () {60 vars in 1144 bytes} [Mon Aug 22 16:10:44 2022] POST /identity/v3/users => generated 299 bytes in 372 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 22 16:10:45.275268 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7898a97e-9d53-4b01-8658-be8d114f0c99 None None] Authenticating user token {{(pid=81767) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 16:10:45.293313 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7898a97e-9d53-4b01-8658-be8d114f0c99 None None] Validating token access rules against request {{(pid=81767) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 16:10:45.307493 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb7f4c79-4037-427c-83b9-aa7e85aa4338 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a98b1e4431940ab8d1fc95df2858a6c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81767) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 16:10:45.308140 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb7f4c79-4037-427c-83b9-aa7e85aa4338 admin admin] REQUEST_METHOD: `GET` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:10:45.308423 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb7f4c79-4037-427c-83b9-aa7e85aa4338 admin admin] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:10:45.308754 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb7f4c79-4037-427c-83b9-aa7e85aa4338 admin admin] PATH_INFO: `/v3/roles` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:10:45.309477 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb7f4c79-4037-427c-83b9-aa7e85aa4338 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 16:10:45.313593 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb7f4c79-4037-427c-83b9-aa7e85aa4338 admin admin] RBAC: Authorization granted {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 16:10:45.340221 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dcfbae6-30d1-4fd7-9231-817225d12b11 admin admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 16:10:45.341749 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 820/1662] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Aug 22 16:10:45 2022] GET /identity/v3/roles => generated 1420 bytes in 67 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 22 16:10:45.354547 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dcfbae6-30d1-4fd7-9231-817225d12b11 admin admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 16:10:45.371337 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f1359d2-5ce2-43ea-b577-2d389fce8ba2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a98b1e4431940ab8d1fc95df2858a6c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 16:10:45.372171 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f1359d2-5ce2-43ea-b577-2d389fce8ba2 admin admin] REQUEST_METHOD: `PUT` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:10:45.372457 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f1359d2-5ce2-43ea-b577-2d389fce8ba2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:10:45.373081 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f1359d2-5ce2-43ea-b577-2d389fce8ba2 admin admin] PATH_INFO: `/v3/projects/38c257ac94544f0bace83551b519d5bc/users/a52a3afe77094bde8222ca0101b8747e/roles/9d9848cfd22e4b089db443c350e56670` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:10:45.415323 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f1359d2-5ce2-43ea-b577-2d389fce8ba2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=38c257ac94544f0bace83551b519d5bc, user_id=a52a3afe77094bde8222ca0101b8747e, role_id=9d9848cfd22e4b089db443c350e56670)` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 16:10:45.420396 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f1359d2-5ce2-43ea-b577-2d389fce8ba2 admin admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 16:10:45.539698 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 843/1663] 10.222.0.15 () {60 vars in 1483 bytes} [Mon Aug 22 16:10:45 2022] PUT /identity/v3/projects/38c257ac94544f0bace83551b519d5bc/users/a52a3afe77094bde8222ca0101b8747e/roles/9d9848cfd22e4b089db443c350e56670 => generated 0 bytes in 200 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 16:10:45.543679 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb7f4c79-4037-427c-83b9-aa7e85aa4338 admin admin] Authenticating user token {{(pid=81767) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 16:10:45.557359 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb7f4c79-4037-427c-83b9-aa7e85aa4338 admin admin] Validating token access rules against request {{(pid=81767) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 16:10:45.739546 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b9afd40-a413-42a5-82e2-0c23448aeba5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a98b1e4431940ab8d1fc95df2858a6c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81767) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 16:10:45.741466 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b9afd40-a413-42a5-82e2-0c23448aeba5 admin admin] REQUEST_METHOD: `GET` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:10:45.744791 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b9afd40-a413-42a5-82e2-0c23448aeba5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:10:45.744791 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b9afd40-a413-42a5-82e2-0c23448aeba5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:10:45.744791 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b9afd40-a413-42a5-82e2-0c23448aeba5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 16:10:45.745986 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f1359d2-5ce2-43ea-b577-2d389fce8ba2 admin admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 16:10:45.748098 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-6f1359d2-5ce2-43ea-b577-2d389fce8ba2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 16:10:45.748788 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b9afd40-a413-42a5-82e2-0c23448aeba5 admin admin] RBAC: Authorization granted {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 16:10:45.768592 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 821/1664] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Aug 22 16:10:45 2022] GET /identity/v3/roles => generated 1420 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 22 16:10:45.770421 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b9afd40-a413-42a5-82e2-0c23448aeba5 admin admin] Authenticating user token {{(pid=81767) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 16:10:45.780291 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f1359d2-5ce2-43ea-b577-2d389fce8ba2 admin admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 16:10:45.796088 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56565f85-c613-455d-8817-8c557cb0626f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a98b1e4431940ab8d1fc95df2858a6c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 16:10:45.799088 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56565f85-c613-455d-8817-8c557cb0626f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:10:45.800623 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56565f85-c613-455d-8817-8c557cb0626f admin admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:10:45.801985 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56565f85-c613-455d-8817-8c557cb0626f admin admin] PATH_INFO: `/v3/projects/c4b14e0de34d42cba3ac8629f8f21549` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:10:45.806172 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56565f85-c613-455d-8817-8c557cb0626f admin admin] RBAC: Authorizing `identity:delete_project(project_id=c4b14e0de34d42cba3ac8629f8f21549)` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 16:10:45.815139 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56565f85-c613-455d-8817-8c557cb0626f admin admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 16:10:45.817253 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b9afd40-a413-42a5-82e2-0c23448aeba5 admin admin] Validating token access rules against request {{(pid=81767) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 16:10:45.833704 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c802a755-4918-43ef-b080-bad1d0ac20a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a98b1e4431940ab8d1fc95df2858a6c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81767) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 16:10:45.839859 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c802a755-4918-43ef-b080-bad1d0ac20a1 admin admin] REQUEST_METHOD: `PUT` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:10:45.840355 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c802a755-4918-43ef-b080-bad1d0ac20a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:10:45.840760 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c802a755-4918-43ef-b080-bad1d0ac20a1 admin admin] PATH_INFO: `/v3/projects/38c257ac94544f0bace83551b519d5bc/users/a52a3afe77094bde8222ca0101b8747e/roles/9d9848cfd22e4b089db443c350e56670` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:10:45.850102 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c802a755-4918-43ef-b080-bad1d0ac20a1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=38c257ac94544f0bace83551b519d5bc, user_id=a52a3afe77094bde8222ca0101b8747e, role_id=9d9848cfd22e4b089db443c350e56670)` {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 16:10:45.861133 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c802a755-4918-43ef-b080-bad1d0ac20a1 admin admin] RBAC: Authorization granted {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 16:10:45.931502 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 822/1665] 10.222.0.15 () {60 vars in 1483 bytes} [Mon Aug 22 16:10:45 2022] PUT /identity/v3/projects/38c257ac94544f0bace83551b519d5bc/users/a52a3afe77094bde8222ca0101b8747e/roles/9d9848cfd22e4b089db443c350e56670 => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 22 16:10:45.939925 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f415f1d-6227-416b-970c-12459de061d1 None None] REQUEST_METHOD: `POST` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:10:45.940328 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f415f1d-6227-416b-970c-12459de061d1 None None] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:10:45.940710 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f415f1d-6227-416b-970c-12459de061d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:10:46.016596 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.auth.core [None req-8f415f1d-6227-416b-970c-12459de061d1 None None] MFA Rules not processed for user `a52a3afe77094bde8222ca0101b8747e`. Rule list: `[]` (Enabled: `True`). {{(pid=81767) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 16:10:46.020609 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.fernet_utils [None req-8f415f1d-6227-416b-970c-12459de061d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81767) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 16:10:46.056276 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG dbcounter [-] [81767] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81767) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 16:10:46.066752 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-56565f85-c613-455d-8817-8c557cb0626f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c4b14e0de34d42cba3ac8629f8f21549', 'request_id': 'req-56565f85-c613-455d-8817-8c557cb0626f'} {{(pid=81768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 16:10:46.090267 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-56565f85-c613-455d-8817-8c557cb0626f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c4b14e0de34d42cba3ac8629f8f21549', 'request_id': 'req-56565f85-c613-455d-8817-8c557cb0626f'} {{(pid=81768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 16:10:46.211841 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 823/1666] 10.222.0.15 () {58 vars in 958 bytes} [Mon Aug 22 16:10:45 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 274 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 16:10:46.226861 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f415f1d-6227-416b-970c-12459de061d1 None None] Authenticating user token {{(pid=81767) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 16:10:46.226861 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.fernet_utils [None req-8f415f1d-6227-416b-970c-12459de061d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81767) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 16:10:46.314380 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-56565f85-c613-455d-8817-8c557cb0626f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c4b14e0de34d42cba3ac8629f8f21549', 'request_id': 'req-56565f85-c613-455d-8817-8c557cb0626f'} {{(pid=81768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 16:10:46.329128 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-56565f85-c613-455d-8817-8c557cb0626f admin admin] The token cache is being invalidate because project c4b14e0de34d42cba3ac8629f8f21549 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81768) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 22 16:10:46.337508 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-56565f85-c613-455d-8817-8c557cb0626f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 16:10:46.342478 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-56565f85-c613-455d-8817-8c557cb0626f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 16:10:46.344457 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 844/1667] 10.222.0.15 () {58 vars in 1233 bytes} [Mon Aug 22 16:10:45 2022] DELETE /identity/v3/projects/c4b14e0de34d42cba3ac8629f8f21549 => generated 0 bytes in 599 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 22 16:10:46.349417 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efb00966-54e3-4fca-8c9a-12bf84e483d3 None None] REQUEST_METHOD: `POST` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:10:46.350548 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efb00966-54e3-4fca-8c9a-12bf84e483d3 None None] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:10:46.351058 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efb00966-54e3-4fca-8c9a-12bf84e483d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:10:46.388444 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f415f1d-6227-416b-970c-12459de061d1 None None] Validating token access rules against request {{(pid=81767) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 16:10:46.392326 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.auth.core [None req-efb00966-54e3-4fca-8c9a-12bf84e483d3 None None] MFA Rules not processed for user `6b2199ae0fb0480aa917ada391f5ec54`. Rule list: `[]` (Enabled: `True`). {{(pid=81768) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 16:10:46.394615 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-efb00966-54e3-4fca-8c9a-12bf84e483d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 16:10:46.399352 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cea429c4-c048-44fb-8e5b-7993cdfd3562 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79706d480f864998b9f8594be407ceab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4ee0413f9ef4d4fb1f95a660babe090', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81767) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 16:10:46.400151 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea429c4-c048-44fb-8e5b-7993cdfd3562 service cinder] REQUEST_METHOD: `GET` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:10:46.400432 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea429c4-c048-44fb-8e5b-7993cdfd3562 service cinder] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:10:46.400704 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea429c4-c048-44fb-8e5b-7993cdfd3562 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:10:46.402728 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.fernet_utils [None req-cea429c4-c048-44fb-8e5b-7993cdfd3562 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81767) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 16:10:46.433896 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cea429c4-c048-44fb-8e5b-7993cdfd3562 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 16:10:46.435600 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cea429c4-c048-44fb-8e5b-7993cdfd3562 service cinder] RBAC: Authorization granted {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 16:10:46.469869 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 824/1668] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Aug 22 16:10:46 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 246 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 22 16:10:46.573719 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 845/1669] 10.222.0.15 () {58 vars in 958 bytes} [Mon Aug 22 16:10:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 225 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 16:10:46.582335 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9355cd3-0b15-44bf-ace8-bd57f267f958 None None] REQUEST_METHOD: `POST` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:10:46.582733 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9355cd3-0b15-44bf-ace8-bd57f267f958 None None] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:10:46.583048 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9355cd3-0b15-44bf-ace8-bd57f267f958 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:10:46.638940 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.auth.core [None req-f9355cd3-0b15-44bf-ace8-bd57f267f958 None None] MFA Rules not processed for user `6b2199ae0fb0480aa917ada391f5ec54`. Rule list: `[]` (Enabled: `True`). {{(pid=81767) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 16:10:46.645666 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.fernet_utils [None req-f9355cd3-0b15-44bf-ace8-bd57f267f958 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81767) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 16:10:46.668805 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 825/1670] 10.222.0.15 () {58 vars in 958 bytes} [Mon Aug 22 16:10:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 16:10:46.670004 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efb00966-54e3-4fca-8c9a-12bf84e483d3 None None] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 16:10:46.685606 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efb00966-54e3-4fca-8c9a-12bf84e483d3 None None] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 16:10:46.701248 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9595cd1-8a4c-48f4-afbb-5c50b01f4f62 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a98b1e4431940ab8d1fc95df2858a6c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 16:10:46.701879 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9595cd1-8a4c-48f4-afbb-5c50b01f4f62 admin admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:10:46.702160 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9595cd1-8a4c-48f4-afbb-5c50b01f4f62 admin admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:10:46.702482 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9595cd1-8a4c-48f4-afbb-5c50b01f4f62 admin admin] PATH_INFO: `/v3/domains` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:10:46.703153 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9595cd1-8a4c-48f4-afbb-5c50b01f4f62 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81768) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 16:10:46.703531 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9595cd1-8a4c-48f4-afbb-5c50b01f4f62 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 16:10:46.705260 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9595cd1-8a4c-48f4-afbb-5c50b01f4f62 admin admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 16:10:46.752742 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 846/1671] 10.222.0.15 () {58 vars in 1153 bytes} [Mon Aug 22 16:10:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 16:10:46.757777 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9355cd3-0b15-44bf-ace8-bd57f267f958 None None] Authenticating user token {{(pid=81767) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 16:10:46.775718 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9355cd3-0b15-44bf-ace8-bd57f267f958 None None] Validating token access rules against request {{(pid=81767) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 16:10:46.785488 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efd6ca5c-216d-4218-b015-93bebaa2640f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a98b1e4431940ab8d1fc95df2858a6c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81767) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 16:10:46.786776 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efd6ca5c-216d-4218-b015-93bebaa2640f admin admin] REQUEST_METHOD: `POST` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:10:46.788003 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efd6ca5c-216d-4218-b015-93bebaa2640f admin admin] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:10:46.788451 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efd6ca5c-216d-4218-b015-93bebaa2640f admin admin] PATH_INFO: `/v3/projects` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:10:46.789303 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efd6ca5c-216d-4218-b015-93bebaa2640f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 16:10:46.792014 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efd6ca5c-216d-4218-b015-93bebaa2640f admin admin] RBAC: Authorization granted {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 16:10:46.888437 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 826/1672] 10.222.0.15 () {60 vars in 1153 bytes} [Mon Aug 22 16:10:46 2022] POST /identity/v3/projects => generated 365 bytes in 131 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 22 16:10:46.893019 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9595cd1-8a4c-48f4-afbb-5c50b01f4f62 admin admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 16:10:46.913211 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9595cd1-8a4c-48f4-afbb-5c50b01f4f62 admin admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 16:10:47.015938 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 16:10:47.093196 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c857d943-4635-466e-9678-9b50b5b80bfa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a98b1e4431940ab8d1fc95df2858a6c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 16:10:47.093196 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c857d943-4635-466e-9678-9b50b5b80bfa admin admin] REQUEST_METHOD: `POST` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:10:47.093196 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c857d943-4635-466e-9678-9b50b5b80bfa admin admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:10:47.093196 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c857d943-4635-466e-9678-9b50b5b80bfa admin admin] PATH_INFO: `/v3/users` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:10:47.095672 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0364806f-3ae7-4c90-8e0e-280f4cf036b1 None None] REQUEST_METHOD: `POST` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:10:47.095672 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0364806f-3ae7-4c90-8e0e-280f4cf036b1 None None] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:10:47.095672 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0364806f-3ae7-4c90-8e0e-280f4cf036b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:10:47.107018 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c857d943-4635-466e-9678-9b50b5b80bfa admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 16:10:47.112174 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c857d943-4635-466e-9678-9b50b5b80bfa admin admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 16:10:47.185054 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.auth.core [None req-0364806f-3ae7-4c90-8e0e-280f4cf036b1 None None] MFA Rules not processed for user `4763c7fefe5a4fcdbead877717f760b8`. Rule list: `[]` (Enabled: `True`). {{(pid=81767) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 16:10:47.186461 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.fernet_utils [None req-0364806f-3ae7-4c90-8e0e-280f4cf036b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81767) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 16:10:47.235080 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 847/1673] 10.222.0.15 () {60 vars in 1144 bytes} [Mon Aug 22 16:10:46 2022] POST /identity/v3/users => generated 283 bytes in 342 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 22 16:10:47.239432 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c857d943-4635-466e-9678-9b50b5b80bfa admin admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 16:10:47.267945 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c857d943-4635-466e-9678-9b50b5b80bfa admin admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 16:10:47.275863 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4985e437-0e43-4ebf-92d4-13550ea619be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a98b1e4431940ab8d1fc95df2858a6c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 16:10:47.276823 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4985e437-0e43-4ebf-92d4-13550ea619be admin admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:10:47.277404 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4985e437-0e43-4ebf-92d4-13550ea619be admin admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:10:47.277925 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4985e437-0e43-4ebf-92d4-13550ea619be admin admin] PATH_INFO: `/v3/roles` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:10:47.280200 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4985e437-0e43-4ebf-92d4-13550ea619be admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 16:10:47.282495 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4985e437-0e43-4ebf-92d4-13550ea619be admin admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 16:10:47.300323 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 848/1674] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Aug 22 16:10:47 2022] GET /identity/v3/roles => generated 1420 bytes in 61 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 22 16:10:47.304225 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4985e437-0e43-4ebf-92d4-13550ea619be admin admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 16:10:47.335529 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4985e437-0e43-4ebf-92d4-13550ea619be admin admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 16:10:47.350062 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95868c61-1077-44ff-8050-3f31cbf669d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a98b1e4431940ab8d1fc95df2858a6c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 16:10:47.351207 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95868c61-1077-44ff-8050-3f31cbf669d1 admin admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:10:47.351605 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95868c61-1077-44ff-8050-3f31cbf669d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:10:47.352031 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95868c61-1077-44ff-8050-3f31cbf669d1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:10:47.353892 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95868c61-1077-44ff-8050-3f31cbf669d1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 16:10:47.356152 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95868c61-1077-44ff-8050-3f31cbf669d1 admin admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 16:10:47.375234 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 849/1675] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Aug 22 16:10:47 2022] GET /identity/v3/roles => generated 1420 bytes in 71 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 22 16:10:47.380259 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95868c61-1077-44ff-8050-3f31cbf669d1 admin admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 16:10:47.395252 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95868c61-1077-44ff-8050-3f31cbf669d1 admin admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 16:10:47.405667 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 827/1676] 10.222.0.15 () {58 vars in 958 bytes} [Mon Aug 22 16:10:47 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 312 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 16:10:47.420563 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dde86e1-f3e0-4e8a-ac19-b46a90882f6e None None] REQUEST_METHOD: `POST` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:10:47.420848 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dde86e1-f3e0-4e8a-ac19-b46a90882f6e None None] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:10:47.421123 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dde86e1-f3e0-4e8a-ac19-b46a90882f6e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:10:47.422485 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eb5292b-b691-4ac7-b27e-f82f669f2582 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a98b1e4431940ab8d1fc95df2858a6c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 16:10:47.423524 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eb5292b-b691-4ac7-b27e-f82f669f2582 admin admin] REQUEST_METHOD: `PUT` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:10:47.423956 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eb5292b-b691-4ac7-b27e-f82f669f2582 admin admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:10:47.424384 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eb5292b-b691-4ac7-b27e-f82f669f2582 admin admin] PATH_INFO: `/v3/projects/cde535a938a94617955ad88a414b0373/users/7464b05692444d5ca04516e3aa74faec/roles/e6fc1ee3fb974125ba9b4fe55b12b45e` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:10:47.456634 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eb5292b-b691-4ac7-b27e-f82f669f2582 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cde535a938a94617955ad88a414b0373, user_id=7464b05692444d5ca04516e3aa74faec, role_id=e6fc1ee3fb974125ba9b4fe55b12b45e)` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 16:10:47.468503 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.auth.core [None req-0dde86e1-f3e0-4e8a-ac19-b46a90882f6e None None] MFA Rules not processed for user `4763c7fefe5a4fcdbead877717f760b8`. Rule list: `[]` (Enabled: `True`). {{(pid=81767) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 16:10:47.471561 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.fernet_utils [None req-0dde86e1-f3e0-4e8a-ac19-b46a90882f6e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81767) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 16:10:47.485177 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 828/1677] 10.222.0.15 () {58 vars in 958 bytes} [Mon Aug 22 16:10:47 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 65 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 16:10:47.517042 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eb5292b-b691-4ac7-b27e-f82f669f2582 admin admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 16:10:47.518980 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dde86e1-f3e0-4e8a-ac19-b46a90882f6e None None] Authenticating user token {{(pid=81767) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 16:10:47.520557 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.fernet_utils [None req-0dde86e1-f3e0-4e8a-ac19-b46a90882f6e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81767) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 16:10:47.535120 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG dbcounter [-] [81767] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81767) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 16:10:47.654106 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 850/1678] 10.222.0.15 () {60 vars in 1483 bytes} [Mon Aug 22 16:10:47 2022] PUT /identity/v3/projects/cde535a938a94617955ad88a414b0373/users/7464b05692444d5ca04516e3aa74faec/roles/e6fc1ee3fb974125ba9b4fe55b12b45e => generated 0 bytes in 275 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 16:10:47.665716 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5db494eb-38ec-4457-a82f-48529aec792d None None] REQUEST_METHOD: `POST` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:10:47.666206 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5db494eb-38ec-4457-a82f-48529aec792d None None] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:10:47.667159 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5db494eb-38ec-4457-a82f-48529aec792d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:10:47.698384 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dde86e1-f3e0-4e8a-ac19-b46a90882f6e None None] Validating token access rules against request {{(pid=81767) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 16:10:47.737412 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.auth.core [None req-5db494eb-38ec-4457-a82f-48529aec792d None None] MFA Rules not processed for user `7464b05692444d5ca04516e3aa74faec`. Rule list: `[]` (Enabled: `True`). {{(pid=81768) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 16:10:47.739966 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-5db494eb-38ec-4457-a82f-48529aec792d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 16:10:47.905154 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 851/1679] 10.222.0.15 () {58 vars in 958 bytes} [Mon Aug 22 16:10:47 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 240 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 16:10:47.907961 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aa53e5e-732d-4baf-a396-d398184e050a None None] REQUEST_METHOD: `POST` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:10:47.908299 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ec7a5d2-4715-494b-8866-06e9ea2f2898 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4ef569e41fb46d0b721bb1c7663616f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4ee0413f9ef4d4fb1f95a660babe090', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81767) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 16:10:47.909095 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ec7a5d2-4715-494b-8866-06e9ea2f2898 service nova] REQUEST_METHOD: `GET` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:10:47.909508 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aa53e5e-732d-4baf-a396-d398184e050a None None] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:10:47.909891 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ec7a5d2-4715-494b-8866-06e9ea2f2898 service nova] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:10:47.910522 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aa53e5e-732d-4baf-a396-d398184e050a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:10:47.911153 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ec7a5d2-4715-494b-8866-06e9ea2f2898 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:10:48.003667 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.auth.core [None req-5aa53e5e-732d-4baf-a396-d398184e050a None None] MFA Rules not processed for user `6ec6b84e494c481abe9c9772a10037c9`. Rule list: `[]` (Enabled: `True`). {{(pid=81768) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 16:10:48.007622 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-5aa53e5e-732d-4baf-a396-d398184e050a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 16:10:48.088563 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ec7a5d2-4715-494b-8866-06e9ea2f2898 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 16:10:48.093461 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ec7a5d2-4715-494b-8866-06e9ea2f2898 service nova] RBAC: Authorization granted {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 16:10:48.128686 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 829/1680] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Aug 22 16:10:47 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 610 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 22 16:10:48.130571 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ec7a5d2-4715-494b-8866-06e9ea2f2898 service nova] Authenticating user token {{(pid=81767) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 16:10:48.136704 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.fernet_utils [None req-8ec7a5d2-4715-494b-8866-06e9ea2f2898 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81767) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 16:10:48.206555 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 852/1681] 10.222.0.15 () {58 vars in 958 bytes} [Mon Aug 22 16:10:47 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 293 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 16:10:48.216326 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aa53e5e-732d-4baf-a396-d398184e050a None None] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 16:10:48.217950 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-5aa53e5e-732d-4baf-a396-d398184e050a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 16:10:48.270849 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aa53e5e-732d-4baf-a396-d398184e050a None None] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 16:10:48.285017 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-801fba25-e460-4e68-96fe-610d78a69edd service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4ef569e41fb46d0b721bb1c7663616f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4ee0413f9ef4d4fb1f95a660babe090', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 16:10:48.285649 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-801fba25-e460-4e68-96fe-610d78a69edd service nova] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:10:48.285929 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-801fba25-e460-4e68-96fe-610d78a69edd service nova] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:10:48.286202 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-801fba25-e460-4e68-96fe-610d78a69edd service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:10:48.317010 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 16:10:48.321829 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-801fba25-e460-4e68-96fe-610d78a69edd service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 16:10:48.345384 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG dbcounter [-] [81767] Writing DB stats keystone:SELECT=100 {{(pid=81767) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 16:10:48.350131 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-801fba25-e460-4e68-96fe-610d78a69edd service nova] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 16:10:48.353730 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ec7a5d2-4715-494b-8866-06e9ea2f2898 service nova] Validating token access rules against request {{(pid=81767) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 16:10:48.366417 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96c4e6d9-a8fe-418b-828d-aa81dc401e1e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79706d480f864998b9f8594be407ceab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4ee0413f9ef4d4fb1f95a660babe090', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81767) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 16:10:48.367350 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96c4e6d9-a8fe-418b-828d-aa81dc401e1e service cinder] REQUEST_METHOD: `GET` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:10:48.367705 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96c4e6d9-a8fe-418b-828d-aa81dc401e1e service cinder] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:10:48.368835 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96c4e6d9-a8fe-418b-828d-aa81dc401e1e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:10:48.384585 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 853/1682] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Aug 22 16:10:48 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 169 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 16:10:48.394053 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96c4e6d9-a8fe-418b-828d-aa81dc401e1e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 16:10:48.396772 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96c4e6d9-a8fe-418b-828d-aa81dc401e1e service cinder] RBAC: Authorization granted {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 16:10:48.439095 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 830/1683] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Aug 22 16:10:48 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 16:10:48.719478 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-801fba25-e460-4e68-96fe-610d78a69edd service nova] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 16:10:48.720971 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-801fba25-e460-4e68-96fe-610d78a69edd service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 16:10:48.747131 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-801fba25-e460-4e68-96fe-610d78a69edd service nova] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 16:10:48.767735 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b4d994c-22e5-4bae-a760-ac68fa530cb5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79706d480f864998b9f8594be407ceab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4ee0413f9ef4d4fb1f95a660babe090', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 16:10:48.769383 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b4d994c-22e5-4bae-a760-ac68fa530cb5 service cinder] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:10:48.769873 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b4d994c-22e5-4bae-a760-ac68fa530cb5 service cinder] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:10:48.770300 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b4d994c-22e5-4bae-a760-ac68fa530cb5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:10:48.772530 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-8b4d994c-22e5-4bae-a760-ac68fa530cb5 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 16:10:48.874608 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b4d994c-22e5-4bae-a760-ac68fa530cb5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 16:10:48.877893 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b4d994c-22e5-4bae-a760-ac68fa530cb5 service cinder] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 16:10:48.900152 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 854/1684] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Aug 22 16:10:48 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 181 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 16:10:49.226704 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38654ce6-8415-45eb-baf7-083e7fbd807b None None] REQUEST_METHOD: `GET` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:10:49.227255 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38654ce6-8415-45eb-baf7-083e7fbd807b None None] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:10:49.227255 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38654ce6-8415-45eb-baf7-083e7fbd807b None None] PATH_INFO: `/` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:10:49.229146 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 831/1685] 10.222.0.15 () {54 vars in 921 bytes} [Mon Aug 22 16:10:49 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 16:10:49.239576 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fcc45d7-283f-4e06-809e-8842f57c993b None None] REQUEST_METHOD: `POST` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:10:49.240719 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fcc45d7-283f-4e06-809e-8842f57c993b None None] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:10:49.245718 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fcc45d7-283f-4e06-809e-8842f57c993b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:10:49.286668 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.auth.core [None req-1fcc45d7-283f-4e06-809e-8842f57c993b None None] MFA Rules not processed for user `08159bf8edda4246bb463f7b11b0894b`. Rule list: `[]` (Enabled: `True`). {{(pid=81768) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 16:10:49.290588 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-1fcc45d7-283f-4e06-809e-8842f57c993b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 16:10:49.469724 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 855/1686] 10.222.0.15 () {58 vars in 1018 bytes} [Mon Aug 22 16:10:49 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 231 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 16:10:49.515692 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25d5d120-e869-47f3-ada2-2f8af44ec571 None None] REQUEST_METHOD: `GET` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:10:49.515978 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25d5d120-e869-47f3-ada2-2f8af44ec571 None None] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:10:49.516257 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25d5d120-e869-47f3-ada2-2f8af44ec571 None None] PATH_INFO: `/` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:10:49.518034 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 832/1687] 10.222.0.15 () {54 vars in 961 bytes} [Mon Aug 22 16:10:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 16:10:49.531110 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6badeb17-e4bf-4e8d-9387-f6ce49fc4905 None None] REQUEST_METHOD: `POST` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:10:49.531670 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6badeb17-e4bf-4e8d-9387-f6ce49fc4905 None None] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:10:49.532357 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6badeb17-e4bf-4e8d-9387-f6ce49fc4905 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:10:49.580987 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.auth.core [None req-6badeb17-e4bf-4e8d-9387-f6ce49fc4905 None None] MFA Rules not processed for user `08159bf8edda4246bb463f7b11b0894b`. Rule list: `[]` (Enabled: `True`). {{(pid=81768) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 16:10:49.583844 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-6badeb17-e4bf-4e8d-9387-f6ce49fc4905 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 16:10:49.615619 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25d5d120-e869-47f3-ada2-2f8af44ec571 None None] Authenticating user token {{(pid=81767) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 16:10:49.630292 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 856/1688] 10.222.0.15 () {58 vars in 1058 bytes} [Mon Aug 22 16:10:49 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 22 16:10:49.647829 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25d5d120-e869-47f3-ada2-2f8af44ec571 None None] Validating token access rules against request {{(pid=81767) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 16:10:49.662356 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7669e120-0439-4909-bb2b-62e446e099a8 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '08159bf8edda4246bb463f7b11b0894b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4ee0413f9ef4d4fb1f95a660babe090', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81767) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 16:10:49.662984 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7669e120-0439-4909-bb2b-62e446e099a8 service placement] REQUEST_METHOD: `GET` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:10:49.663265 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7669e120-0439-4909-bb2b-62e446e099a8 service placement] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:10:49.664311 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7669e120-0439-4909-bb2b-62e446e099a8 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:10:49.687060 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7669e120-0439-4909-bb2b-62e446e099a8 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 16:10:49.688149 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7669e120-0439-4909-bb2b-62e446e099a8 service placement] RBAC: Authorization granted {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 16:10:49.726189 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 833/1689] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Aug 22 16:10:49 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 16:10:50.176519 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9570001b-445e-4ea8-abc6-ff99537cec38 None None] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:10:50.176810 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9570001b-445e-4ea8-abc6-ff99537cec38 None None] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:10:50.177081 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9570001b-445e-4ea8-abc6-ff99537cec38 None None] PATH_INFO: `/` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:10:50.178741 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 857/1690] 10.222.0.15 () {54 vars in 912 bytes} [Mon Aug 22 16:10:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 16:10:50.189332 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b0cf103-7414-4455-a469-3ffad0d75631 None None] REQUEST_METHOD: `POST` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:10:50.189834 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b0cf103-7414-4455-a469-3ffad0d75631 None None] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:10:50.190362 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b0cf103-7414-4455-a469-3ffad0d75631 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:10:50.201303 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application [None req-3b0cf103-7414-4455-a469-3ffad0d75631 None None] Expecting to find domain in project. The server could not comply with the request since it is either malformed or otherwise incorrect. The client is assumed to be in error.: keystone.exception.ValidationError: Expecting to find domain in project. The server could not comply with the request since it is either malformed or otherwise incorrect. The client is assumed to be in error. Aug 22 16:10:50.201303 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 16:10:50.201303 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 16:10:50.201303 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 16:10:50.201303 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 16:10:50.201303 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 16:10:50.201303 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 16:10:50.201303 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 16:10:50.201303 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 16:10:50.201303 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 16:10:50.201303 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 16:10:50.201303 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 16:10:50.201303 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/server/flask/common.py", line 1064, in wrapper Aug 22 16:10:50.202274 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return f(*args, **kwargs) Aug 22 16:10:50.202274 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/auth.py", line 315, in post Aug 22 16:10:50.202274 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application token = authentication.authenticate_for_token(auth_data) Aug 22 16:10:50.202274 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/_shared/authentication.py", line 182, in authenticate_for_token Aug 22 16:10:50.202274 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application auth_info = core.AuthInfo.create(auth=auth) Aug 22 16:10:50.202274 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/auth/core.py", line 126, in create Aug 22 16:10:50.202274 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application auth_info._validate_and_normalize_auth_data(scope_only) Aug 22 16:10:50.202274 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/auth/core.py", line 340, in _validate_and_normalize_auth_data Aug 22 16:10:50.202274 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application self._validate_and_normalize_scope_data() Aug 22 16:10:50.202274 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/auth/core.py", line 290, in _validate_and_normalize_scope_data Aug 22 16:10:50.202274 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application project_ref = self._lookup_project(self.auth['scope']['project']) Aug 22 16:10:50.202274 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/auth/core.py", line 194, in _lookup_project Aug 22 16:10:50.202274 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application raise exception.ValidationError(attribute='domain', Aug 22 16:10:50.202274 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application keystone.exception.ValidationError: Expecting to find domain in project. The server could not comply with the request since it is either malformed or otherwise incorrect. The client is assumed to be in error. Aug 22 16:10:50.202274 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application Aug 22 16:10:50.215671 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 834/1691] 10.222.0.15 () {58 vars in 1009 bytes} [Mon Aug 22 16:10:50 2022] POST /identity/v3/auth/tokens => generated 230 bytes in 24 msecs (HTTP/1.1 400) 5 headers in 186 bytes (2 switches on core 0) Aug 22 16:10:52.046810 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9570001b-445e-4ea8-abc6-ff99537cec38 None None] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 16:10:52.055590 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-9570001b-445e-4ea8-abc6-ff99537cec38 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 16:10:52.282627 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9570001b-445e-4ea8-abc6-ff99537cec38 None None] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 16:10:52.293920 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8e6ae9c-a9fd-497a-b0b5-ebc5ddba265d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a98b1e4431940ab8d1fc95df2858a6c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 16:10:52.294807 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8e6ae9c-a9fd-497a-b0b5-ebc5ddba265d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:10:52.295235 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8e6ae9c-a9fd-497a-b0b5-ebc5ddba265d admin admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:10:52.296344 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8e6ae9c-a9fd-497a-b0b5-ebc5ddba265d admin admin] PATH_INFO: `/v3/users/844f37c1f52e47c692ddabb07d807f39` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:10:52.299260 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8e6ae9c-a9fd-497a-b0b5-ebc5ddba265d admin admin] RBAC: Authorizing `identity:delete_user(user_id=844f37c1f52e47c692ddabb07d807f39)` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 16:10:52.301551 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8e6ae9c-a9fd-497a-b0b5-ebc5ddba265d admin admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 16:10:52.481118 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=96,keystone:DELETE=4 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 16:10:52.618123 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-a8e6ae9c-a9fd-497a-b0b5-ebc5ddba265d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '844f37c1f52e47c692ddabb07d807f39'} {{(pid=81768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 16:10:52.623004 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-a8e6ae9c-a9fd-497a-b0b5-ebc5ddba265d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '844f37c1f52e47c692ddabb07d807f39'} {{(pid=81768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 16:10:52.623928 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-a8e6ae9c-a9fd-497a-b0b5-ebc5ddba265d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '844f37c1f52e47c692ddabb07d807f39'} {{(pid=81768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 16:10:52.654574 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-a8e6ae9c-a9fd-497a-b0b5-ebc5ddba265d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '844f37c1f52e47c692ddabb07d807f39'} {{(pid=81768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 16:10:52.737235 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-a8e6ae9c-a9fd-497a-b0b5-ebc5ddba265d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '844f37c1f52e47c692ddabb07d807f39'} {{(pid=81768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 16:10:52.756609 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b57ea9b9-5541-4a11-98c1-b94931353eeb None None] REQUEST_METHOD: `POST` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:10:52.757490 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b57ea9b9-5541-4a11-98c1-b94931353eeb None None] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:10:52.765356 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b57ea9b9-5541-4a11-98c1-b94931353eeb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:10:52.788029 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 858/1692] 10.222.0.15 () {58 vars in 1224 bytes} [Mon Aug 22 16:10:52 2022] DELETE /identity/v3/users/844f37c1f52e47c692ddabb07d807f39 => generated 0 bytes in 742 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 16:10:52.814584 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8e6ae9c-a9fd-497a-b0b5-ebc5ddba265d admin admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 16:10:52.816350 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-a8e6ae9c-a9fd-497a-b0b5-ebc5ddba265d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 16:10:52.840057 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.auth.core [None req-b57ea9b9-5541-4a11-98c1-b94931353eeb None None] MFA Rules not processed for user `dbb8092b0c6d410e93a3f397e282157f`. Rule list: `[]` (Enabled: `True`). {{(pid=81767) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 16:10:52.841842 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.fernet_utils [None req-b57ea9b9-5541-4a11-98c1-b94931353eeb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81767) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 16:10:53.071440 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8e6ae9c-a9fd-497a-b0b5-ebc5ddba265d admin admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 16:10:53.103125 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 835/1693] 10.222.0.15 () {58 vars in 958 bytes} [Mon Aug 22 16:10:52 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 347 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 16:10:53.115999 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3c0e395-7cca-49d7-b2ab-d5c32e895079 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a98b1e4431940ab8d1fc95df2858a6c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 16:10:53.116294 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b57ea9b9-5541-4a11-98c1-b94931353eeb None None] Authenticating user token {{(pid=81767) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 16:10:53.116653 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c0e395-7cca-49d7-b2ab-d5c32e895079 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:10:53.117038 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c0e395-7cca-49d7-b2ab-d5c32e895079 admin admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:10:53.117306 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c0e395-7cca-49d7-b2ab-d5c32e895079 admin admin] PATH_INFO: `/v3/users/a293f8ba66b74eb5bd616ce7674e177f` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:10:53.119506 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3c0e395-7cca-49d7-b2ab-d5c32e895079 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a293f8ba66b74eb5bd616ce7674e177f)` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 16:10:53.119774 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.fernet_utils [None req-b57ea9b9-5541-4a11-98c1-b94931353eeb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81767) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 16:10:53.121451 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3c0e395-7cca-49d7-b2ab-d5c32e895079 admin admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 16:10:53.371729 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b57ea9b9-5541-4a11-98c1-b94931353eeb None None] Validating token access rules against request {{(pid=81767) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 16:10:53.385463 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8abbcea6-fe23-4436-8601-b97b89864763 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4ef569e41fb46d0b721bb1c7663616f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4ee0413f9ef4d4fb1f95a660babe090', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81767) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 16:10:53.386470 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8abbcea6-fe23-4436-8601-b97b89864763 service nova] REQUEST_METHOD: `GET` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:10:53.386880 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8abbcea6-fe23-4436-8601-b97b89864763 service nova] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:10:53.387301 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8abbcea6-fe23-4436-8601-b97b89864763 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:10:53.405144 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-a3c0e395-7cca-49d7-b2ab-d5c32e895079 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a293f8ba66b74eb5bd616ce7674e177f'} {{(pid=81768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 16:10:53.408140 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-a3c0e395-7cca-49d7-b2ab-d5c32e895079 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a293f8ba66b74eb5bd616ce7674e177f'} {{(pid=81768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 16:10:53.409298 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-a3c0e395-7cca-49d7-b2ab-d5c32e895079 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a293f8ba66b74eb5bd616ce7674e177f'} {{(pid=81768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 16:10:53.425620 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8abbcea6-fe23-4436-8601-b97b89864763 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 16:10:53.427193 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8abbcea6-fe23-4436-8601-b97b89864763 service nova] RBAC: Authorization granted {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 16:10:53.439094 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-a3c0e395-7cca-49d7-b2ab-d5c32e895079 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a293f8ba66b74eb5bd616ce7674e177f'} {{(pid=81768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 16:10:53.471700 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 836/1694] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Aug 22 16:10:53 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 356 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 16:10:53.552013 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-a3c0e395-7cca-49d7-b2ab-d5c32e895079 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a293f8ba66b74eb5bd616ce7674e177f'} {{(pid=81768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 16:10:53.600213 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 859/1695] 10.222.0.15 () {58 vars in 1224 bytes} [Mon Aug 22 16:10:52 2022] DELETE /identity/v3/users/a293f8ba66b74eb5bd616ce7674e177f => generated 0 bytes in 790 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 16:10:53.605106 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e36eb67-5fd1-4e8a-9a53-56d91700511e None None] REQUEST_METHOD: `POST` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:10:53.605601 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e36eb67-5fd1-4e8a-9a53-56d91700511e None None] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:10:53.606051 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e36eb67-5fd1-4e8a-9a53-56d91700511e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:10:53.652081 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.auth.core [None req-7e36eb67-5fd1-4e8a-9a53-56d91700511e None None] MFA Rules not processed for user `6b2199ae0fb0480aa917ada391f5ec54`. Rule list: `[]` (Enabled: `True`). {{(pid=81767) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 16:10:53.660716 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.fernet_utils [None req-7e36eb67-5fd1-4e8a-9a53-56d91700511e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81767) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 16:10:53.817292 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG dbcounter [-] [81767] Writing DB stats keystone:SELECT=100 {{(pid=81767) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 16:10:53.840588 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 837/1696] 10.222.0.15 () {58 vars in 958 bytes} [Mon Aug 22 16:10:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 234 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 16:10:53.863772 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3c0e395-7cca-49d7-b2ab-d5c32e895079 admin admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 16:10:53.865736 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-a3c0e395-7cca-49d7-b2ab-d5c32e895079 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 16:10:53.920507 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:DELETE=16,keystone:SELECT=82,keystone:INSERT=2 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 16:10:54.001484 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3c0e395-7cca-49d7-b2ab-d5c32e895079 admin admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 16:10:54.012564 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a607543-6293-4dfa-a100-018bd652be82 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '234687adad594f1aa45775a565ffbdc7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4ee0413f9ef4d4fb1f95a660babe090', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 16:10:54.012769 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a607543-6293-4dfa-a100-018bd652be82 service neutron] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:10:54.012973 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a607543-6293-4dfa-a100-018bd652be82 service neutron] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:10:54.013245 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a607543-6293-4dfa-a100-018bd652be82 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:10:54.028763 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a607543-6293-4dfa-a100-018bd652be82 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 16:10:54.029970 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a607543-6293-4dfa-a100-018bd652be82 service neutron] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 16:10:54.061554 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 860/1697] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Aug 22 16:10:53 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 199 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 16:10:54.500641 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e36eb67-5fd1-4e8a-9a53-56d91700511e None None] Authenticating user token {{(pid=81767) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 16:10:54.502538 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.fernet_utils [None req-7e36eb67-5fd1-4e8a-9a53-56d91700511e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81767) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 16:10:54.528758 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e36eb67-5fd1-4e8a-9a53-56d91700511e None None] Validating token access rules against request {{(pid=81767) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 16:10:54.541938 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed48b29c-4678-4458-b83e-d44f708d3e68 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a98b1e4431940ab8d1fc95df2858a6c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81767) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 16:10:54.542048 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed48b29c-4678-4458-b83e-d44f708d3e68 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:10:54.542270 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed48b29c-4678-4458-b83e-d44f708d3e68 admin admin] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:10:54.543946 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed48b29c-4678-4458-b83e-d44f708d3e68 admin admin] PATH_INFO: `/v3/projects/aa723f69df45423b88bbed0c958be4b6` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:10:54.552646 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed48b29c-4678-4458-b83e-d44f708d3e68 admin admin] RBAC: Authorizing `identity:delete_project(project_id=aa723f69df45423b88bbed0c958be4b6)` {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 16:10:54.555544 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed48b29c-4678-4458-b83e-d44f708d3e68 admin admin] RBAC: Authorization granted {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 16:10:54.793907 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.notifications [None req-ed48b29c-4678-4458-b83e-d44f708d3e68 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'aa723f69df45423b88bbed0c958be4b6', 'request_id': 'req-ed48b29c-4678-4458-b83e-d44f708d3e68'} {{(pid=81767) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 16:10:54.811432 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.notifications [None req-ed48b29c-4678-4458-b83e-d44f708d3e68 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'aa723f69df45423b88bbed0c958be4b6', 'request_id': 'req-ed48b29c-4678-4458-b83e-d44f708d3e68'} {{(pid=81767) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 16:10:54.976849 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.notifications [None req-ed48b29c-4678-4458-b83e-d44f708d3e68 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'aa723f69df45423b88bbed0c958be4b6', 'request_id': 'req-ed48b29c-4678-4458-b83e-d44f708d3e68'} {{(pid=81767) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 16:10:54.994357 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.notifications [None req-ed48b29c-4678-4458-b83e-d44f708d3e68 admin admin] The token cache is being invalidate because project aa723f69df45423b88bbed0c958be4b6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81767) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 22 16:10:54.995143 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.notifications [None req-ed48b29c-4678-4458-b83e-d44f708d3e68 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81767) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 16:10:54.996300 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.notifications [None req-ed48b29c-4678-4458-b83e-d44f708d3e68 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81767) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 16:10:55.002644 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 838/1698] 10.222.0.15 () {58 vars in 1233 bytes} [Mon Aug 22 16:10:54 2022] DELETE /identity/v3/projects/aa723f69df45423b88bbed0c958be4b6 => generated 0 bytes in 499 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 16:10:55.006401 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a607543-6293-4dfa-a100-018bd652be82 service neutron] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 16:10:55.028914 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a607543-6293-4dfa-a100-018bd652be82 service neutron] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 16:10:55.126556 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b617121f-be67-4f7b-859d-2edd450b678d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a98b1e4431940ab8d1fc95df2858a6c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 16:10:55.127194 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b617121f-be67-4f7b-859d-2edd450b678d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:10:55.127499 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b617121f-be67-4f7b-859d-2edd450b678d admin admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:10:55.127770 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b617121f-be67-4f7b-859d-2edd450b678d admin admin] PATH_INFO: `/v3/users/c3a0c958112f44d28aa13df75d5612cc` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:10:55.129070 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b617121f-be67-4f7b-859d-2edd450b678d admin admin] RBAC: Authorizing `identity:delete_user(user_id=c3a0c958112f44d28aa13df75d5612cc)` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 16:10:55.131770 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b617121f-be67-4f7b-859d-2edd450b678d admin admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 16:10:55.437741 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-b617121f-be67-4f7b-859d-2edd450b678d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c3a0c958112f44d28aa13df75d5612cc'} {{(pid=81768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 16:10:55.449995 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-b617121f-be67-4f7b-859d-2edd450b678d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c3a0c958112f44d28aa13df75d5612cc'} {{(pid=81768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 16:10:55.451171 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-b617121f-be67-4f7b-859d-2edd450b678d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c3a0c958112f44d28aa13df75d5612cc'} {{(pid=81768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 16:10:55.465608 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-b617121f-be67-4f7b-859d-2edd450b678d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c3a0c958112f44d28aa13df75d5612cc'} {{(pid=81768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 16:10:55.538614 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-b617121f-be67-4f7b-859d-2edd450b678d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c3a0c958112f44d28aa13df75d5612cc'} {{(pid=81768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 16:10:55.576152 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 861/1699] 10.222.0.15 () {58 vars in 1224 bytes} [Mon Aug 22 16:10:55 2022] DELETE /identity/v3/users/c3a0c958112f44d28aa13df75d5612cc => generated 0 bytes in 573 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 22 16:10:55.722950 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed48b29c-4678-4458-b83e-d44f708d3e68 admin admin] Authenticating user token {{(pid=81767) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 16:10:55.731006 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.fernet_utils [None req-ed48b29c-4678-4458-b83e-d44f708d3e68 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81767) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 16:10:55.771483 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed48b29c-4678-4458-b83e-d44f708d3e68 admin admin] Validating token access rules against request {{(pid=81767) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 16:10:55.801510 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee553db1-5d17-49f4-b1b7-0403a6abf09d None None] REQUEST_METHOD: `POST` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:10:55.801805 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee553db1-5d17-49f4-b1b7-0403a6abf09d None None] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:10:55.802723 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee553db1-5d17-49f4-b1b7-0403a6abf09d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:10:55.846368 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.auth.core [None req-ee553db1-5d17-49f4-b1b7-0403a6abf09d None None] MFA Rules not processed for user `6ec6b84e494c481abe9c9772a10037c9`. Rule list: `[]` (Enabled: `True`). {{(pid=81768) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 16:10:55.848957 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-ee553db1-5d17-49f4-b1b7-0403a6abf09d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 16:10:55.964853 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-567e0817-4de5-48c3-927e-985532aa51ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a98b1e4431940ab8d1fc95df2858a6c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81767) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 16:10:55.965645 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-567e0817-4de5-48c3-927e-985532aa51ec admin admin] REQUEST_METHOD: `DELETE` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:10:55.966007 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-567e0817-4de5-48c3-927e-985532aa51ec admin admin] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:10:55.966281 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-567e0817-4de5-48c3-927e-985532aa51ec admin admin] PATH_INFO: `/v3/projects/847871389fa24926ba5b1608ec48b7d2` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:10:55.969365 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-567e0817-4de5-48c3-927e-985532aa51ec admin admin] RBAC: Authorizing `identity:delete_project(project_id=847871389fa24926ba5b1608ec48b7d2)` {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 16:10:55.972122 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-567e0817-4de5-48c3-927e-985532aa51ec admin admin] RBAC: Authorization granted {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 16:10:55.990217 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 862/1700] 10.222.0.15 () {58 vars in 958 bytes} [Mon Aug 22 16:10:55 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 16:10:55.990217 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee553db1-5d17-49f4-b1b7-0403a6abf09d None None] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 16:10:55.990217 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-ee553db1-5d17-49f4-b1b7-0403a6abf09d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 16:10:56.167619 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.notifications [None req-567e0817-4de5-48c3-927e-985532aa51ec admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '847871389fa24926ba5b1608ec48b7d2', 'request_id': 'req-567e0817-4de5-48c3-927e-985532aa51ec'} {{(pid=81767) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 16:10:56.178548 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.notifications [None req-567e0817-4de5-48c3-927e-985532aa51ec admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '847871389fa24926ba5b1608ec48b7d2', 'request_id': 'req-567e0817-4de5-48c3-927e-985532aa51ec'} {{(pid=81767) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 16:10:56.213450 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee553db1-5d17-49f4-b1b7-0403a6abf09d None None] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 16:10:56.249280 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.notifications [None req-567e0817-4de5-48c3-927e-985532aa51ec admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '847871389fa24926ba5b1608ec48b7d2', 'request_id': 'req-567e0817-4de5-48c3-927e-985532aa51ec'} {{(pid=81767) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 16:10:56.261069 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG dbcounter [-] [81767] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81767) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 16:10:56.268805 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.notifications [None req-567e0817-4de5-48c3-927e-985532aa51ec admin admin] The token cache is being invalidate because project 847871389fa24926ba5b1608ec48b7d2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81767) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 22 16:10:56.269510 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.notifications [None req-567e0817-4de5-48c3-927e-985532aa51ec admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81767) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 16:10:56.270430 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.notifications [None req-567e0817-4de5-48c3-927e-985532aa51ec admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81767) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 16:10:56.274417 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 839/1701] 10.222.0.15 () {58 vars in 1233 bytes} [Mon Aug 22 16:10:55 2022] DELETE /identity/v3/projects/847871389fa24926ba5b1608ec48b7d2 => generated 0 bytes in 552 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 16:10:56.276197 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b3e9801-02fb-4c2e-a3b3-60338299674c None None] REQUEST_METHOD: `POST` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:10:56.277026 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b3e9801-02fb-4c2e-a3b3-60338299674c None None] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:10:56.277702 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b3e9801-02fb-4c2e-a3b3-60338299674c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:10:56.295540 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7d86913-c1f3-40a1-99ef-0b17abcfd7b7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79706d480f864998b9f8594be407ceab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4ee0413f9ef4d4fb1f95a660babe090', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 16:10:56.298501 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7d86913-c1f3-40a1-99ef-0b17abcfd7b7 service cinder] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:10:56.298501 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7d86913-c1f3-40a1-99ef-0b17abcfd7b7 service cinder] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:10:56.298501 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7d86913-c1f3-40a1-99ef-0b17abcfd7b7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:10:56.299766 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-f7d86913-c1f3-40a1-99ef-0b17abcfd7b7 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 16:10:56.353197 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.auth.core [None req-9b3e9801-02fb-4c2e-a3b3-60338299674c None None] MFA Rules not processed for user `a377dda3bddf4346b9b85a2d99228ae1`. Rule list: `[]` (Enabled: `True`). {{(pid=81767) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 16:10:56.358792 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.fernet_utils [None req-9b3e9801-02fb-4c2e-a3b3-60338299674c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81767) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 16:10:56.460006 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7d86913-c1f3-40a1-99ef-0b17abcfd7b7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 16:10:56.461846 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7d86913-c1f3-40a1-99ef-0b17abcfd7b7 service cinder] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 16:10:56.474466 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 863/1702] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Aug 22 16:10:55 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 488 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 16:10:56.475741 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7d86913-c1f3-40a1-99ef-0b17abcfd7b7 service cinder] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 16:10:56.477560 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-f7d86913-c1f3-40a1-99ef-0b17abcfd7b7 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 16:10:56.548823 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 840/1703] 10.222.0.15 () {58 vars in 958 bytes} [Mon Aug 22 16:10:56 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 273 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 16:10:56.560198 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60350a71-3e14-42b2-9465-53462d97af00 None None] REQUEST_METHOD: `POST` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:10:56.560550 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60350a71-3e14-42b2-9465-53462d97af00 None None] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:10:56.560754 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60350a71-3e14-42b2-9465-53462d97af00 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:10:56.643793 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.auth.core [None req-60350a71-3e14-42b2-9465-53462d97af00 None None] MFA Rules not processed for user `6b2199ae0fb0480aa917ada391f5ec54`. Rule list: `[]` (Enabled: `True`). {{(pid=81767) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 16:10:56.645606 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.fernet_utils [None req-60350a71-3e14-42b2-9465-53462d97af00 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81767) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 16:10:56.690901 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 841/1704] 10.222.0.15 () {58 vars in 958 bytes} [Mon Aug 22 16:10:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 16:10:56.696736 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f471b8de-abe9-4a17-af07-8dec3fd59c8e None None] REQUEST_METHOD: `POST` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:10:56.705501 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f471b8de-abe9-4a17-af07-8dec3fd59c8e None None] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:10:56.705796 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f471b8de-abe9-4a17-af07-8dec3fd59c8e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:10:56.706664 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 16:10:56.713286 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7d86913-c1f3-40a1-99ef-0b17abcfd7b7 service cinder] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 16:10:56.724822 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c67da992-9906-4807-a25f-dcca666e9ec8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a98b1e4431940ab8d1fc95df2858a6c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 16:10:56.724822 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c67da992-9906-4807-a25f-dcca666e9ec8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:10:56.724822 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c67da992-9906-4807-a25f-dcca666e9ec8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:10:56.733756 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c67da992-9906-4807-a25f-dcca666e9ec8 admin admin] PATH_INFO: `/v3/projects/db211d338fe64470831d2b171e5fd88e` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:10:56.735884 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c67da992-9906-4807-a25f-dcca666e9ec8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=db211d338fe64470831d2b171e5fd88e)` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 16:10:56.737996 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c67da992-9906-4807-a25f-dcca666e9ec8 admin admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 16:10:56.747921 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.auth.core [None req-f471b8de-abe9-4a17-af07-8dec3fd59c8e None None] MFA Rules not processed for user `a377dda3bddf4346b9b85a2d99228ae1`. Rule list: `[]` (Enabled: `True`). {{(pid=81767) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 16:10:56.749158 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.fernet_utils [None req-f471b8de-abe9-4a17-af07-8dec3fd59c8e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81767) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 16:10:56.768838 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 842/1705] 10.222.0.15 () {58 vars in 958 bytes} [Mon Aug 22 16:10:56 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 16:10:56.771859 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed01b957-7c8b-46b2-9aae-e4fabf0e1c1f None None] REQUEST_METHOD: `POST` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:10:56.773865 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed01b957-7c8b-46b2-9aae-e4fabf0e1c1f None None] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:10:56.776267 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed01b957-7c8b-46b2-9aae-e4fabf0e1c1f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:10:56.815811 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.auth.core [None req-ed01b957-7c8b-46b2-9aae-e4fabf0e1c1f None None] MFA Rules not processed for user `6b2199ae0fb0480aa917ada391f5ec54`. Rule list: `[]` (Enabled: `True`). {{(pid=81767) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 16:10:56.817232 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.fernet_utils [None req-ed01b957-7c8b-46b2-9aae-e4fabf0e1c1f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81767) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 16:10:56.827383 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 843/1706] 10.222.0.15 () {58 vars in 958 bytes} [Mon Aug 22 16:10:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 58 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 16:10:56.828478 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed01b957-7c8b-46b2-9aae-e4fabf0e1c1f None None] Authenticating user token {{(pid=81767) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 16:10:56.830474 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.fernet_utils [None req-ed01b957-7c8b-46b2-9aae-e4fabf0e1c1f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81767) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 16:10:56.908243 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-c67da992-9906-4807-a25f-dcca666e9ec8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'db211d338fe64470831d2b171e5fd88e', 'request_id': 'req-c67da992-9906-4807-a25f-dcca666e9ec8'} {{(pid=81768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 16:10:56.919107 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-c67da992-9906-4807-a25f-dcca666e9ec8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'db211d338fe64470831d2b171e5fd88e', 'request_id': 'req-c67da992-9906-4807-a25f-dcca666e9ec8'} {{(pid=81768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 16:10:56.963933 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed01b957-7c8b-46b2-9aae-e4fabf0e1c1f None None] Validating token access rules against request {{(pid=81767) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 16:10:57.005224 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-c67da992-9906-4807-a25f-dcca666e9ec8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'db211d338fe64470831d2b171e5fd88e', 'request_id': 'req-c67da992-9906-4807-a25f-dcca666e9ec8'} {{(pid=81768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 16:10:57.030636 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-c67da992-9906-4807-a25f-dcca666e9ec8 admin admin] The token cache is being invalidate because project db211d338fe64470831d2b171e5fd88e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81768) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 22 16:10:57.030636 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-c67da992-9906-4807-a25f-dcca666e9ec8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 16:10:57.032945 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-c67da992-9906-4807-a25f-dcca666e9ec8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 16:10:57.038639 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 864/1707] 10.222.0.15 () {58 vars in 1233 bytes} [Mon Aug 22 16:10:56 2022] DELETE /identity/v3/projects/db211d338fe64470831d2b171e5fd88e => generated 0 bytes in 563 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 16:10:57.039615 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c67da992-9906-4807-a25f-dcca666e9ec8 admin admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 16:10:57.048342 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-c67da992-9906-4807-a25f-dcca666e9ec8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 16:10:57.097449 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f6e73b4-f13b-44fe-aaa7-adaf3f1d549c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4ef569e41fb46d0b721bb1c7663616f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4ee0413f9ef4d4fb1f95a660babe090', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81767) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 16:10:57.099686 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f6e73b4-f13b-44fe-aaa7-adaf3f1d549c service nova] REQUEST_METHOD: `GET` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:10:57.101548 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f6e73b4-f13b-44fe-aaa7-adaf3f1d549c service nova] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:10:57.101925 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f6e73b4-f13b-44fe-aaa7-adaf3f1d549c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:10:57.104044 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.fernet_utils [None req-8f6e73b4-f13b-44fe-aaa7-adaf3f1d549c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81767) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 16:10:57.163844 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c67da992-9906-4807-a25f-dcca666e9ec8 admin admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 16:10:57.169998 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e8d9f30-ec67-4d8b-b518-0285d514b646 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a98b1e4431940ab8d1fc95df2858a6c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 16:10:57.170631 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e8d9f30-ec67-4d8b-b518-0285d514b646 admin admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:10:57.171885 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e8d9f30-ec67-4d8b-b518-0285d514b646 admin admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:10:57.172558 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e8d9f30-ec67-4d8b-b518-0285d514b646 admin admin] PATH_INFO: `/v3/domains` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:10:57.173360 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e8d9f30-ec67-4d8b-b518-0285d514b646 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81768) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 16:10:57.174478 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e8d9f30-ec67-4d8b-b518-0285d514b646 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 16:10:57.176253 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e8d9f30-ec67-4d8b-b518-0285d514b646 admin admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 16:10:57.195335 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 865/1708] 10.222.0.15 () {58 vars in 1153 bytes} [Mon Aug 22 16:10:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 16:10:57.196635 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-670ffb0c-df08-4457-a65e-a81389047660 None None] REQUEST_METHOD: `POST` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:10:57.197090 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-670ffb0c-df08-4457-a65e-a81389047660 None None] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:10:57.197637 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-670ffb0c-df08-4457-a65e-a81389047660 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:10:57.205428 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application [None req-670ffb0c-df08-4457-a65e-a81389047660 None None] Expecting to find domain in project. The server could not comply with the request since it is either malformed or otherwise incorrect. The client is assumed to be in error.: keystone.exception.ValidationError: Expecting to find domain in project. The server could not comply with the request since it is either malformed or otherwise incorrect. The client is assumed to be in error. Aug 22 16:10:57.205428 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 16:10:57.205428 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 16:10:57.205428 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 16:10:57.205428 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 16:10:57.205428 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 16:10:57.205428 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 16:10:57.205428 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 16:10:57.205428 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 16:10:57.205428 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 16:10:57.205428 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 16:10:57.205428 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 16:10:57.205428 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/server/flask/common.py", line 1064, in wrapper Aug 22 16:10:57.206262 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return f(*args, **kwargs) Aug 22 16:10:57.206262 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/auth.py", line 315, in post Aug 22 16:10:57.206262 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application token = authentication.authenticate_for_token(auth_data) Aug 22 16:10:57.206262 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/_shared/authentication.py", line 182, in authenticate_for_token Aug 22 16:10:57.206262 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application auth_info = core.AuthInfo.create(auth=auth) Aug 22 16:10:57.206262 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/auth/core.py", line 126, in create Aug 22 16:10:57.206262 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application auth_info._validate_and_normalize_auth_data(scope_only) Aug 22 16:10:57.206262 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/auth/core.py", line 340, in _validate_and_normalize_auth_data Aug 22 16:10:57.206262 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application self._validate_and_normalize_scope_data() Aug 22 16:10:57.206262 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/auth/core.py", line 290, in _validate_and_normalize_scope_data Aug 22 16:10:57.206262 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application project_ref = self._lookup_project(self.auth['scope']['project']) Aug 22 16:10:57.206262 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/auth/core.py", line 194, in _lookup_project Aug 22 16:10:57.206262 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application raise exception.ValidationError(attribute='domain', Aug 22 16:10:57.206262 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application keystone.exception.ValidationError: Expecting to find domain in project. The server could not comply with the request since it is either malformed or otherwise incorrect. The client is assumed to be in error. Aug 22 16:10:57.206262 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application Aug 22 16:10:57.207600 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 866/1709] 10.222.0.15 () {58 vars in 1009 bytes} [Mon Aug 22 16:10:57 2022] POST /identity/v3/auth/tokens => generated 230 bytes in 11 msecs (HTTP/1.1 400) 5 headers in 186 bytes (1 switches on core 0) Aug 22 16:10:57.215962 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a45c4d-2653-482c-a9b2-a1cfc253b592 None None] REQUEST_METHOD: `POST` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:10:57.215962 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a45c4d-2653-482c-a9b2-a1cfc253b592 None None] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:10:57.215962 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a45c4d-2653-482c-a9b2-a1cfc253b592 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:10:57.236963 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f6e73b4-f13b-44fe-aaa7-adaf3f1d549c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 16:10:57.239228 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f6e73b4-f13b-44fe-aaa7-adaf3f1d549c service nova] RBAC: Authorization granted {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 16:10:57.251562 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.auth.core [None req-13a45c4d-2653-482c-a9b2-a1cfc253b592 None None] MFA Rules not processed for user `6b2199ae0fb0480aa917ada391f5ec54`. Rule list: `[]` (Enabled: `True`). {{(pid=81768) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 16:10:57.258088 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-13a45c4d-2653-482c-a9b2-a1cfc253b592 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 16:10:57.260479 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 844/1710] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Aug 22 16:10:56 2022] GET /identity/v3/auth/tokens => generated 3270 bytes in 432 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 16:10:57.262913 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f6e73b4-f13b-44fe-aaa7-adaf3f1d549c service nova] Authenticating user token {{(pid=81767) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 16:10:57.286790 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f6e73b4-f13b-44fe-aaa7-adaf3f1d549c service nova] Validating token access rules against request {{(pid=81767) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 16:10:57.297633 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e9dfa1a-fd00-4f1d-83f3-41d4f5cd8315 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a98b1e4431940ab8d1fc95df2858a6c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81767) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 16:10:57.298543 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e9dfa1a-fd00-4f1d-83f3-41d4f5cd8315 admin admin] REQUEST_METHOD: `POST` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:10:57.300916 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e9dfa1a-fd00-4f1d-83f3-41d4f5cd8315 admin admin] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:10:57.301361 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e9dfa1a-fd00-4f1d-83f3-41d4f5cd8315 admin admin] PATH_INFO: `/v3/projects` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:10:57.302708 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e9dfa1a-fd00-4f1d-83f3-41d4f5cd8315 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 16:10:57.305890 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 867/1711] 10.222.0.15 () {58 vars in 958 bytes} [Mon Aug 22 16:10:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 16:10:57.308159 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e9dfa1a-fd00-4f1d-83f3-41d4f5cd8315 admin admin] RBAC: Authorization granted {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 16:10:57.308995 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5a1e5be-a8c4-471b-b458-fbe65ded0af3 None None] REQUEST_METHOD: `POST` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:10:57.309280 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5a1e5be-a8c4-471b-b458-fbe65ded0af3 None None] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:10:57.309551 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5a1e5be-a8c4-471b-b458-fbe65ded0af3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:10:57.368531 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.auth.core [None req-f5a1e5be-a8c4-471b-b458-fbe65ded0af3 None None] MFA Rules not processed for user `6b2199ae0fb0480aa917ada391f5ec54`. Rule list: `[]` (Enabled: `True`). {{(pid=81768) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 16:10:57.370324 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-f5a1e5be-a8c4-471b-b458-fbe65ded0af3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 16:10:57.387909 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 868/1712] 10.222.0.15 () {58 vars in 958 bytes} [Mon Aug 22 16:10:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 16:10:57.392695 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5a1e5be-a8c4-471b-b458-fbe65ded0af3 None None] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 16:10:57.416098 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5a1e5be-a8c4-471b-b458-fbe65ded0af3 None None] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 16:10:57.431846 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d0e347e-84f7-4286-9c92-99b3304b13c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a98b1e4431940ab8d1fc95df2858a6c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 16:10:57.432479 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d0e347e-84f7-4286-9c92-99b3304b13c6 admin admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:10:57.432766 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d0e347e-84f7-4286-9c92-99b3304b13c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:10:57.433025 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d0e347e-84f7-4286-9c92-99b3304b13c6 admin admin] PATH_INFO: `/v3/domains` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:10:57.433684 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d0e347e-84f7-4286-9c92-99b3304b13c6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81768) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 16:10:57.434009 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d0e347e-84f7-4286-9c92-99b3304b13c6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 16:10:57.436166 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d0e347e-84f7-4286-9c92-99b3304b13c6 admin admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 16:10:57.455862 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 845/1713] 10.222.0.15 () {60 vars in 1153 bytes} [Mon Aug 22 16:10:57 2022] POST /identity/v3/projects => generated 373 bytes in 195 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 22 16:10:57.460822 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e9dfa1a-fd00-4f1d-83f3-41d4f5cd8315 admin admin] Authenticating user token {{(pid=81767) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 16:10:57.486930 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 869/1714] 10.222.0.15 () {58 vars in 1153 bytes} [Mon Aug 22 16:10:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 95 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 16:10:57.491724 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d0e347e-84f7-4286-9c92-99b3304b13c6 admin admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 16:10:57.506509 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d0e347e-84f7-4286-9c92-99b3304b13c6 admin admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 16:10:57.601014 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e9dfa1a-fd00-4f1d-83f3-41d4f5cd8315 admin admin] Validating token access rules against request {{(pid=81767) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 16:10:57.606305 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5818ecb9-62dd-4c49-9dae-22b8c58ea361 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a98b1e4431940ab8d1fc95df2858a6c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81767) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 16:10:57.606916 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5818ecb9-62dd-4c49-9dae-22b8c58ea361 admin admin] REQUEST_METHOD: `POST` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:10:57.607191 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5818ecb9-62dd-4c49-9dae-22b8c58ea361 admin admin] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:10:57.607491 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5818ecb9-62dd-4c49-9dae-22b8c58ea361 admin admin] PATH_INFO: `/v3/users` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:10:57.608316 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5818ecb9-62dd-4c49-9dae-22b8c58ea361 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 16:10:57.610343 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5818ecb9-62dd-4c49-9dae-22b8c58ea361 admin admin] RBAC: Authorization granted {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 16:10:57.621556 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d22b2418-8284-4251-abfd-09e20420c681 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a98b1e4431940ab8d1fc95df2858a6c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 16:10:57.622617 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d22b2418-8284-4251-abfd-09e20420c681 admin admin] REQUEST_METHOD: `POST` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:10:57.623552 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d22b2418-8284-4251-abfd-09e20420c681 admin admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:10:57.623972 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d22b2418-8284-4251-abfd-09e20420c681 admin admin] PATH_INFO: `/v3/projects` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:10:57.624756 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d22b2418-8284-4251-abfd-09e20420c681 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 16:10:57.627184 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d22b2418-8284-4251-abfd-09e20420c681 admin admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 16:10:57.759673 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 846/1715] 10.222.0.15 () {60 vars in 1144 bytes} [Mon Aug 22 16:10:57 2022] POST /identity/v3/users => generated 287 bytes in 297 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 22 16:10:57.760958 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5818ecb9-62dd-4c49-9dae-22b8c58ea361 admin admin] Authenticating user token {{(pid=81767) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 16:10:57.780764 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5818ecb9-62dd-4c49-9dae-22b8c58ea361 admin admin] Validating token access rules against request {{(pid=81767) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 16:10:57.798778 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68b4882b-2c1d-4772-ae3c-db5a8d943a8a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a98b1e4431940ab8d1fc95df2858a6c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81767) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 16:10:57.799529 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68b4882b-2c1d-4772-ae3c-db5a8d943a8a admin admin] REQUEST_METHOD: `GET` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:10:57.800287 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68b4882b-2c1d-4772-ae3c-db5a8d943a8a admin admin] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:10:57.804298 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68b4882b-2c1d-4772-ae3c-db5a8d943a8a admin admin] PATH_INFO: `/v3/roles` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:10:57.805221 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68b4882b-2c1d-4772-ae3c-db5a8d943a8a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 16:10:57.806989 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68b4882b-2c1d-4772-ae3c-db5a8d943a8a admin admin] RBAC: Authorization granted {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 16:10:57.837329 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 847/1716] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Aug 22 16:10:57 2022] GET /identity/v3/roles => generated 1420 bytes in 77 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 22 16:10:57.840630 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68b4882b-2c1d-4772-ae3c-db5a8d943a8a admin admin] Authenticating user token {{(pid=81767) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 16:10:57.853446 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 870/1717] 10.222.0.15 () {60 vars in 1153 bytes} [Mon Aug 22 16:10:57 2022] POST /identity/v3/projects => generated 365 bytes in 361 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 22 16:10:57.855861 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d22b2418-8284-4251-abfd-09e20420c681 admin admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 16:10:57.863048 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68b4882b-2c1d-4772-ae3c-db5a8d943a8a admin admin] Validating token access rules against request {{(pid=81767) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 16:10:57.996001 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ff26f67-0c02-4863-a222-0bc0a1426d43 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a98b1e4431940ab8d1fc95df2858a6c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81767) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 16:10:57.996679 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff26f67-0c02-4863-a222-0bc0a1426d43 admin admin] REQUEST_METHOD: `POST` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:10:57.996968 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff26f67-0c02-4863-a222-0bc0a1426d43 admin admin] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:10:57.997244 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff26f67-0c02-4863-a222-0bc0a1426d43 admin admin] PATH_INFO: `/v3/users` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:10:57.997883 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ff26f67-0c02-4863-a222-0bc0a1426d43 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 16:10:57.999875 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ff26f67-0c02-4863-a222-0bc0a1426d43 admin admin] RBAC: Authorization granted {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 16:10:58.074758 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d22b2418-8284-4251-abfd-09e20420c681 admin admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 16:10:58.087308 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-507703d6-a459-47fe-975c-c8643b84d088 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a98b1e4431940ab8d1fc95df2858a6c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 16:10:58.089449 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-507703d6-a459-47fe-975c-c8643b84d088 admin admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:10:58.092129 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-507703d6-a459-47fe-975c-c8643b84d088 admin admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:10:58.092487 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-507703d6-a459-47fe-975c-c8643b84d088 admin admin] PATH_INFO: `/v3/roles` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:10:58.093271 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-507703d6-a459-47fe-975c-c8643b84d088 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 16:10:58.097106 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-507703d6-a459-47fe-975c-c8643b84d088 admin admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 16:10:58.125874 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 871/1718] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Aug 22 16:10:57 2022] GET /identity/v3/roles => generated 1420 bytes in 271 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 22 16:10:58.130094 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-507703d6-a459-47fe-975c-c8643b84d088 admin admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 16:10:58.153546 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-507703d6-a459-47fe-975c-c8643b84d088 admin admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 16:10:58.159183 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26881480-dada-4145-814c-640750030c63 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a98b1e4431940ab8d1fc95df2858a6c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 16:10:58.160033 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26881480-dada-4145-814c-640750030c63 admin admin] REQUEST_METHOD: `PUT` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:10:58.160431 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26881480-dada-4145-814c-640750030c63 admin admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:10:58.160784 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26881480-dada-4145-814c-640750030c63 admin admin] PATH_INFO: `/v3/projects/51cc967a50404cd6a847e08800d56ac0/users/f0b9bc811f26412b9f18ab4588b852f7/roles/e6fc1ee3fb974125ba9b4fe55b12b45e` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:10:58.181073 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26881480-dada-4145-814c-640750030c63 admin admin] RBAC: Authorizing `identity:create_grant(project_id=51cc967a50404cd6a847e08800d56ac0, user_id=f0b9bc811f26412b9f18ab4588b852f7, role_id=e6fc1ee3fb974125ba9b4fe55b12b45e)` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 16:10:58.184945 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26881480-dada-4145-814c-640750030c63 admin admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 16:10:58.187306 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 848/1719] 10.222.0.15 () {60 vars in 1144 bytes} [Mon Aug 22 16:10:57 2022] POST /identity/v3/users => generated 283 bytes in 347 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 22 16:10:58.188465 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ff26f67-0c02-4863-a222-0bc0a1426d43 admin admin] Authenticating user token {{(pid=81767) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 16:10:58.201530 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ff26f67-0c02-4863-a222-0bc0a1426d43 admin admin] Validating token access rules against request {{(pid=81767) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 16:10:58.209443 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6486e481-d467-4e2f-a0f4-a5f8b2fea442 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a98b1e4431940ab8d1fc95df2858a6c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81767) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 16:10:58.210478 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6486e481-d467-4e2f-a0f4-a5f8b2fea442 admin admin] REQUEST_METHOD: `GET` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:10:58.210920 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6486e481-d467-4e2f-a0f4-a5f8b2fea442 admin admin] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:10:58.211378 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6486e481-d467-4e2f-a0f4-a5f8b2fea442 admin admin] PATH_INFO: `/v3/roles` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:10:58.216318 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6486e481-d467-4e2f-a0f4-a5f8b2fea442 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 16:10:58.216318 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6486e481-d467-4e2f-a0f4-a5f8b2fea442 admin admin] RBAC: Authorization granted {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 16:10:58.229698 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 849/1720] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Aug 22 16:10:58 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 22 16:10:58.237984 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6486e481-d467-4e2f-a0f4-a5f8b2fea442 admin admin] Authenticating user token {{(pid=81767) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 16:10:58.263774 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6486e481-d467-4e2f-a0f4-a5f8b2fea442 admin admin] Validating token access rules against request {{(pid=81767) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 16:10:58.266786 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG dbcounter [-] [81767] Writing DB stats keystone:SELECT=100 {{(pid=81767) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 16:10:58.279524 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 872/1721] 10.222.0.15 () {60 vars in 1483 bytes} [Mon Aug 22 16:10:58 2022] PUT /identity/v3/projects/51cc967a50404cd6a847e08800d56ac0/users/f0b9bc811f26412b9f18ab4588b852f7/roles/e6fc1ee3fb974125ba9b4fe55b12b45e => generated 0 bytes in 142 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 16:10:58.283056 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-313a1fd9-4431-4085-ac81-610c6831a567 None None] REQUEST_METHOD: `POST` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:10:58.283706 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-313a1fd9-4431-4085-ac81-610c6831a567 None None] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:10:58.284122 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-313a1fd9-4431-4085-ac81-610c6831a567 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:10:58.342019 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.auth.core [None req-313a1fd9-4431-4085-ac81-610c6831a567 None None] MFA Rules not processed for user `f0b9bc811f26412b9f18ab4588b852f7`. Rule list: `[]` (Enabled: `True`). {{(pid=81768) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 16:10:58.344237 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-313a1fd9-4431-4085-ac81-610c6831a567 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 16:10:58.402092 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-036119a3-cb27-4ab4-9db1-cc5b6ba9861d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a98b1e4431940ab8d1fc95df2858a6c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81767) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 16:10:58.402714 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-036119a3-cb27-4ab4-9db1-cc5b6ba9861d admin admin] REQUEST_METHOD: `GET` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:10:58.402997 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-036119a3-cb27-4ab4-9db1-cc5b6ba9861d admin admin] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:10:58.403354 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-036119a3-cb27-4ab4-9db1-cc5b6ba9861d admin admin] PATH_INFO: `/v3/roles` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:10:58.403931 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-036119a3-cb27-4ab4-9db1-cc5b6ba9861d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 16:10:58.405562 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-036119a3-cb27-4ab4-9db1-cc5b6ba9861d admin admin] RBAC: Authorization granted {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 16:10:58.433916 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 850/1722] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Aug 22 16:10:58 2022] GET /identity/v3/roles => generated 1420 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 22 16:10:58.435477 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dd98a84-97d2-471a-a43a-caae21891710 None None] REQUEST_METHOD: `GET` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:10:58.435891 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dd98a84-97d2-471a-a43a-caae21891710 None None] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:10:58.436289 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dd98a84-97d2-471a-a43a-caae21891710 None None] PATH_INFO: `/` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:10:58.437784 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 851/1723] 10.222.0.15 () {54 vars in 912 bytes} [Mon Aug 22 16:10:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 16:10:58.438625 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dd98a84-97d2-471a-a43a-caae21891710 None None] Authenticating user token {{(pid=81767) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 16:10:58.468992 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dd98a84-97d2-471a-a43a-caae21891710 None None] Validating token access rules against request {{(pid=81767) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 16:10:58.478787 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9209ee92-2546-4520-9a89-291929409c77 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a98b1e4431940ab8d1fc95df2858a6c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81767) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 16:10:58.479563 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9209ee92-2546-4520-9a89-291929409c77 admin admin] REQUEST_METHOD: `PUT` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:10:58.479834 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9209ee92-2546-4520-9a89-291929409c77 admin admin] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:10:58.480272 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9209ee92-2546-4520-9a89-291929409c77 admin admin] PATH_INFO: `/v3/projects/d7dea9a491404d7d90e4811f75cfe30a/users/a9f3880cbd9e4918ac9deafa18911cb0/roles/e6fc1ee3fb974125ba9b4fe55b12b45e` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:10:58.498930 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 873/1724] 10.222.0.15 () {58 vars in 958 bytes} [Mon Aug 22 16:10:58 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 217 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 16:10:58.501003 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-655ef008-ea63-4f02-80a1-4d37a06db2e7 None None] REQUEST_METHOD: `POST` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:10:58.501529 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-655ef008-ea63-4f02-80a1-4d37a06db2e7 None None] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:10:58.502048 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-655ef008-ea63-4f02-80a1-4d37a06db2e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:10:58.510618 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application [None req-655ef008-ea63-4f02-80a1-4d37a06db2e7 None None] Expecting to find domain in project. The server could not comply with the request since it is either malformed or otherwise incorrect. The client is assumed to be in error.: keystone.exception.ValidationError: Expecting to find domain in project. The server could not comply with the request since it is either malformed or otherwise incorrect. The client is assumed to be in error. Aug 22 16:10:58.510618 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 16:10:58.510618 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 16:10:58.510618 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 16:10:58.510618 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 16:10:58.510618 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 16:10:58.510618 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 16:10:58.510618 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 16:10:58.510618 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 16:10:58.510618 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 16:10:58.510618 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 16:10:58.510618 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 16:10:58.510618 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/server/flask/common.py", line 1064, in wrapper Aug 22 16:10:58.511422 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return f(*args, **kwargs) Aug 22 16:10:58.511422 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/auth.py", line 315, in post Aug 22 16:10:58.511422 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application token = authentication.authenticate_for_token(auth_data) Aug 22 16:10:58.511422 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/_shared/authentication.py", line 182, in authenticate_for_token Aug 22 16:10:58.511422 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application auth_info = core.AuthInfo.create(auth=auth) Aug 22 16:10:58.511422 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/auth/core.py", line 126, in create Aug 22 16:10:58.511422 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application auth_info._validate_and_normalize_auth_data(scope_only) Aug 22 16:10:58.511422 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/auth/core.py", line 340, in _validate_and_normalize_auth_data Aug 22 16:10:58.511422 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application self._validate_and_normalize_scope_data() Aug 22 16:10:58.511422 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/auth/core.py", line 290, in _validate_and_normalize_scope_data Aug 22 16:10:58.511422 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application project_ref = self._lookup_project(self.auth['scope']['project']) Aug 22 16:10:58.511422 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/auth/core.py", line 194, in _lookup_project Aug 22 16:10:58.511422 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application raise exception.ValidationError(attribute='domain', Aug 22 16:10:58.511422 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application keystone.exception.ValidationError: Expecting to find domain in project. The server could not comply with the request since it is either malformed or otherwise incorrect. The client is assumed to be in error. Aug 22 16:10:58.511422 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application Aug 22 16:10:58.514143 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9209ee92-2546-4520-9a89-291929409c77 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d7dea9a491404d7d90e4811f75cfe30a, user_id=a9f3880cbd9e4918ac9deafa18911cb0, role_id=e6fc1ee3fb974125ba9b4fe55b12b45e)` {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 16:10:58.514454 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 874/1725] 10.222.0.15 () {58 vars in 1009 bytes} [Mon Aug 22 16:10:58 2022] POST /identity/v3/auth/tokens => generated 230 bytes in 15 msecs (HTTP/1.1 400) 5 headers in 186 bytes (1 switches on core 0) Aug 22 16:10:58.515608 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-655ef008-ea63-4f02-80a1-4d37a06db2e7 None None] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 16:10:58.516944 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9209ee92-2546-4520-9a89-291929409c77 admin admin] RBAC: Authorization granted {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 16:10:58.532554 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-655ef008-ea63-4f02-80a1-4d37a06db2e7 None None] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 16:10:58.581151 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38a0ed39-bece-430e-8c60-1dc6be807f6f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a98b1e4431940ab8d1fc95df2858a6c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 16:10:58.581996 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38a0ed39-bece-430e-8c60-1dc6be807f6f admin admin] REQUEST_METHOD: `POST` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:10:58.583173 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38a0ed39-bece-430e-8c60-1dc6be807f6f admin admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:10:58.583896 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38a0ed39-bece-430e-8c60-1dc6be807f6f admin admin] PATH_INFO: `/v3/projects` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:10:58.585178 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38a0ed39-bece-430e-8c60-1dc6be807f6f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 16:10:58.601806 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38a0ed39-bece-430e-8c60-1dc6be807f6f admin admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 16:10:58.657178 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 852/1726] 10.222.0.15 () {60 vars in 1483 bytes} [Mon Aug 22 16:10:58 2022] PUT /identity/v3/projects/d7dea9a491404d7d90e4811f75cfe30a/users/a9f3880cbd9e4918ac9deafa18911cb0/roles/e6fc1ee3fb974125ba9b4fe55b12b45e => generated 0 bytes in 219 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 16:10:58.672051 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16a43f6b-789e-4214-b8c8-fa5019720f23 None None] REQUEST_METHOD: `POST` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:10:58.672490 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16a43f6b-789e-4214-b8c8-fa5019720f23 None None] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:10:58.673095 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16a43f6b-789e-4214-b8c8-fa5019720f23 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:10:58.685565 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=100 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 16:10:58.757379 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.auth.core [None req-16a43f6b-789e-4214-b8c8-fa5019720f23 None None] MFA Rules not processed for user `a9f3880cbd9e4918ac9deafa18911cb0`. Rule list: `[]` (Enabled: `True`). {{(pid=81767) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 16:10:58.759097 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.fernet_utils [None req-16a43f6b-789e-4214-b8c8-fa5019720f23 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81767) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 16:10:58.817027 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 875/1727] 10.222.0.15 () {60 vars in 1153 bytes} [Mon Aug 22 16:10:58 2022] POST /identity/v3/projects => generated 371 bytes in 302 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 22 16:10:58.818510 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38a0ed39-bece-430e-8c60-1dc6be807f6f admin admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 16:10:58.936141 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 853/1728] 10.222.0.15 () {58 vars in 958 bytes} [Mon Aug 22 16:10:58 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 265 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 16:10:58.940346 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16a43f6b-789e-4214-b8c8-fa5019720f23 None None] Authenticating user token {{(pid=81767) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 16:10:59.004153 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG dbcounter [-] [81767] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81767) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 16:10:59.102339 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16a43f6b-789e-4214-b8c8-fa5019720f23 None None] Validating token access rules against request {{(pid=81767) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 16:10:59.115601 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38a0ed39-bece-430e-8c60-1dc6be807f6f admin admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 16:10:59.128738 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dfec082-130c-45d2-8f9a-98a014374e85 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a98b1e4431940ab8d1fc95df2858a6c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81767) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 16:10:59.129621 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dfec082-130c-45d2-8f9a-98a014374e85 admin admin] REQUEST_METHOD: `POST` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:10:59.130082 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dfec082-130c-45d2-8f9a-98a014374e85 admin admin] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:10:59.131046 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dfec082-130c-45d2-8f9a-98a014374e85 admin admin] PATH_INFO: `/v3/projects` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:10:59.131886 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dfec082-130c-45d2-8f9a-98a014374e85 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 16:10:59.134781 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dfec082-130c-45d2-8f9a-98a014374e85 admin admin] RBAC: Authorization granted {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 16:10:59.134933 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8141eab0-2530-4b99-8972-4e08126578db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a98b1e4431940ab8d1fc95df2858a6c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 16:10:59.135562 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8141eab0-2530-4b99-8972-4e08126578db admin admin] REQUEST_METHOD: `POST` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:10:59.135844 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8141eab0-2530-4b99-8972-4e08126578db admin admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:10:59.136193 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8141eab0-2530-4b99-8972-4e08126578db admin admin] PATH_INFO: `/v3/users` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:10:59.136831 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8141eab0-2530-4b99-8972-4e08126578db admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 16:10:59.139057 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8141eab0-2530-4b99-8972-4e08126578db admin admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 16:10:59.257741 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 854/1729] 10.222.0.15 () {60 vars in 1153 bytes} [Mon Aug 22 16:10:58 2022] POST /identity/v3/projects => generated 363 bytes in 318 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 22 16:10:59.259166 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c169452-5963-4e93-ac0f-f049941e5520 None None] REQUEST_METHOD: `POST` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:10:59.259513 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c169452-5963-4e93-ac0f-f049941e5520 None None] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:10:59.259782 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c169452-5963-4e93-ac0f-f049941e5520 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:10:59.261654 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 876/1730] 10.222.0.15 () {60 vars in 1144 bytes} [Mon Aug 22 16:10:58 2022] POST /identity/v3/users => generated 292 bytes in 444 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 22 16:10:59.262785 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8141eab0-2530-4b99-8972-4e08126578db admin admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 16:10:59.283148 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8141eab0-2530-4b99-8972-4e08126578db admin admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 16:10:59.291853 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.auth.core [None req-9c169452-5963-4e93-ac0f-f049941e5520 None None] MFA Rules not processed for user `dbb8092b0c6d410e93a3f397e282157f`. Rule list: `[]` (Enabled: `True`). {{(pid=81767) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 16:10:59.300867 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.fernet_utils [None req-9c169452-5963-4e93-ac0f-f049941e5520 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81767) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 16:10:59.400610 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 16:10:59.494289 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-659fbb21-4501-4bff-a4aa-681fefe6b05a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a98b1e4431940ab8d1fc95df2858a6c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 16:10:59.497466 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-659fbb21-4501-4bff-a4aa-681fefe6b05a admin admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:10:59.497466 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-659fbb21-4501-4bff-a4aa-681fefe6b05a admin admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:10:59.497466 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-659fbb21-4501-4bff-a4aa-681fefe6b05a admin admin] PATH_INFO: `/v3/roles` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:10:59.497466 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-659fbb21-4501-4bff-a4aa-681fefe6b05a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 16:10:59.498997 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 855/1731] 10.222.0.15 () {58 vars in 958 bytes} [Mon Aug 22 16:10:59 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 241 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 16:10:59.500062 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c169452-5963-4e93-ac0f-f049941e5520 None None] Authenticating user token {{(pid=81767) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 16:10:59.500642 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-659fbb21-4501-4bff-a4aa-681fefe6b05a admin admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 16:10:59.530873 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c169452-5963-4e93-ac0f-f049941e5520 None None] Validating token access rules against request {{(pid=81767) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 16:10:59.540068 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 877/1732] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Aug 22 16:10:59 2022] GET /identity/v3/roles => generated 1420 bytes in 276 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 22 16:10:59.541514 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-659fbb21-4501-4bff-a4aa-681fefe6b05a admin admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 16:10:59.545520 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcbb6e41-b073-4ef2-90a9-74c7bb67c8ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a98b1e4431940ab8d1fc95df2858a6c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81767) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 16:10:59.546694 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcbb6e41-b073-4ef2-90a9-74c7bb67c8ae admin admin] REQUEST_METHOD: `POST` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:10:59.550249 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcbb6e41-b073-4ef2-90a9-74c7bb67c8ae admin admin] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:10:59.550741 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcbb6e41-b073-4ef2-90a9-74c7bb67c8ae admin admin] PATH_INFO: `/v3/users` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:10:59.554881 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcbb6e41-b073-4ef2-90a9-74c7bb67c8ae admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 16:10:59.560284 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcbb6e41-b073-4ef2-90a9-74c7bb67c8ae admin admin] RBAC: Authorization granted {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 16:10:59.600555 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-659fbb21-4501-4bff-a4aa-681fefe6b05a admin admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 16:10:59.619263 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-428ee335-ec2c-4f46-bb25-81bae807f7c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a98b1e4431940ab8d1fc95df2858a6c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 16:10:59.624189 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-428ee335-ec2c-4f46-bb25-81bae807f7c9 admin admin] REQUEST_METHOD: `PUT` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:10:59.624699 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-428ee335-ec2c-4f46-bb25-81bae807f7c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:10:59.627200 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-428ee335-ec2c-4f46-bb25-81bae807f7c9 admin admin] PATH_INFO: `/v3/projects/097634d5db204edab1408e1752d662ba/users/d7c5d886b4a54300b818b5e7c6bdbe1d/roles/9d9848cfd22e4b089db443c350e56670` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:10:59.681659 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-428ee335-ec2c-4f46-bb25-81bae807f7c9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=097634d5db204edab1408e1752d662ba, user_id=d7c5d886b4a54300b818b5e7c6bdbe1d, role_id=9d9848cfd22e4b089db443c350e56670)` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 16:10:59.697038 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 856/1733] 10.222.0.15 () {60 vars in 1144 bytes} [Mon Aug 22 16:10:59 2022] POST /identity/v3/users => generated 288 bytes in 197 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 22 16:10:59.701211 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcbb6e41-b073-4ef2-90a9-74c7bb67c8ae admin admin] Authenticating user token {{(pid=81767) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 16:10:59.701748 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-428ee335-ec2c-4f46-bb25-81bae807f7c9 admin admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 16:10:59.727835 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcbb6e41-b073-4ef2-90a9-74c7bb67c8ae admin admin] Validating token access rules against request {{(pid=81767) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 16:10:59.737978 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f6ff3ca-52bf-47a4-81ea-621b1b764f7c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a98b1e4431940ab8d1fc95df2858a6c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81767) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 16:10:59.738908 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f6ff3ca-52bf-47a4-81ea-621b1b764f7c admin admin] REQUEST_METHOD: `GET` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:10:59.739500 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f6ff3ca-52bf-47a4-81ea-621b1b764f7c admin admin] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:10:59.740660 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f6ff3ca-52bf-47a4-81ea-621b1b764f7c admin admin] PATH_INFO: `/v3/roles` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:10:59.742569 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f6ff3ca-52bf-47a4-81ea-621b1b764f7c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 16:10:59.746158 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f6ff3ca-52bf-47a4-81ea-621b1b764f7c admin admin] RBAC: Authorization granted {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 16:10:59.764933 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 857/1734] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Aug 22 16:10:59 2022] GET /identity/v3/roles => generated 1420 bytes in 64 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 22 16:10:59.766381 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f6ff3ca-52bf-47a4-81ea-621b1b764f7c admin admin] Authenticating user token {{(pid=81767) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 16:10:59.780312 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f6ff3ca-52bf-47a4-81ea-621b1b764f7c admin admin] Validating token access rules against request {{(pid=81767) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 16:10:59.787597 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a234706d-ec8f-4850-a367-8551faad94e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a98b1e4431940ab8d1fc95df2858a6c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81767) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 16:10:59.788630 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a234706d-ec8f-4850-a367-8551faad94e0 admin admin] REQUEST_METHOD: `PUT` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:10:59.789446 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a234706d-ec8f-4850-a367-8551faad94e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:10:59.790091 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a234706d-ec8f-4850-a367-8551faad94e0 admin admin] PATH_INFO: `/v3/projects/c34f017c0c144e37a7b9e7cfd21860e1/users/7867ce27391c4ef2847d61fece719937/roles/9d9848cfd22e4b089db443c350e56670` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:10:59.813993 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 878/1735] 10.222.0.15 () {60 vars in 1483 bytes} [Mon Aug 22 16:10:59 2022] PUT /identity/v3/projects/097634d5db204edab1408e1752d662ba/users/d7c5d886b4a54300b818b5e7c6bdbe1d/roles/9d9848cfd22e4b089db443c350e56670 => generated 0 bytes in 273 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 16:10:59.818510 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-428ee335-ec2c-4f46-bb25-81bae807f7c9 admin admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 16:10:59.842122 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a234706d-ec8f-4850-a367-8551faad94e0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c34f017c0c144e37a7b9e7cfd21860e1, user_id=7867ce27391c4ef2847d61fece719937, role_id=9d9848cfd22e4b089db443c350e56670)` {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 16:10:59.849910 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a234706d-ec8f-4850-a367-8551faad94e0 admin admin] RBAC: Authorization granted {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 16:10:59.939045 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 858/1736] 10.222.0.15 () {60 vars in 1483 bytes} [Mon Aug 22 16:10:59 2022] PUT /identity/v3/projects/c34f017c0c144e37a7b9e7cfd21860e1/users/7867ce27391c4ef2847d61fece719937/roles/9d9848cfd22e4b089db443c350e56670 => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 16:10:59.943588 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a234706d-ec8f-4850-a367-8551faad94e0 admin admin] Authenticating user token {{(pid=81767) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 16:10:59.971446 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-428ee335-ec2c-4f46-bb25-81bae807f7c9 admin admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 16:11:00.081952 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a234706d-ec8f-4850-a367-8551faad94e0 admin admin] Validating token access rules against request {{(pid=81767) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 16:11:00.101218 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7884d386-2203-421e-b62e-ee051325a6c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a98b1e4431940ab8d1fc95df2858a6c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81767) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 16:11:00.101218 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7884d386-2203-421e-b62e-ee051325a6c4 admin admin] REQUEST_METHOD: `GET` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:11:00.101218 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7884d386-2203-421e-b62e-ee051325a6c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:11:00.101545 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7884d386-2203-421e-b62e-ee051325a6c4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:11:00.102576 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7884d386-2203-421e-b62e-ee051325a6c4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 16:11:00.104541 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7884d386-2203-421e-b62e-ee051325a6c4 admin admin] RBAC: Authorization granted {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 16:11:00.112722 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7d051cf-4ea0-4518-acfe-36426f46f47c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a98b1e4431940ab8d1fc95df2858a6c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 16:11:00.113304 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7d051cf-4ea0-4518-acfe-36426f46f47c admin admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:11:00.113591 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7d051cf-4ea0-4518-acfe-36426f46f47c admin admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:11:00.113859 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7d051cf-4ea0-4518-acfe-36426f46f47c admin admin] PATH_INFO: `/v3/roles` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:11:00.119618 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7d051cf-4ea0-4518-acfe-36426f46f47c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 16:11:00.121882 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7d051cf-4ea0-4518-acfe-36426f46f47c admin admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 16:11:00.125587 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 859/1737] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Aug 22 16:10:59 2022] GET /identity/v3/roles => generated 1420 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 22 16:11:00.129430 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7884d386-2203-421e-b62e-ee051325a6c4 admin admin] Authenticating user token {{(pid=81767) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 16:11:00.145001 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 879/1738] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Aug 22 16:10:59 2022] GET /identity/v3/roles => generated 1420 bytes in 327 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 22 16:11:00.149309 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7d051cf-4ea0-4518-acfe-36426f46f47c admin admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 16:11:00.168031 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7884d386-2203-421e-b62e-ee051325a6c4 admin admin] Validating token access rules against request {{(pid=81767) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 16:11:00.171928 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7d051cf-4ea0-4518-acfe-36426f46f47c admin admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 16:11:00.183689 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f683d783-8cd7-4439-a380-347388d935c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a98b1e4431940ab8d1fc95df2858a6c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 16:11:00.184374 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f683d783-8cd7-4439-a380-347388d935c8 admin admin] REQUEST_METHOD: `PUT` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:11:00.184710 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f683d783-8cd7-4439-a380-347388d935c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:11:00.184986 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f683d783-8cd7-4439-a380-347388d935c8 admin admin] PATH_INFO: `/v3/projects/097634d5db204edab1408e1752d662ba/users/d7c5d886b4a54300b818b5e7c6bdbe1d/roles/9d9848cfd22e4b089db443c350e56670` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:11:00.188561 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b92c93ca-fedb-4cb2-a55f-ea525f49814a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a98b1e4431940ab8d1fc95df2858a6c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81767) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 16:11:00.189257 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b92c93ca-fedb-4cb2-a55f-ea525f49814a admin admin] REQUEST_METHOD: `PUT` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:11:00.190188 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f683d783-8cd7-4439-a380-347388d935c8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=097634d5db204edab1408e1752d662ba, user_id=d7c5d886b4a54300b818b5e7c6bdbe1d, role_id=9d9848cfd22e4b089db443c350e56670)` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 16:11:00.191436 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b92c93ca-fedb-4cb2-a55f-ea525f49814a admin admin] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:11:00.192877 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b92c93ca-fedb-4cb2-a55f-ea525f49814a admin admin] PATH_INFO: `/v3/projects/c34f017c0c144e37a7b9e7cfd21860e1/users/7867ce27391c4ef2847d61fece719937/roles/9d9848cfd22e4b089db443c350e56670` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:11:00.195891 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f683d783-8cd7-4439-a380-347388d935c8 admin admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 16:11:00.200546 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b92c93ca-fedb-4cb2-a55f-ea525f49814a admin admin] RBAC: Authorizing `identity:create_grant(project_id=c34f017c0c144e37a7b9e7cfd21860e1, user_id=7867ce27391c4ef2847d61fece719937, role_id=9d9848cfd22e4b089db443c350e56670)` {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 16:11:00.207892 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b92c93ca-fedb-4cb2-a55f-ea525f49814a admin admin] RBAC: Authorization granted {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 16:11:00.223340 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 880/1739] 10.222.0.15 () {60 vars in 1483 bytes} [Mon Aug 22 16:11:00 2022] PUT /identity/v3/projects/097634d5db204edab1408e1752d662ba/users/d7c5d886b4a54300b818b5e7c6bdbe1d/roles/9d9848cfd22e4b089db443c350e56670 => generated 0 bytes in 74 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 16:11:00.234273 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-140a322d-c968-490b-8290-ddd81166688e None None] REQUEST_METHOD: `POST` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:11:00.234587 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-140a322d-c968-490b-8290-ddd81166688e None None] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:11:00.234833 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-140a322d-c968-490b-8290-ddd81166688e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:11:00.263365 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 860/1740] 10.222.0.15 () {60 vars in 1483 bytes} [Mon Aug 22 16:11:00 2022] PUT /identity/v3/projects/c34f017c0c144e37a7b9e7cfd21860e1/users/7867ce27391c4ef2847d61fece719937/roles/9d9848cfd22e4b089db443c350e56670 => generated 0 bytes in 126 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 16:11:00.265718 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f717c58-a21e-4bac-8fbf-285445492830 None None] REQUEST_METHOD: `POST` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:11:00.265957 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f717c58-a21e-4bac-8fbf-285445492830 None None] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:11:00.266522 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f717c58-a21e-4bac-8fbf-285445492830 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:11:00.294084 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.auth.core [None req-140a322d-c968-490b-8290-ddd81166688e None None] MFA Rules not processed for user `d7c5d886b4a54300b818b5e7c6bdbe1d`. Rule list: `[]` (Enabled: `True`). {{(pid=81768) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 16:11:00.305860 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-140a322d-c968-490b-8290-ddd81166688e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 16:11:00.347344 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.auth.core [None req-8f717c58-a21e-4bac-8fbf-285445492830 None None] MFA Rules not processed for user `7867ce27391c4ef2847d61fece719937`. Rule list: `[]` (Enabled: `True`). {{(pid=81767) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 16:11:00.352571 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.fernet_utils [None req-8f717c58-a21e-4bac-8fbf-285445492830 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81767) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 16:11:00.411361 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG dbcounter [-] [81767] Writing DB stats keystone:INSERT=3,keystone:SELECT=97 {{(pid=81767) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 16:11:00.523568 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 861/1741] 10.222.0.15 () {58 vars in 958 bytes} [Mon Aug 22 16:11:00 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 259 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 16:11:00.541380 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f717c58-a21e-4bac-8fbf-285445492830 None None] Authenticating user token {{(pid=81767) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 16:11:00.541760 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 881/1742] 10.222.0.15 () {58 vars in 958 bytes} [Mon Aug 22 16:11:00 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 309 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 22 16:11:00.545013 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.fernet_utils [None req-8f717c58-a21e-4bac-8fbf-285445492830 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81767) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 16:11:00.547474 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3b8231a-c863-4f2f-841d-e52a460b36f6 None None] REQUEST_METHOD: `POST` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:11:00.549296 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3b8231a-c863-4f2f-841d-e52a460b36f6 None None] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:11:00.549659 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3b8231a-c863-4f2f-841d-e52a460b36f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:11:00.565865 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application [None req-f3b8231a-c863-4f2f-841d-e52a460b36f6 None None] Expecting to find domain in project. The server could not comply with the request since it is either malformed or otherwise incorrect. The client is assumed to be in error.: keystone.exception.ValidationError: Expecting to find domain in project. The server could not comply with the request since it is either malformed or otherwise incorrect. The client is assumed to be in error. Aug 22 16:11:00.565865 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 16:11:00.565865 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 16:11:00.565865 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 16:11:00.565865 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 16:11:00.565865 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 16:11:00.565865 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 16:11:00.565865 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 16:11:00.565865 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 16:11:00.565865 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 16:11:00.565865 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 16:11:00.565865 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 16:11:00.565865 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/server/flask/common.py", line 1064, in wrapper Aug 22 16:11:00.567425 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return f(*args, **kwargs) Aug 22 16:11:00.567425 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/auth.py", line 315, in post Aug 22 16:11:00.567425 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application token = authentication.authenticate_for_token(auth_data) Aug 22 16:11:00.567425 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/_shared/authentication.py", line 182, in authenticate_for_token Aug 22 16:11:00.567425 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application auth_info = core.AuthInfo.create(auth=auth) Aug 22 16:11:00.567425 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/auth/core.py", line 126, in create Aug 22 16:11:00.567425 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application auth_info._validate_and_normalize_auth_data(scope_only) Aug 22 16:11:00.567425 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/auth/core.py", line 340, in _validate_and_normalize_auth_data Aug 22 16:11:00.567425 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application self._validate_and_normalize_scope_data() Aug 22 16:11:00.567425 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/auth/core.py", line 290, in _validate_and_normalize_scope_data Aug 22 16:11:00.567425 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application project_ref = self._lookup_project(self.auth['scope']['project']) Aug 22 16:11:00.567425 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/auth/core.py", line 194, in _lookup_project Aug 22 16:11:00.567425 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application raise exception.ValidationError(attribute='domain', Aug 22 16:11:00.567425 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application keystone.exception.ValidationError: Expecting to find domain in project. The server could not comply with the request since it is either malformed or otherwise incorrect. The client is assumed to be in error. Aug 22 16:11:00.567425 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application Aug 22 16:11:00.575852 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 882/1743] 10.222.0.15 () {58 vars in 1009 bytes} [Mon Aug 22 16:11:00 2022] POST /identity/v3/auth/tokens => generated 230 bytes in 14 msecs (HTTP/1.1 400) 5 headers in 186 bytes (2 switches on core 0) Aug 22 16:11:00.669748 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f717c58-a21e-4bac-8fbf-285445492830 None None] Validating token access rules against request {{(pid=81767) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 16:11:00.681261 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbc22239-0023-4edb-aee1-f8b168cc3820 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79706d480f864998b9f8594be407ceab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4ee0413f9ef4d4fb1f95a660babe090', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81767) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 16:11:00.681904 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbc22239-0023-4edb-aee1-f8b168cc3820 service cinder] REQUEST_METHOD: `GET` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:11:00.682186 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbc22239-0023-4edb-aee1-f8b168cc3820 service cinder] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:11:00.682472 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbc22239-0023-4edb-aee1-f8b168cc3820 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:11:00.719560 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbc22239-0023-4edb-aee1-f8b168cc3820 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 16:11:00.721067 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbc22239-0023-4edb-aee1-f8b168cc3820 service cinder] RBAC: Authorization granted {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 16:11:00.743026 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 862/1744] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Aug 22 16:11:00 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 202 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 16:11:00.836922 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3b8231a-c863-4f2f-841d-e52a460b36f6 None None] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 16:11:00.839365 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-f3b8231a-c863-4f2f-841d-e52a460b36f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 16:11:00.861548 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3b8231a-c863-4f2f-841d-e52a460b36f6 None None] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 16:11:00.888986 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01eb1823-57b6-4ff6-a8a3-c1e091fb0fe0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79706d480f864998b9f8594be407ceab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4ee0413f9ef4d4fb1f95a660babe090', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 16:11:00.890508 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01eb1823-57b6-4ff6-a8a3-c1e091fb0fe0 service cinder] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:11:00.890971 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01eb1823-57b6-4ff6-a8a3-c1e091fb0fe0 service cinder] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:11:00.891485 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01eb1823-57b6-4ff6-a8a3-c1e091fb0fe0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:11:00.908353 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01eb1823-57b6-4ff6-a8a3-c1e091fb0fe0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 16:11:00.930521 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01eb1823-57b6-4ff6-a8a3-c1e091fb0fe0 service cinder] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 16:11:00.962197 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 883/1745] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Aug 22 16:11:00 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 126 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 16:11:01.500263 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 16:11:01.525793 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbc22239-0023-4edb-aee1-f8b168cc3820 service cinder] Authenticating user token {{(pid=81767) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 16:11:01.540399 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbc22239-0023-4edb-aee1-f8b168cc3820 service cinder] Validating token access rules against request {{(pid=81767) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 16:11:01.551609 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bef8c0c8-08f9-4d55-b39b-4c5a806e6a98 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79706d480f864998b9f8594be407ceab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4ee0413f9ef4d4fb1f95a660babe090', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81767) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 16:11:01.557672 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bef8c0c8-08f9-4d55-b39b-4c5a806e6a98 service cinder] REQUEST_METHOD: `GET` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:11:01.558261 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bef8c0c8-08f9-4d55-b39b-4c5a806e6a98 service cinder] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:11:01.558713 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bef8c0c8-08f9-4d55-b39b-4c5a806e6a98 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:11:01.848475 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bef8c0c8-08f9-4d55-b39b-4c5a806e6a98 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 16:11:01.850021 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bef8c0c8-08f9-4d55-b39b-4c5a806e6a98 service cinder] RBAC: Authorization granted {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 16:11:01.883698 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG dbcounter [-] [81767] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81767) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 16:11:01.889318 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 863/1746] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Aug 22 16:11:01 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 363 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 16:11:02.782913 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a2c8f0d-7ced-44ca-a7a4-4e3f68ffd82e None None] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:11:02.783480 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a2c8f0d-7ced-44ca-a7a4-4e3f68ffd82e None None] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:11:02.783599 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a2c8f0d-7ced-44ca-a7a4-4e3f68ffd82e None None] PATH_INFO: `/` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:11:02.786011 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 884/1747] 10.222.0.15 () {54 vars in 921 bytes} [Mon Aug 22 16:11:02 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 16:11:02.837727 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1fb8162-fc02-4562-9c0f-ef1b57cd0f77 None None] REQUEST_METHOD: `POST` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:11:02.838335 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1fb8162-fc02-4562-9c0f-ef1b57cd0f77 None None] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:11:02.838335 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1fb8162-fc02-4562-9c0f-ef1b57cd0f77 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:11:02.905727 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.auth.core [None req-a1fb8162-fc02-4562-9c0f-ef1b57cd0f77 None None] MFA Rules not processed for user `08159bf8edda4246bb463f7b11b0894b`. Rule list: `[]` (Enabled: `True`). {{(pid=81767) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 16:11:02.907704 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.fernet_utils [None req-a1fb8162-fc02-4562-9c0f-ef1b57cd0f77 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81767) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 16:11:02.967128 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 16:11:03.059511 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 864/1748] 10.222.0.15 () {58 vars in 1018 bytes} [Mon Aug 22 16:11:02 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 223 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 22 16:11:03.092319 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a2c8f0d-7ced-44ca-a7a4-4e3f68ffd82e None None] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 16:11:03.094101 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-4a2c8f0d-7ced-44ca-a7a4-4e3f68ffd82e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 16:11:03.122963 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1fb8162-fc02-4562-9c0f-ef1b57cd0f77 None None] Authenticating user token {{(pid=81767) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 16:11:03.125031 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.fernet_utils [None req-a1fb8162-fc02-4562-9c0f-ef1b57cd0f77 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81767) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 16:11:03.144041 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a2c8f0d-7ced-44ca-a7a4-4e3f68ffd82e None None] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 16:11:03.155009 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a758fa58-c472-44f3-bd9c-c22d5257886a service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '08159bf8edda4246bb463f7b11b0894b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4ee0413f9ef4d4fb1f95a660babe090', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 16:11:03.158027 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a758fa58-c472-44f3-bd9c-c22d5257886a service placement] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:11:03.158027 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a758fa58-c472-44f3-bd9c-c22d5257886a service placement] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:11:03.158027 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a758fa58-c472-44f3-bd9c-c22d5257886a service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:11:03.174304 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a758fa58-c472-44f3-bd9c-c22d5257886a service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 16:11:03.176753 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a758fa58-c472-44f3-bd9c-c22d5257886a service placement] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 16:11:03.209992 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 885/1749] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Aug 22 16:11:03 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 16:11:03.211594 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a758fa58-c472-44f3-bd9c-c22d5257886a service placement] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 16:11:03.216985 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-a758fa58-c472-44f3-bd9c-c22d5257886a service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 16:11:03.276101 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1fb8162-fc02-4562-9c0f-ef1b57cd0f77 None None] Validating token access rules against request {{(pid=81767) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 16:11:03.281878 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1ffe353-933c-4f5b-8f90-fefe1ee7adab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a98b1e4431940ab8d1fc95df2858a6c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81767) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 16:11:03.283186 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1ffe353-933c-4f5b-8f90-fefe1ee7adab admin admin] REQUEST_METHOD: `DELETE` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:11:03.284205 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1ffe353-933c-4f5b-8f90-fefe1ee7adab admin admin] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:11:03.284501 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1ffe353-933c-4f5b-8f90-fefe1ee7adab admin admin] PATH_INFO: `/v3/users/4763c7fefe5a4fcdbead877717f760b8` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:11:03.286111 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1ffe353-933c-4f5b-8f90-fefe1ee7adab admin admin] RBAC: Authorizing `identity:delete_user(user_id=4763c7fefe5a4fcdbead877717f760b8)` {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 16:11:03.288183 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1ffe353-933c-4f5b-8f90-fefe1ee7adab admin admin] RBAC: Authorization granted {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 16:11:03.314200 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG dbcounter [-] [81767] Writing DB stats keystone:SELECT=100 {{(pid=81767) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 16:11:03.363872 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a758fa58-c472-44f3-bd9c-c22d5257886a service placement] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 16:11:03.374054 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8359623-dc39-4e36-b451-8d3059ae94b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a98b1e4431940ab8d1fc95df2858a6c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 16:11:03.375628 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8359623-dc39-4e36-b451-8d3059ae94b5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:11:03.375628 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8359623-dc39-4e36-b451-8d3059ae94b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:11:03.376608 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8359623-dc39-4e36-b451-8d3059ae94b5 admin admin] PATH_INFO: `/v3/users/d7d5682e9ccb437daea3a9d2526cb60e` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:11:03.378039 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8359623-dc39-4e36-b451-8d3059ae94b5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d7d5682e9ccb437daea3a9d2526cb60e)` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 16:11:03.380432 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8359623-dc39-4e36-b451-8d3059ae94b5 admin admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 16:11:03.546848 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.notifications [None req-d1ffe353-933c-4f5b-8f90-fefe1ee7adab admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4763c7fefe5a4fcdbead877717f760b8'} {{(pid=81767) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 16:11:03.547571 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.notifications [None req-d1ffe353-933c-4f5b-8f90-fefe1ee7adab admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4763c7fefe5a4fcdbead877717f760b8'} {{(pid=81767) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 16:11:03.624089 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-b8359623-dc39-4e36-b451-8d3059ae94b5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd7d5682e9ccb437daea3a9d2526cb60e'} {{(pid=81768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 16:11:03.624842 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-b8359623-dc39-4e36-b451-8d3059ae94b5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd7d5682e9ccb437daea3a9d2526cb60e'} {{(pid=81768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 16:11:03.625513 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-b8359623-dc39-4e36-b451-8d3059ae94b5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd7d5682e9ccb437daea3a9d2526cb60e'} {{(pid=81768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 16:11:03.636489 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-b8359623-dc39-4e36-b451-8d3059ae94b5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd7d5682e9ccb437daea3a9d2526cb60e'} {{(pid=81768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 16:11:03.643195 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.notifications [None req-d1ffe353-933c-4f5b-8f90-fefe1ee7adab admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4763c7fefe5a4fcdbead877717f760b8'} {{(pid=81767) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 16:11:03.654066 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.notifications [None req-d1ffe353-933c-4f5b-8f90-fefe1ee7adab admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4763c7fefe5a4fcdbead877717f760b8'} {{(pid=81767) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 16:11:03.667505 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.notifications [None req-d1ffe353-933c-4f5b-8f90-fefe1ee7adab admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4763c7fefe5a4fcdbead877717f760b8'} {{(pid=81767) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 16:11:03.680496 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 865/1750] 10.222.0.15 () {58 vars in 1224 bytes} [Mon Aug 22 16:11:03 2022] DELETE /identity/v3/users/4763c7fefe5a4fcdbead877717f760b8 => generated 0 bytes in 558 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 16:11:03.737653 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-b8359623-dc39-4e36-b451-8d3059ae94b5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd7d5682e9ccb437daea3a9d2526cb60e'} {{(pid=81768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 16:11:03.775423 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 886/1751] 10.222.0.15 () {58 vars in 1224 bytes} [Mon Aug 22 16:11:03 2022] DELETE /identity/v3/users/d7d5682e9ccb437daea3a9d2526cb60e => generated 0 bytes in 562 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 16:11:03.778002 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35bba08d-97f8-4385-a27b-cee8d1ce2d7d None None] REQUEST_METHOD: `POST` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:11:03.778295 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35bba08d-97f8-4385-a27b-cee8d1ce2d7d None None] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:11:03.778573 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35bba08d-97f8-4385-a27b-cee8d1ce2d7d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:11:03.806194 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.auth.core [None req-35bba08d-97f8-4385-a27b-cee8d1ce2d7d None None] MFA Rules not processed for user `6b2199ae0fb0480aa917ada391f5ec54`. Rule list: `[]` (Enabled: `True`). {{(pid=81767) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 16:11:03.808313 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.fernet_utils [None req-35bba08d-97f8-4385-a27b-cee8d1ce2d7d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81767) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 16:11:03.817108 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8359623-dc39-4e36-b451-8d3059ae94b5 admin admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 16:11:03.819175 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-b8359623-dc39-4e36-b451-8d3059ae94b5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 16:11:03.907179 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 866/1752] 10.222.0.15 () {58 vars in 958 bytes} [Mon Aug 22 16:11:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 16:11:03.929020 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35bba08d-97f8-4385-a27b-cee8d1ce2d7d None None] Authenticating user token {{(pid=81767) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 16:11:03.930045 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8359623-dc39-4e36-b451-8d3059ae94b5 admin admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 16:11:03.931128 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.fernet_utils [None req-35bba08d-97f8-4385-a27b-cee8d1ce2d7d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81767) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 16:11:03.937970 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e8f3a8c-ccea-45cd-b0f8-552c5a234e9b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a98b1e4431940ab8d1fc95df2858a6c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 16:11:03.939126 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e8f3a8c-ccea-45cd-b0f8-552c5a234e9b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:11:03.940187 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e8f3a8c-ccea-45cd-b0f8-552c5a234e9b admin admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:11:03.940682 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e8f3a8c-ccea-45cd-b0f8-552c5a234e9b admin admin] PATH_INFO: `/v3/users/f0b9bc811f26412b9f18ab4588b852f7` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:11:03.943092 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e8f3a8c-ccea-45cd-b0f8-552c5a234e9b admin admin] RBAC: Authorizing `identity:delete_user(user_id=f0b9bc811f26412b9f18ab4588b852f7)` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 16:11:03.946772 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e8f3a8c-ccea-45cd-b0f8-552c5a234e9b admin admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 16:11:04.066882 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG dbcounter [-] [81767] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81767) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 16:11:04.138161 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35bba08d-97f8-4385-a27b-cee8d1ce2d7d None None] Validating token access rules against request {{(pid=81767) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 16:11:04.146926 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95413c1c-d232-46b2-a3fb-94673611154b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '234687adad594f1aa45775a565ffbdc7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4ee0413f9ef4d4fb1f95a660babe090', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81767) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 16:11:04.147862 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95413c1c-d232-46b2-a3fb-94673611154b service neutron] REQUEST_METHOD: `GET` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:11:04.149653 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95413c1c-d232-46b2-a3fb-94673611154b service neutron] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:11:04.149813 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-2e8f3a8c-ccea-45cd-b0f8-552c5a234e9b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f0b9bc811f26412b9f18ab4588b852f7'} {{(pid=81768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 16:11:04.150219 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95413c1c-d232-46b2-a3fb-94673611154b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:11:04.152337 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-2e8f3a8c-ccea-45cd-b0f8-552c5a234e9b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f0b9bc811f26412b9f18ab4588b852f7'} {{(pid=81768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 16:11:04.153617 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-2e8f3a8c-ccea-45cd-b0f8-552c5a234e9b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f0b9bc811f26412b9f18ab4588b852f7'} {{(pid=81768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 16:11:04.170067 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95413c1c-d232-46b2-a3fb-94673611154b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 16:11:04.170305 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-2e8f3a8c-ccea-45cd-b0f8-552c5a234e9b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f0b9bc811f26412b9f18ab4588b852f7'} {{(pid=81768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 16:11:04.176496 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95413c1c-d232-46b2-a3fb-94673611154b service neutron] RBAC: Authorization granted {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 16:11:04.198670 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 867/1753] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Aug 22 16:11:03 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 270 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 16:11:04.200508 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95413c1c-d232-46b2-a3fb-94673611154b service neutron] Authenticating user token {{(pid=81767) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 16:11:04.204879 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.fernet_utils [None req-95413c1c-d232-46b2-a3fb-94673611154b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81767) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 16:11:04.218486 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95413c1c-d232-46b2-a3fb-94673611154b service neutron] Validating token access rules against request {{(pid=81767) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 16:11:04.228774 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02af0023-23d4-4d33-a4d8-924b2f97cbfe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a98b1e4431940ab8d1fc95df2858a6c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81767) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 16:11:04.228774 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02af0023-23d4-4d33-a4d8-924b2f97cbfe admin admin] REQUEST_METHOD: `DELETE` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:11:04.228774 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02af0023-23d4-4d33-a4d8-924b2f97cbfe admin admin] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:11:04.228774 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02af0023-23d4-4d33-a4d8-924b2f97cbfe admin admin] PATH_INFO: `/v3/users/7464b05692444d5ca04516e3aa74faec` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:11:04.230179 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02af0023-23d4-4d33-a4d8-924b2f97cbfe admin admin] RBAC: Authorizing `identity:delete_user(user_id=7464b05692444d5ca04516e3aa74faec)` {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 16:11:04.235097 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02af0023-23d4-4d33-a4d8-924b2f97cbfe admin admin] RBAC: Authorization granted {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 16:11:04.279400 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-2e8f3a8c-ccea-45cd-b0f8-552c5a234e9b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f0b9bc811f26412b9f18ab4588b852f7'} {{(pid=81768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 16:11:04.309241 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 887/1754] 10.222.0.15 () {58 vars in 1224 bytes} [Mon Aug 22 16:11:03 2022] DELETE /identity/v3/users/f0b9bc811f26412b9f18ab4588b852f7 => generated 0 bytes in 493 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 22 16:11:04.314064 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59176048-1988-4ac4-8b20-72b12bce43b0 None None] REQUEST_METHOD: `POST` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:11:04.314370 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59176048-1988-4ac4-8b20-72b12bce43b0 None None] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:11:04.314636 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59176048-1988-4ac4-8b20-72b12bce43b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:11:04.338433 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.auth.core [None req-59176048-1988-4ac4-8b20-72b12bce43b0 None None] MFA Rules not processed for user `6b2199ae0fb0480aa917ada391f5ec54`. Rule list: `[]` (Enabled: `True`). {{(pid=81768) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 16:11:04.338433 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-59176048-1988-4ac4-8b20-72b12bce43b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 16:11:04.418209 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 888/1755] 10.222.0.15 () {58 vars in 958 bytes} [Mon Aug 22 16:11:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 16:11:04.440804 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59176048-1988-4ac4-8b20-72b12bce43b0 None None] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 16:11:04.443597 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-59176048-1988-4ac4-8b20-72b12bce43b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 16:11:04.523043 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.notifications [None req-02af0023-23d4-4d33-a4d8-924b2f97cbfe admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7464b05692444d5ca04516e3aa74faec'} {{(pid=81767) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 16:11:04.524229 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.notifications [None req-02af0023-23d4-4d33-a4d8-924b2f97cbfe admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7464b05692444d5ca04516e3aa74faec'} {{(pid=81767) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 16:11:04.530388 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59176048-1988-4ac4-8b20-72b12bce43b0 None None] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 16:11:04.534155 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e570a48b-f89e-407c-9693-f6b6d0794b99 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '234687adad594f1aa45775a565ffbdc7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4ee0413f9ef4d4fb1f95a660babe090', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 16:11:04.534843 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e570a48b-f89e-407c-9693-f6b6d0794b99 service neutron] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:11:04.535079 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e570a48b-f89e-407c-9693-f6b6d0794b99 service neutron] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:11:04.535394 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e570a48b-f89e-407c-9693-f6b6d0794b99 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:11:04.537827 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-e570a48b-f89e-407c-9693-f6b6d0794b99 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 16:11:04.551173 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e570a48b-f89e-407c-9693-f6b6d0794b99 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 16:11:04.553367 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e570a48b-f89e-407c-9693-f6b6d0794b99 service neutron] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 16:11:04.564078 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 889/1756] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Aug 22 16:11:04 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 124 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 22 16:11:04.612088 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.notifications [None req-02af0023-23d4-4d33-a4d8-924b2f97cbfe admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7464b05692444d5ca04516e3aa74faec'} {{(pid=81767) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 16:11:04.622057 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e570a48b-f89e-407c-9693-f6b6d0794b99 service neutron] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 16:11:04.624417 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-e570a48b-f89e-407c-9693-f6b6d0794b99 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 16:11:04.626335 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.notifications [None req-02af0023-23d4-4d33-a4d8-924b2f97cbfe admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7464b05692444d5ca04516e3aa74faec'} {{(pid=81767) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 16:11:04.643870 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e570a48b-f89e-407c-9693-f6b6d0794b99 service neutron] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 16:11:04.656431 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85f6fb67-a66f-4be6-8814-28a59663f71f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a98b1e4431940ab8d1fc95df2858a6c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 16:11:04.656740 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85f6fb67-a66f-4be6-8814-28a59663f71f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:11:04.657060 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85f6fb67-a66f-4be6-8814-28a59663f71f admin admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:11:04.657333 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85f6fb67-a66f-4be6-8814-28a59663f71f admin admin] PATH_INFO: `/v3/projects/ceb2b51fac23410cba12504825341c28` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:11:04.658980 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85f6fb67-a66f-4be6-8814-28a59663f71f admin admin] RBAC: Authorizing `identity:delete_project(project_id=ceb2b51fac23410cba12504825341c28)` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 16:11:04.660591 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.notifications [None req-02af0023-23d4-4d33-a4d8-924b2f97cbfe admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7464b05692444d5ca04516e3aa74faec'} {{(pid=81767) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 16:11:04.664936 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85f6fb67-a66f-4be6-8814-28a59663f71f admin admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 16:11:04.674423 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 868/1757] 10.222.0.15 () {58 vars in 1224 bytes} [Mon Aug 22 16:11:04 2022] DELETE /identity/v3/users/7464b05692444d5ca04516e3aa74faec => generated 0 bytes in 475 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 16:11:04.683123 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d55171c-90db-4d04-a414-7c29e11df39d None None] REQUEST_METHOD: `POST` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:11:04.683123 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d55171c-90db-4d04-a414-7c29e11df39d None None] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:11:04.683123 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d55171c-90db-4d04-a414-7c29e11df39d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:11:04.702565 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=98,keystone:INSERT=1,keystone:DELETE=1 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 16:11:04.727395 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.auth.core [None req-7d55171c-90db-4d04-a414-7c29e11df39d None None] MFA Rules not processed for user `6b2199ae0fb0480aa917ada391f5ec54`. Rule list: `[]` (Enabled: `True`). {{(pid=81767) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 16:11:04.730654 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.fernet_utils [None req-7d55171c-90db-4d04-a414-7c29e11df39d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81767) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 16:11:04.807617 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 869/1758] 10.222.0.15 () {58 vars in 958 bytes} [Mon Aug 22 16:11:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 22 16:11:04.816269 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d55171c-90db-4d04-a414-7c29e11df39d None None] Authenticating user token {{(pid=81767) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 16:11:04.818310 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.fernet_utils [None req-7d55171c-90db-4d04-a414-7c29e11df39d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81767) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 16:11:04.906565 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-85f6fb67-a66f-4be6-8814-28a59663f71f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ceb2b51fac23410cba12504825341c28', 'request_id': 'req-85f6fb67-a66f-4be6-8814-28a59663f71f'} {{(pid=81768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 16:11:04.915803 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-85f6fb67-a66f-4be6-8814-28a59663f71f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ceb2b51fac23410cba12504825341c28', 'request_id': 'req-85f6fb67-a66f-4be6-8814-28a59663f71f'} {{(pid=81768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 16:11:04.958286 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d55171c-90db-4d04-a414-7c29e11df39d None None] Validating token access rules against request {{(pid=81767) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 16:11:04.994175 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-85f6fb67-a66f-4be6-8814-28a59663f71f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ceb2b51fac23410cba12504825341c28', 'request_id': 'req-85f6fb67-a66f-4be6-8814-28a59663f71f'} {{(pid=81768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 16:11:05.026649 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-85f6fb67-a66f-4be6-8814-28a59663f71f admin admin] The token cache is being invalidate because project ceb2b51fac23410cba12504825341c28 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81768) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 22 16:11:05.026954 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-85f6fb67-a66f-4be6-8814-28a59663f71f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 16:11:05.029078 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-85f6fb67-a66f-4be6-8814-28a59663f71f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 16:11:05.036389 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 890/1759] 10.222.0.15 () {58 vars in 1233 bytes} [Mon Aug 22 16:11:04 2022] DELETE /identity/v3/projects/ceb2b51fac23410cba12504825341c28 => generated 0 bytes in 412 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 16:11:05.044681 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d56fc119-df71-427a-9cc0-4dca8e6f6261 None None] REQUEST_METHOD: `POST` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:11:05.045027 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d56fc119-df71-427a-9cc0-4dca8e6f6261 None None] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:11:05.045379 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d56fc119-df71-427a-9cc0-4dca8e6f6261 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:11:05.114871 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.auth.core [None req-d56fc119-df71-427a-9cc0-4dca8e6f6261 None None] MFA Rules not processed for user `6b2199ae0fb0480aa917ada391f5ec54`. Rule list: `[]` (Enabled: `True`). {{(pid=81768) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 16:11:05.116191 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-d56fc119-df71-427a-9cc0-4dca8e6f6261 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 16:11:05.146566 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f20f57a-044b-4055-a171-4e2a1b17b422 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '234687adad594f1aa45775a565ffbdc7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4ee0413f9ef4d4fb1f95a660babe090', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81767) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 16:11:05.147718 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f20f57a-044b-4055-a171-4e2a1b17b422 service neutron] REQUEST_METHOD: `GET` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:11:05.148245 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f20f57a-044b-4055-a171-4e2a1b17b422 service neutron] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:11:05.148714 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f20f57a-044b-4055-a171-4e2a1b17b422 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:11:05.151133 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.fernet_utils [None req-4f20f57a-044b-4055-a171-4e2a1b17b422 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81767) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 16:11:05.227318 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 891/1760] 10.222.0.15 () {58 vars in 958 bytes} [Mon Aug 22 16:11:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 16:11:05.235687 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa8df90f-f3ef-4c45-a100-8955d9b9c1d2 None None] REQUEST_METHOD: `POST` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:11:05.236142 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa8df90f-f3ef-4c45-a100-8955d9b9c1d2 None None] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:11:05.237439 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa8df90f-f3ef-4c45-a100-8955d9b9c1d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:11:05.269650 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f20f57a-044b-4055-a171-4e2a1b17b422 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 16:11:05.271700 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f20f57a-044b-4055-a171-4e2a1b17b422 service neutron] RBAC: Authorization granted {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 16:11:05.293757 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.auth.core [None req-aa8df90f-f3ef-4c45-a100-8955d9b9c1d2 None None] MFA Rules not processed for user `6b2199ae0fb0480aa917ada391f5ec54`. Rule list: `[]` (Enabled: `True`). {{(pid=81768) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 16:11:05.294521 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 870/1761] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Aug 22 16:11:04 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 479 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 16:11:05.296580 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-aa8df90f-f3ef-4c45-a100-8955d9b9c1d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 16:11:05.313489 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 892/1762] 10.222.0.15 () {58 vars in 958 bytes} [Mon Aug 22 16:11:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 22 16:11:05.319136 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f20f57a-044b-4055-a171-4e2a1b17b422 service neutron] Authenticating user token {{(pid=81767) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 16:11:05.335490 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f20f57a-044b-4055-a171-4e2a1b17b422 service neutron] Validating token access rules against request {{(pid=81767) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 16:11:05.342186 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47bdf6d8-018f-42c3-973a-cd825b1850b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a98b1e4431940ab8d1fc95df2858a6c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81767) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 16:11:05.342896 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47bdf6d8-018f-42c3-973a-cd825b1850b4 admin admin] REQUEST_METHOD: `GET` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:11:05.343417 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47bdf6d8-018f-42c3-973a-cd825b1850b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:11:05.343711 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47bdf6d8-018f-42c3-973a-cd825b1850b4 admin admin] PATH_INFO: `/v3/domains` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:11:05.344681 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47bdf6d8-018f-42c3-973a-cd825b1850b4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81767) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 16:11:05.345171 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47bdf6d8-018f-42c3-973a-cd825b1850b4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 16:11:05.347154 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47bdf6d8-018f-42c3-973a-cd825b1850b4 admin admin] RBAC: Authorization granted {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 16:11:05.370826 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 871/1763] 10.222.0.15 () {58 vars in 1153 bytes} [Mon Aug 22 16:11:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 16:11:05.374138 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa8df90f-f3ef-4c45-a100-8955d9b9c1d2 None None] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 16:11:05.382458 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa8df90f-f3ef-4c45-a100-8955d9b9c1d2 None None] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 16:11:05.389316 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-290d78d7-0c0f-495a-a7eb-69f3d71b3b7c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a98b1e4431940ab8d1fc95df2858a6c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 16:11:05.390255 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-290d78d7-0c0f-495a-a7eb-69f3d71b3b7c admin admin] REQUEST_METHOD: `POST` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:11:05.390709 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-290d78d7-0c0f-495a-a7eb-69f3d71b3b7c admin admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:11:05.395744 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-290d78d7-0c0f-495a-a7eb-69f3d71b3b7c admin admin] PATH_INFO: `/v3/projects` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:11:05.396452 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-290d78d7-0c0f-495a-a7eb-69f3d71b3b7c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 16:11:05.399695 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-290d78d7-0c0f-495a-a7eb-69f3d71b3b7c admin admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 16:11:05.404514 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47bdf6d8-018f-42c3-973a-cd825b1850b4 admin admin] Authenticating user token {{(pid=81767) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 16:11:05.406283 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.fernet_utils [None req-47bdf6d8-018f-42c3-973a-cd825b1850b4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81767) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 16:11:05.437671 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47bdf6d8-018f-42c3-973a-cd825b1850b4 admin admin] Validating token access rules against request {{(pid=81767) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 16:11:05.461703 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bc9d72f-f55e-4d86-8bff-9ca48de1edd0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a98b1e4431940ab8d1fc95df2858a6c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81767) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 16:11:05.462247 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bc9d72f-f55e-4d86-8bff-9ca48de1edd0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:11:05.462528 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bc9d72f-f55e-4d86-8bff-9ca48de1edd0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:11:05.462803 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bc9d72f-f55e-4d86-8bff-9ca48de1edd0 admin admin] PATH_INFO: `/v3/projects/51cc967a50404cd6a847e08800d56ac0` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:11:05.464220 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bc9d72f-f55e-4d86-8bff-9ca48de1edd0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=51cc967a50404cd6a847e08800d56ac0)` {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 16:11:05.466190 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bc9d72f-f55e-4d86-8bff-9ca48de1edd0 admin admin] RBAC: Authorization granted {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 16:11:05.521652 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 893/1764] 10.222.0.15 () {60 vars in 1153 bytes} [Mon Aug 22 16:11:05 2022] POST /identity/v3/projects => generated 405 bytes in 150 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 22 16:11:05.523152 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-290d78d7-0c0f-495a-a7eb-69f3d71b3b7c admin admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 16:11:05.530899 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-290d78d7-0c0f-495a-a7eb-69f3d71b3b7c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 16:11:05.636340 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.notifications [None req-9bc9d72f-f55e-4d86-8bff-9ca48de1edd0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '51cc967a50404cd6a847e08800d56ac0', 'request_id': 'req-9bc9d72f-f55e-4d86-8bff-9ca48de1edd0'} {{(pid=81767) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 16:11:05.641731 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-290d78d7-0c0f-495a-a7eb-69f3d71b3b7c admin admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 16:11:05.645172 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.notifications [None req-9bc9d72f-f55e-4d86-8bff-9ca48de1edd0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '51cc967a50404cd6a847e08800d56ac0', 'request_id': 'req-9bc9d72f-f55e-4d86-8bff-9ca48de1edd0'} {{(pid=81767) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 16:11:05.716287 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9988da56-f33d-4743-8235-39d543619ca7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a98b1e4431940ab8d1fc95df2858a6c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 16:11:05.717176 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9988da56-f33d-4743-8235-39d543619ca7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:11:05.717176 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9988da56-f33d-4743-8235-39d543619ca7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:11:05.717382 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9988da56-f33d-4743-8235-39d543619ca7 admin admin] PATH_INFO: `/v3/projects/0d4e62dbc6314addaa25d8bab6380b7c` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:11:05.718672 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9988da56-f33d-4743-8235-39d543619ca7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0d4e62dbc6314addaa25d8bab6380b7c)` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 16:11:05.721004 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9988da56-f33d-4743-8235-39d543619ca7 admin admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 16:11:05.762409 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.notifications [None req-9bc9d72f-f55e-4d86-8bff-9ca48de1edd0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '51cc967a50404cd6a847e08800d56ac0', 'request_id': 'req-9bc9d72f-f55e-4d86-8bff-9ca48de1edd0'} {{(pid=81767) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 16:11:05.778544 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.notifications [None req-9bc9d72f-f55e-4d86-8bff-9ca48de1edd0 admin admin] The token cache is being invalidate because project 51cc967a50404cd6a847e08800d56ac0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81767) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 22 16:11:05.778836 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.notifications [None req-9bc9d72f-f55e-4d86-8bff-9ca48de1edd0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81767) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 16:11:05.779669 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.notifications [None req-9bc9d72f-f55e-4d86-8bff-9ca48de1edd0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81767) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 16:11:05.781199 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 872/1765] 10.222.0.15 () {58 vars in 1233 bytes} [Mon Aug 22 16:11:05 2022] DELETE /identity/v3/projects/51cc967a50404cd6a847e08800d56ac0 => generated 0 bytes in 380 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 16:11:05.785229 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bc9d72f-f55e-4d86-8bff-9ca48de1edd0 admin admin] Authenticating user token {{(pid=81767) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 16:11:05.787208 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.fernet_utils [None req-9bc9d72f-f55e-4d86-8bff-9ca48de1edd0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81767) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 16:11:05.812837 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bc9d72f-f55e-4d86-8bff-9ca48de1edd0 admin admin] Validating token access rules against request {{(pid=81767) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 16:11:05.818226 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a72937e-560c-46a4-8688-f9aa2c66e6d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a98b1e4431940ab8d1fc95df2858a6c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81767) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 16:11:05.818859 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a72937e-560c-46a4-8688-f9aa2c66e6d0 admin admin] REQUEST_METHOD: `POST` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:11:05.819134 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a72937e-560c-46a4-8688-f9aa2c66e6d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:11:05.819439 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a72937e-560c-46a4-8688-f9aa2c66e6d0 admin admin] PATH_INFO: `/v3/users` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:11:05.820071 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a72937e-560c-46a4-8688-f9aa2c66e6d0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 16:11:05.822226 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a72937e-560c-46a4-8688-f9aa2c66e6d0 admin admin] RBAC: Authorization granted {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 16:11:05.950754 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 873/1766] 10.222.0.15 () {60 vars in 1144 bytes} [Mon Aug 22 16:11:05 2022] POST /identity/v3/users => generated 303 bytes in 169 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 22 16:11:05.952019 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a72937e-560c-46a4-8688-f9aa2c66e6d0 admin admin] Authenticating user token {{(pid=81767) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 16:11:05.958138 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.fernet_utils [None req-1a72937e-560c-46a4-8688-f9aa2c66e6d0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81767) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 16:11:05.991383 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a72937e-560c-46a4-8688-f9aa2c66e6d0 admin admin] Validating token access rules against request {{(pid=81767) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 16:11:05.995125 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-9988da56-f33d-4743-8235-39d543619ca7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0d4e62dbc6314addaa25d8bab6380b7c', 'request_id': 'req-9988da56-f33d-4743-8235-39d543619ca7'} {{(pid=81768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 16:11:06.021885 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-9988da56-f33d-4743-8235-39d543619ca7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0d4e62dbc6314addaa25d8bab6380b7c', 'request_id': 'req-9988da56-f33d-4743-8235-39d543619ca7'} {{(pid=81768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 16:11:06.125398 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-307c01a7-d7e2-4b5c-a58d-a155f4130402 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a98b1e4431940ab8d1fc95df2858a6c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81767) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 16:11:06.126367 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-307c01a7-d7e2-4b5c-a58d-a155f4130402 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:11:06.127062 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-307c01a7-d7e2-4b5c-a58d-a155f4130402 admin admin] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:11:06.127443 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-307c01a7-d7e2-4b5c-a58d-a155f4130402 admin admin] PATH_INFO: `/v3/users/d7c5d886b4a54300b818b5e7c6bdbe1d` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:11:06.133237 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-307c01a7-d7e2-4b5c-a58d-a155f4130402 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d7c5d886b4a54300b818b5e7c6bdbe1d)` {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 16:11:06.136013 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-307c01a7-d7e2-4b5c-a58d-a155f4130402 admin admin] RBAC: Authorization granted {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 16:11:06.170385 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-9988da56-f33d-4743-8235-39d543619ca7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0d4e62dbc6314addaa25d8bab6380b7c', 'request_id': 'req-9988da56-f33d-4743-8235-39d543619ca7'} {{(pid=81768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 16:11:06.193687 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-9988da56-f33d-4743-8235-39d543619ca7 admin admin] The token cache is being invalidate because project 0d4e62dbc6314addaa25d8bab6380b7c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81768) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 22 16:11:06.193980 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-9988da56-f33d-4743-8235-39d543619ca7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 16:11:06.195635 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-9988da56-f33d-4743-8235-39d543619ca7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 16:11:06.201405 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 894/1767] 10.222.0.15 () {58 vars in 1233 bytes} [Mon Aug 22 16:11:05 2022] DELETE /identity/v3/projects/0d4e62dbc6314addaa25d8bab6380b7c => generated 0 bytes in 679 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 16:11:06.204322 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-383542c0-941e-4acd-ae85-ebe5aa69ae4d None None] REQUEST_METHOD: `POST` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:11:06.215586 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-383542c0-941e-4acd-ae85-ebe5aa69ae4d None None] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:11:06.216106 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-383542c0-941e-4acd-ae85-ebe5aa69ae4d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:11:06.252120 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.auth.core [None req-383542c0-941e-4acd-ae85-ebe5aa69ae4d None None] MFA Rules not processed for user `6ec6b84e494c481abe9c9772a10037c9`. Rule list: `[]` (Enabled: `True`). {{(pid=81768) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 16:11:06.253200 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-383542c0-941e-4acd-ae85-ebe5aa69ae4d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 16:11:06.347376 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 895/1768] 10.222.0.15 () {58 vars in 958 bytes} [Mon Aug 22 16:11:06 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 16:11:06.347376 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-383542c0-941e-4acd-ae85-ebe5aa69ae4d None None] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 16:11:06.347376 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-383542c0-941e-4acd-ae85-ebe5aa69ae4d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 16:11:06.372237 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-383542c0-941e-4acd-ae85-ebe5aa69ae4d None None] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 16:11:06.377190 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-748208ba-2b64-4a5d-aade-32585d2ad0f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a98b1e4431940ab8d1fc95df2858a6c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 16:11:06.377841 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-748208ba-2b64-4a5d-aade-32585d2ad0f4 admin admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:11:06.378121 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-748208ba-2b64-4a5d-aade-32585d2ad0f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:11:06.378389 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-748208ba-2b64-4a5d-aade-32585d2ad0f4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:11:06.379179 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-748208ba-2b64-4a5d-aade-32585d2ad0f4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 16:11:06.381389 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-748208ba-2b64-4a5d-aade-32585d2ad0f4 admin admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 16:11:06.422153 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 896/1769] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Aug 22 16:11:06 2022] GET /identity/v3/roles => generated 1420 bytes in 75 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 22 16:11:06.422153 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-748208ba-2b64-4a5d-aade-32585d2ad0f4 admin admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 16:11:06.431450 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-748208ba-2b64-4a5d-aade-32585d2ad0f4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 16:11:06.452927 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.notifications [None req-307c01a7-d7e2-4b5c-a58d-a155f4130402 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd7c5d886b4a54300b818b5e7c6bdbe1d'} {{(pid=81767) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 16:11:06.453659 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.notifications [None req-307c01a7-d7e2-4b5c-a58d-a155f4130402 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd7c5d886b4a54300b818b5e7c6bdbe1d'} {{(pid=81767) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 16:11:06.454203 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-748208ba-2b64-4a5d-aade-32585d2ad0f4 admin admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 16:11:06.462861 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70ccdf4c-b972-4dae-b592-81bdecc82311 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a98b1e4431940ab8d1fc95df2858a6c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 16:11:06.463677 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ccdf4c-b972-4dae-b592-81bdecc82311 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:11:06.463960 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ccdf4c-b972-4dae-b592-81bdecc82311 admin admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:11:06.464966 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ccdf4c-b972-4dae-b592-81bdecc82311 admin admin] PATH_INFO: `/v3/users/6c03ee9b42d74adf842178380c59f28e` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:11:06.465780 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70ccdf4c-b972-4dae-b592-81bdecc82311 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6c03ee9b42d74adf842178380c59f28e)` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 16:11:06.467717 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70ccdf4c-b972-4dae-b592-81bdecc82311 admin admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 16:11:06.600342 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.notifications [None req-307c01a7-d7e2-4b5c-a58d-a155f4130402 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd7c5d886b4a54300b818b5e7c6bdbe1d'} {{(pid=81767) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 16:11:06.616031 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.notifications [None req-307c01a7-d7e2-4b5c-a58d-a155f4130402 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd7c5d886b4a54300b818b5e7c6bdbe1d'} {{(pid=81767) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 16:11:06.630672 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.notifications [None req-307c01a7-d7e2-4b5c-a58d-a155f4130402 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd7c5d886b4a54300b818b5e7c6bdbe1d'} {{(pid=81767) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 16:11:06.639853 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 874/1770] 10.222.0.15 () {58 vars in 1224 bytes} [Mon Aug 22 16:11:05 2022] DELETE /identity/v3/users/d7c5d886b4a54300b818b5e7c6bdbe1d => generated 0 bytes in 688 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 16:11:06.658778 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-307c01a7-d7e2-4b5c-a58d-a155f4130402 admin admin] Authenticating user token {{(pid=81767) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 16:11:06.662638 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.fernet_utils [None req-307c01a7-d7e2-4b5c-a58d-a155f4130402 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81767) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 16:11:06.740234 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-70ccdf4c-b972-4dae-b592-81bdecc82311 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6c03ee9b42d74adf842178380c59f28e'} {{(pid=81768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 16:11:06.742554 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-70ccdf4c-b972-4dae-b592-81bdecc82311 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6c03ee9b42d74adf842178380c59f28e'} {{(pid=81768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 16:11:06.744653 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-70ccdf4c-b972-4dae-b592-81bdecc82311 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6c03ee9b42d74adf842178380c59f28e'} {{(pid=81768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 16:11:06.757186 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG dbcounter [-] [81767] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81767) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 16:11:06.766226 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-70ccdf4c-b972-4dae-b592-81bdecc82311 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6c03ee9b42d74adf842178380c59f28e'} {{(pid=81768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 16:11:06.810811 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-307c01a7-d7e2-4b5c-a58d-a155f4130402 admin admin] Validating token access rules against request {{(pid=81767) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 16:11:06.826228 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe1a8588-e4a4-4741-b266-3422b8542944 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a98b1e4431940ab8d1fc95df2858a6c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81767) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 16:11:06.827033 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe1a8588-e4a4-4741-b266-3422b8542944 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:11:06.827515 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe1a8588-e4a4-4741-b266-3422b8542944 admin admin] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:11:06.827619 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe1a8588-e4a4-4741-b266-3422b8542944 admin admin] PATH_INFO: `/v3/projects/cde535a938a94617955ad88a414b0373` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:11:06.828881 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe1a8588-e4a4-4741-b266-3422b8542944 admin admin] RBAC: Authorizing `identity:delete_project(project_id=cde535a938a94617955ad88a414b0373)` {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 16:11:06.830843 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe1a8588-e4a4-4741-b266-3422b8542944 admin admin] RBAC: Authorization granted {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 16:11:06.884885 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-70ccdf4c-b972-4dae-b592-81bdecc82311 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6c03ee9b42d74adf842178380c59f28e'} {{(pid=81768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 16:11:06.912788 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 897/1771] 10.222.0.15 () {58 vars in 1224 bytes} [Mon Aug 22 16:11:06 2022] DELETE /identity/v3/users/6c03ee9b42d74adf842178380c59f28e => generated 0 bytes in 493 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 16:11:06.914897 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70ccdf4c-b972-4dae-b592-81bdecc82311 admin admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 16:11:06.933594 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-70ccdf4c-b972-4dae-b592-81bdecc82311 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 16:11:07.006526 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.notifications [None req-fe1a8588-e4a4-4741-b266-3422b8542944 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cde535a938a94617955ad88a414b0373', 'request_id': 'req-fe1a8588-e4a4-4741-b266-3422b8542944'} {{(pid=81767) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 16:11:07.028710 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70ccdf4c-b972-4dae-b592-81bdecc82311 admin admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 16:11:07.038814 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.notifications [None req-fe1a8588-e4a4-4741-b266-3422b8542944 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cde535a938a94617955ad88a414b0373', 'request_id': 'req-fe1a8588-e4a4-4741-b266-3422b8542944'} {{(pid=81767) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 16:11:07.103473 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06933faf-993e-4b5a-96d7-8c206e77fe70 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a98b1e4431940ab8d1fc95df2858a6c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 16:11:07.104147 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06933faf-993e-4b5a-96d7-8c206e77fe70 admin admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:11:07.104857 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06933faf-993e-4b5a-96d7-8c206e77fe70 admin admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:11:07.104857 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06933faf-993e-4b5a-96d7-8c206e77fe70 admin admin] PATH_INFO: `/v3/roles` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:11:07.105456 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06933faf-993e-4b5a-96d7-8c206e77fe70 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 16:11:07.107126 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06933faf-993e-4b5a-96d7-8c206e77fe70 admin admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 16:11:07.118725 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 898/1772] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Aug 22 16:11:06 2022] GET /identity/v3/roles => generated 1420 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 22 16:11:07.120467 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80011691-1bc4-4674-b870-cd2e41767ee3 None None] REQUEST_METHOD: `POST` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:11:07.120599 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80011691-1bc4-4674-b870-cd2e41767ee3 None None] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:11:07.120815 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80011691-1bc4-4674-b870-cd2e41767ee3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:11:07.127225 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application [None req-80011691-1bc4-4674-b870-cd2e41767ee3 None None] Expecting to find domain in project. The server could not comply with the request since it is either malformed or otherwise incorrect. The client is assumed to be in error.: keystone.exception.ValidationError: Expecting to find domain in project. The server could not comply with the request since it is either malformed or otherwise incorrect. The client is assumed to be in error. Aug 22 16:11:07.127225 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 16:11:07.127225 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 16:11:07.127225 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 16:11:07.127225 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 16:11:07.127225 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 16:11:07.127225 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 16:11:07.127225 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 16:11:07.127225 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 16:11:07.127225 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 16:11:07.127225 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 16:11:07.127225 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 16:11:07.127225 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/server/flask/common.py", line 1064, in wrapper Aug 22 16:11:07.128152 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return f(*args, **kwargs) Aug 22 16:11:07.128152 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/auth.py", line 315, in post Aug 22 16:11:07.128152 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application token = authentication.authenticate_for_token(auth_data) Aug 22 16:11:07.128152 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/_shared/authentication.py", line 182, in authenticate_for_token Aug 22 16:11:07.128152 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application auth_info = core.AuthInfo.create(auth=auth) Aug 22 16:11:07.128152 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/auth/core.py", line 126, in create Aug 22 16:11:07.128152 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application auth_info._validate_and_normalize_auth_data(scope_only) Aug 22 16:11:07.128152 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/auth/core.py", line 340, in _validate_and_normalize_auth_data Aug 22 16:11:07.128152 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application self._validate_and_normalize_scope_data() Aug 22 16:11:07.128152 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/auth/core.py", line 290, in _validate_and_normalize_scope_data Aug 22 16:11:07.128152 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application project_ref = self._lookup_project(self.auth['scope']['project']) Aug 22 16:11:07.128152 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/auth/core.py", line 194, in _lookup_project Aug 22 16:11:07.128152 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application raise exception.ValidationError(attribute='domain', Aug 22 16:11:07.128152 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application keystone.exception.ValidationError: Expecting to find domain in project. The server could not comply with the request since it is either malformed or otherwise incorrect. The client is assumed to be in error. Aug 22 16:11:07.128152 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application Aug 22 16:11:07.128936 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 899/1773] 10.222.0.15 () {58 vars in 1009 bytes} [Mon Aug 22 16:11:07 2022] POST /identity/v3/auth/tokens => generated 230 bytes in 9 msecs (HTTP/1.1 400) 5 headers in 186 bytes (1 switches on core 0) Aug 22 16:11:07.129416 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80011691-1bc4-4674-b870-cd2e41767ee3 None None] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 16:11:07.142033 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.notifications [None req-fe1a8588-e4a4-4741-b266-3422b8542944 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cde535a938a94617955ad88a414b0373', 'request_id': 'req-fe1a8588-e4a4-4741-b266-3422b8542944'} {{(pid=81767) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 16:11:07.185435 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.notifications [None req-fe1a8588-e4a4-4741-b266-3422b8542944 admin admin] The token cache is being invalidate because project cde535a938a94617955ad88a414b0373 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81767) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 22 16:11:07.185728 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.notifications [None req-fe1a8588-e4a4-4741-b266-3422b8542944 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81767) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 16:11:07.189860 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.notifications [None req-fe1a8588-e4a4-4741-b266-3422b8542944 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81767) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 16:11:07.190479 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80011691-1bc4-4674-b870-cd2e41767ee3 None None] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 16:11:07.192498 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 875/1774] 10.222.0.15 () {58 vars in 1233 bytes} [Mon Aug 22 16:11:06 2022] DELETE /identity/v3/projects/cde535a938a94617955ad88a414b0373 => generated 0 bytes in 552 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 16:11:07.197285 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-834216d1-f737-4048-bc98-732b7652dfc0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a98b1e4431940ab8d1fc95df2858a6c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 16:11:07.198128 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a1ce33-728b-43a8-a916-934352023afa None None] REQUEST_METHOD: `POST` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:11:07.198332 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a1ce33-728b-43a8-a916-934352023afa None None] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:11:07.198694 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a1ce33-728b-43a8-a916-934352023afa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:11:07.199622 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-834216d1-f737-4048-bc98-732b7652dfc0 admin admin] REQUEST_METHOD: `PUT` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:11:07.200724 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-834216d1-f737-4048-bc98-732b7652dfc0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:11:07.201735 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-834216d1-f737-4048-bc98-732b7652dfc0 admin admin] PATH_INFO: `/v3/projects/fa3302b8c19c41f481207eab12a8ef2c/users/f4b5760081a2402fa5bdbe84433269b3/roles/e6fc1ee3fb974125ba9b4fe55b12b45e` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:11:07.214356 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=79,keystone:DELETE=19,keystone:INSERT=2 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 16:11:07.221609 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.auth.core [None req-44a1ce33-728b-43a8-a916-934352023afa None None] MFA Rules not processed for user `6b2199ae0fb0480aa917ada391f5ec54`. Rule list: `[]` (Enabled: `True`). {{(pid=81767) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 16:11:07.221609 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.fernet_utils [None req-44a1ce33-728b-43a8-a916-934352023afa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81767) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 16:11:07.224976 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-834216d1-f737-4048-bc98-732b7652dfc0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fa3302b8c19c41f481207eab12a8ef2c, user_id=f4b5760081a2402fa5bdbe84433269b3, role_id=e6fc1ee3fb974125ba9b4fe55b12b45e)` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 16:11:07.229923 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-834216d1-f737-4048-bc98-732b7652dfc0 admin admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 16:11:07.243328 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 876/1775] 10.222.0.15 () {58 vars in 958 bytes} [Mon Aug 22 16:11:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 16:11:07.252984 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d11f72-8356-4df5-947c-246061e871c3 None None] REQUEST_METHOD: `POST` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:11:07.252984 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d11f72-8356-4df5-947c-246061e871c3 None None] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:11:07.252984 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d11f72-8356-4df5-947c-246061e871c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:11:07.280411 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.auth.core [None req-d6d11f72-8356-4df5-947c-246061e871c3 None None] MFA Rules not processed for user `6b2199ae0fb0480aa917ada391f5ec54`. Rule list: `[]` (Enabled: `True`). {{(pid=81767) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 16:11:07.281774 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.fernet_utils [None req-d6d11f72-8356-4df5-947c-246061e871c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81767) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 16:11:07.290307 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 877/1776] 10.222.0.15 () {58 vars in 958 bytes} [Mon Aug 22 16:11:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 16:11:07.294507 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6d11f72-8356-4df5-947c-246061e871c3 None None] Authenticating user token {{(pid=81767) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 16:11:07.307848 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6d11f72-8356-4df5-947c-246061e871c3 None None] Validating token access rules against request {{(pid=81767) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 16:11:07.315162 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8585c5a2-5ff1-4592-b0ef-432fa83682c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a98b1e4431940ab8d1fc95df2858a6c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81767) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 16:11:07.315829 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8585c5a2-5ff1-4592-b0ef-432fa83682c3 admin admin] REQUEST_METHOD: `GET` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:11:07.316670 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8585c5a2-5ff1-4592-b0ef-432fa83682c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:11:07.316670 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8585c5a2-5ff1-4592-b0ef-432fa83682c3 admin admin] PATH_INFO: `/v3/domains` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:11:07.317247 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8585c5a2-5ff1-4592-b0ef-432fa83682c3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81767) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 16:11:07.317611 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8585c5a2-5ff1-4592-b0ef-432fa83682c3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 16:11:07.319465 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8585c5a2-5ff1-4592-b0ef-432fa83682c3 admin admin] RBAC: Authorization granted {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 16:11:07.351767 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 878/1777] 10.222.0.15 () {58 vars in 1153 bytes} [Mon Aug 22 16:11:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 16:11:07.353344 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8585c5a2-5ff1-4592-b0ef-432fa83682c3 admin admin] Authenticating user token {{(pid=81767) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 16:11:07.362129 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 900/1778] 10.222.0.15 () {60 vars in 1483 bytes} [Mon Aug 22 16:11:07 2022] PUT /identity/v3/projects/fa3302b8c19c41f481207eab12a8ef2c/users/f4b5760081a2402fa5bdbe84433269b3/roles/e6fc1ee3fb974125ba9b4fe55b12b45e => generated 0 bytes in 233 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 16:11:07.373628 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8585c5a2-5ff1-4592-b0ef-432fa83682c3 admin admin] Validating token access rules against request {{(pid=81767) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 16:11:07.382744 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df57ee06-b85d-472e-9518-36d27942b378 None None] REQUEST_METHOD: `POST` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:11:07.383086 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df57ee06-b85d-472e-9518-36d27942b378 None None] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:11:07.383396 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df57ee06-b85d-472e-9518-36d27942b378 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:11:07.431086 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.auth.core [None req-df57ee06-b85d-472e-9518-36d27942b378 None None] MFA Rules not processed for user `f4b5760081a2402fa5bdbe84433269b3`. Rule list: `[]` (Enabled: `True`). {{(pid=81768) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 16:11:07.432424 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-df57ee06-b85d-472e-9518-36d27942b378 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 16:11:07.457499 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d16339bf-c6fa-4dfd-9754-e5f7712d05c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a98b1e4431940ab8d1fc95df2858a6c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81767) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 16:11:07.458752 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d16339bf-c6fa-4dfd-9754-e5f7712d05c7 admin admin] REQUEST_METHOD: `POST` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:11:07.459290 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d16339bf-c6fa-4dfd-9754-e5f7712d05c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:11:07.459608 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d16339bf-c6fa-4dfd-9754-e5f7712d05c7 admin admin] PATH_INFO: `/v3/projects` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:11:07.460490 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d16339bf-c6fa-4dfd-9754-e5f7712d05c7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 16:11:07.463222 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d16339bf-c6fa-4dfd-9754-e5f7712d05c7 admin admin] RBAC: Authorization granted {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 16:11:07.522407 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 901/1779] 10.222.0.15 () {58 vars in 958 bytes} [Mon Aug 22 16:11:07 2022] POST /identity/v3/auth/tokens => generated 3282 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 16:11:07.523020 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df57ee06-b85d-472e-9518-36d27942b378 None None] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 16:11:07.524471 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-df57ee06-b85d-472e-9518-36d27942b378 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 16:11:07.542573 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df57ee06-b85d-472e-9518-36d27942b378 None None] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 16:11:07.548266 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1deae2b5-cfd3-4115-953e-58b78ee47ec4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a98b1e4431940ab8d1fc95df2858a6c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 16:11:07.549082 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1deae2b5-cfd3-4115-953e-58b78ee47ec4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:11:07.549463 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1deae2b5-cfd3-4115-953e-58b78ee47ec4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:11:07.549872 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1deae2b5-cfd3-4115-953e-58b78ee47ec4 admin admin] PATH_INFO: `/v3/users/a377dda3bddf4346b9b85a2d99228ae1` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:11:07.551269 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1deae2b5-cfd3-4115-953e-58b78ee47ec4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a377dda3bddf4346b9b85a2d99228ae1)` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 16:11:07.554503 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1deae2b5-cfd3-4115-953e-58b78ee47ec4 admin admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 16:11:07.573967 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 879/1780] 10.222.0.15 () {60 vars in 1153 bytes} [Mon Aug 22 16:11:07 2022] POST /identity/v3/projects => generated 377 bytes in 221 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 22 16:11:07.590139 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d16339bf-c6fa-4dfd-9754-e5f7712d05c7 admin admin] Authenticating user token {{(pid=81767) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 16:11:07.592118 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.fernet_utils [None req-d16339bf-c6fa-4dfd-9754-e5f7712d05c7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81767) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 16:11:07.680256 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d16339bf-c6fa-4dfd-9754-e5f7712d05c7 admin admin] Validating token access rules against request {{(pid=81767) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 16:11:07.685948 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73ee1d15-7938-42b6-bf05-6b2ad670ac23 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a98b1e4431940ab8d1fc95df2858a6c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81767) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 16:11:07.686702 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73ee1d15-7938-42b6-bf05-6b2ad670ac23 admin admin] REQUEST_METHOD: `POST` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:11:07.687064 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73ee1d15-7938-42b6-bf05-6b2ad670ac23 admin admin] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:11:07.687441 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73ee1d15-7938-42b6-bf05-6b2ad670ac23 admin admin] PATH_INFO: `/v3/projects` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:11:07.688141 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73ee1d15-7938-42b6-bf05-6b2ad670ac23 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 16:11:07.690165 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73ee1d15-7938-42b6-bf05-6b2ad670ac23 admin admin] RBAC: Authorization granted {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 16:11:07.885391 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 880/1781] 10.222.0.15 () {60 vars in 1153 bytes} [Mon Aug 22 16:11:07 2022] POST /identity/v3/projects => generated 405 bytes in 292 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 22 16:11:07.885391 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73ee1d15-7938-42b6-bf05-6b2ad670ac23 admin admin] Authenticating user token {{(pid=81767) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 16:11:07.885391 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.fernet_utils [None req-73ee1d15-7938-42b6-bf05-6b2ad670ac23 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81767) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 16:11:07.969703 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73ee1d15-7938-42b6-bf05-6b2ad670ac23 admin admin] Validating token access rules against request {{(pid=81767) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 16:11:07.976058 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a6c9151-15e3-4850-a1e6-bd33ceaf97d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a98b1e4431940ab8d1fc95df2858a6c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81767) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 16:11:07.976712 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a6c9151-15e3-4850-a1e6-bd33ceaf97d6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:11:07.976993 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a6c9151-15e3-4850-a1e6-bd33ceaf97d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:11:07.977269 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a6c9151-15e3-4850-a1e6-bd33ceaf97d6 admin admin] PATH_INFO: `/v3/projects/097634d5db204edab1408e1752d662ba` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:11:07.978401 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a6c9151-15e3-4850-a1e6-bd33ceaf97d6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=097634d5db204edab1408e1752d662ba)` {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 16:11:07.980404 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a6c9151-15e3-4850-a1e6-bd33ceaf97d6 admin admin] RBAC: Authorization granted {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 16:11:07.997679 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-1deae2b5-cfd3-4115-953e-58b78ee47ec4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a377dda3bddf4346b9b85a2d99228ae1'} {{(pid=81768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 16:11:07.998347 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-1deae2b5-cfd3-4115-953e-58b78ee47ec4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a377dda3bddf4346b9b85a2d99228ae1'} {{(pid=81768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 16:11:07.999002 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-1deae2b5-cfd3-4115-953e-58b78ee47ec4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a377dda3bddf4346b9b85a2d99228ae1'} {{(pid=81768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 16:11:08.006590 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-1deae2b5-cfd3-4115-953e-58b78ee47ec4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a377dda3bddf4346b9b85a2d99228ae1'} {{(pid=81768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 16:11:08.085614 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-1deae2b5-cfd3-4115-953e-58b78ee47ec4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a377dda3bddf4346b9b85a2d99228ae1'} {{(pid=81768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 16:11:08.108396 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 902/1782] 10.222.0.15 () {58 vars in 1224 bytes} [Mon Aug 22 16:11:07 2022] DELETE /identity/v3/users/a377dda3bddf4346b9b85a2d99228ae1 => generated 0 bytes in 583 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 16:11:08.108396 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1deae2b5-cfd3-4115-953e-58b78ee47ec4 admin admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 16:11:08.108396 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-1deae2b5-cfd3-4115-953e-58b78ee47ec4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 16:11:08.110451 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.notifications [None req-4a6c9151-15e3-4850-a1e6-bd33ceaf97d6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '097634d5db204edab1408e1752d662ba', 'request_id': 'req-4a6c9151-15e3-4850-a1e6-bd33ceaf97d6'} {{(pid=81767) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 16:11:08.125191 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.notifications [None req-4a6c9151-15e3-4850-a1e6-bd33ceaf97d6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '097634d5db204edab1408e1752d662ba', 'request_id': 'req-4a6c9151-15e3-4850-a1e6-bd33ceaf97d6'} {{(pid=81767) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 16:11:08.181300 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1deae2b5-cfd3-4115-953e-58b78ee47ec4 admin admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 16:11:08.185923 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c11f50e6-a768-4bed-b9ab-35261f330680 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a98b1e4431940ab8d1fc95df2858a6c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 16:11:08.186541 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c11f50e6-a768-4bed-b9ab-35261f330680 admin admin] REQUEST_METHOD: `POST` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:11:08.186817 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c11f50e6-a768-4bed-b9ab-35261f330680 admin admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:11:08.187083 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c11f50e6-a768-4bed-b9ab-35261f330680 admin admin] PATH_INFO: `/v3/users` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:11:08.187741 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c11f50e6-a768-4bed-b9ab-35261f330680 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 16:11:08.189664 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c11f50e6-a768-4bed-b9ab-35261f330680 admin admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 16:11:08.284814 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.notifications [None req-4a6c9151-15e3-4850-a1e6-bd33ceaf97d6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '097634d5db204edab1408e1752d662ba', 'request_id': 'req-4a6c9151-15e3-4850-a1e6-bd33ceaf97d6'} {{(pid=81767) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 16:11:08.299679 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.notifications [None req-4a6c9151-15e3-4850-a1e6-bd33ceaf97d6 admin admin] The token cache is being invalidate because project 097634d5db204edab1408e1752d662ba was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81767) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 22 16:11:08.299971 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.notifications [None req-4a6c9151-15e3-4850-a1e6-bd33ceaf97d6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81767) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 16:11:08.300790 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.notifications [None req-4a6c9151-15e3-4850-a1e6-bd33ceaf97d6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81767) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 16:11:08.302285 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 881/1783] 10.222.0.15 () {58 vars in 1233 bytes} [Mon Aug 22 16:11:07 2022] DELETE /identity/v3/projects/097634d5db204edab1408e1752d662ba => generated 0 bytes in 420 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 16:11:08.303644 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a6c9151-15e3-4850-a1e6-bd33ceaf97d6 admin admin] Authenticating user token {{(pid=81767) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 16:11:08.304944 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.fernet_utils [None req-4a6c9151-15e3-4850-a1e6-bd33ceaf97d6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81767) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 16:11:08.321029 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 903/1784] 10.222.0.15 () {60 vars in 1144 bytes} [Mon Aug 22 16:11:08 2022] POST /identity/v3/users => generated 289 bytes in 215 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 22 16:11:08.321907 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c11f50e6-a768-4bed-b9ab-35261f330680 admin admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 16:11:08.328787 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-c11f50e6-a768-4bed-b9ab-35261f330680 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 16:11:08.342998 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c11f50e6-a768-4bed-b9ab-35261f330680 admin admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 16:11:08.350755 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a18689bb-8ef7-4431-958b-d270bccbc779 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a98b1e4431940ab8d1fc95df2858a6c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 16:11:08.351665 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a18689bb-8ef7-4431-958b-d270bccbc779 admin admin] REQUEST_METHOD: `POST` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:11:08.352160 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a18689bb-8ef7-4431-958b-d270bccbc779 admin admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:11:08.352662 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a18689bb-8ef7-4431-958b-d270bccbc779 admin admin] PATH_INFO: `/v3/users` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:11:08.353509 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a18689bb-8ef7-4431-958b-d270bccbc779 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 16:11:08.356511 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a18689bb-8ef7-4431-958b-d270bccbc779 admin admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 16:11:08.382991 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a6c9151-15e3-4850-a1e6-bd33ceaf97d6 admin admin] Validating token access rules against request {{(pid=81767) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 16:11:08.387810 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94ee0fdf-38cb-4e28-9276-4f31b680a4cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a98b1e4431940ab8d1fc95df2858a6c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81767) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 16:11:08.388447 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94ee0fdf-38cb-4e28-9276-4f31b680a4cf admin admin] REQUEST_METHOD: `DELETE` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:11:08.388725 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94ee0fdf-38cb-4e28-9276-4f31b680a4cf admin admin] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:11:08.389027 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94ee0fdf-38cb-4e28-9276-4f31b680a4cf admin admin] PATH_INFO: `/v3/projects/83c9e9ba49104721ad30e46354b26669` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:11:08.390149 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94ee0fdf-38cb-4e28-9276-4f31b680a4cf admin admin] RBAC: Authorizing `identity:delete_project(project_id=83c9e9ba49104721ad30e46354b26669)` {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 16:11:08.392468 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94ee0fdf-38cb-4e28-9276-4f31b680a4cf admin admin] RBAC: Authorization granted {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 16:11:08.502502 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 904/1785] 10.222.0.15 () {60 vars in 1144 bytes} [Mon Aug 22 16:11:08 2022] POST /identity/v3/users => generated 309 bytes in 181 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 22 16:11:08.503798 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65f1d9ab-c51a-4b44-ab2e-99b91bec86bd None None] REQUEST_METHOD: `POST` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:11:08.504078 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65f1d9ab-c51a-4b44-ab2e-99b91bec86bd None None] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:11:08.504346 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65f1d9ab-c51a-4b44-ab2e-99b91bec86bd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:11:08.527151 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.auth.core [None req-65f1d9ab-c51a-4b44-ab2e-99b91bec86bd None None] MFA Rules not processed for user `dbb8092b0c6d410e93a3f397e282157f`. Rule list: `[]` (Enabled: `True`). {{(pid=81768) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 16:11:08.528558 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-65f1d9ab-c51a-4b44-ab2e-99b91bec86bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 16:11:08.531899 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.notifications [None req-94ee0fdf-38cb-4e28-9276-4f31b680a4cf admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '83c9e9ba49104721ad30e46354b26669', 'request_id': 'req-94ee0fdf-38cb-4e28-9276-4f31b680a4cf'} {{(pid=81767) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 16:11:08.542243 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.notifications [None req-94ee0fdf-38cb-4e28-9276-4f31b680a4cf admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '83c9e9ba49104721ad30e46354b26669', 'request_id': 'req-94ee0fdf-38cb-4e28-9276-4f31b680a4cf'} {{(pid=81767) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 16:11:08.588889 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 905/1786] 10.222.0.15 () {58 vars in 958 bytes} [Mon Aug 22 16:11:08 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 16:11:08.590168 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5797c6b-566b-4308-957e-741465389c5a None None] REQUEST_METHOD: `POST` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:11:08.590450 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5797c6b-566b-4308-957e-741465389c5a None None] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:11:08.590722 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5797c6b-566b-4308-957e-741465389c5a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:11:08.613532 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.auth.core [None req-a5797c6b-566b-4308-957e-741465389c5a None None] MFA Rules not processed for user `6b2199ae0fb0480aa917ada391f5ec54`. Rule list: `[]` (Enabled: `True`). {{(pid=81768) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 16:11:08.616352 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-a5797c6b-566b-4308-957e-741465389c5a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 16:11:08.617034 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.notifications [None req-94ee0fdf-38cb-4e28-9276-4f31b680a4cf admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '83c9e9ba49104721ad30e46354b26669', 'request_id': 'req-94ee0fdf-38cb-4e28-9276-4f31b680a4cf'} {{(pid=81767) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 16:11:08.637901 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.notifications [None req-94ee0fdf-38cb-4e28-9276-4f31b680a4cf admin admin] The token cache is being invalidate because project 83c9e9ba49104721ad30e46354b26669 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81767) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 22 16:11:08.638250 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.notifications [None req-94ee0fdf-38cb-4e28-9276-4f31b680a4cf admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81767) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 16:11:08.638954 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.notifications [None req-94ee0fdf-38cb-4e28-9276-4f31b680a4cf admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81767) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 16:11:08.640699 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 882/1787] 10.222.0.15 () {58 vars in 1233 bytes} [Mon Aug 22 16:11:08 2022] DELETE /identity/v3/projects/83c9e9ba49104721ad30e46354b26669 => generated 0 bytes in 338 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 16:11:08.647359 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94ee0fdf-38cb-4e28-9276-4f31b680a4cf admin admin] Authenticating user token {{(pid=81767) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 16:11:08.649438 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.fernet_utils [None req-94ee0fdf-38cb-4e28-9276-4f31b680a4cf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81767) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 16:11:08.701281 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 906/1788] 10.222.0.15 () {58 vars in 958 bytes} [Mon Aug 22 16:11:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 16:11:08.703544 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5797c6b-566b-4308-957e-741465389c5a None None] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 16:11:08.704878 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-a5797c6b-566b-4308-957e-741465389c5a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 16:11:08.722210 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5797c6b-566b-4308-957e-741465389c5a None None] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 16:11:08.735003 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a79607aa-29cb-44e6-95c4-c1abd70e0f25 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a98b1e4431940ab8d1fc95df2858a6c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 16:11:08.735992 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a79607aa-29cb-44e6-95c4-c1abd70e0f25 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:11:08.736380 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94ee0fdf-38cb-4e28-9276-4f31b680a4cf admin admin] Validating token access rules against request {{(pid=81767) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 16:11:08.742151 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb06d8f7-4803-4bf3-a001-98e210930eda admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a98b1e4431940ab8d1fc95df2858a6c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81767) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 16:11:08.743067 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb06d8f7-4803-4bf3-a001-98e210930eda admin admin] REQUEST_METHOD: `GET` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:11:08.743468 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb06d8f7-4803-4bf3-a001-98e210930eda admin admin] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:11:08.743807 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb06d8f7-4803-4bf3-a001-98e210930eda admin admin] PATH_INFO: `/v3/roles` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:11:08.744530 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb06d8f7-4803-4bf3-a001-98e210930eda admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 16:11:08.745538 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a79607aa-29cb-44e6-95c4-c1abd70e0f25 admin admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:11:08.746302 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a79607aa-29cb-44e6-95c4-c1abd70e0f25 admin admin] PATH_INFO: `/v3/projects/102cf78148744c2aae7ba0c1aff3b3c7` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:11:08.746431 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb06d8f7-4803-4bf3-a001-98e210930eda admin admin] RBAC: Authorization granted {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 16:11:08.747943 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a79607aa-29cb-44e6-95c4-c1abd70e0f25 admin admin] RBAC: Authorizing `identity:delete_project(project_id=102cf78148744c2aae7ba0c1aff3b3c7)` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 16:11:08.749514 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG dbcounter [-] [81767] Writing DB stats keystone:SELECT=96,keystone:DELETE=4 {{(pid=81767) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 16:11:08.750991 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a79607aa-29cb-44e6-95c4-c1abd70e0f25 admin admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 16:11:08.766031 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 883/1789] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Aug 22 16:11:08 2022] GET /identity/v3/roles => generated 1420 bytes in 125 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 22 16:11:08.767090 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb06d8f7-4803-4bf3-a001-98e210930eda admin admin] Authenticating user token {{(pid=81767) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 16:11:08.769149 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.fernet_utils [None req-cb06d8f7-4803-4bf3-a001-98e210930eda admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81767) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 16:11:08.784439 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb06d8f7-4803-4bf3-a001-98e210930eda admin admin] Validating token access rules against request {{(pid=81767) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 16:11:08.789684 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-582c45ad-c665-454b-a90f-d1275999a01f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a98b1e4431940ab8d1fc95df2858a6c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81767) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 16:11:08.790295 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-582c45ad-c665-454b-a90f-d1275999a01f admin admin] REQUEST_METHOD: `GET` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:11:08.790671 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-582c45ad-c665-454b-a90f-d1275999a01f admin admin] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:11:08.791022 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-582c45ad-c665-454b-a90f-d1275999a01f admin admin] PATH_INFO: `/v3/roles` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:11:08.791821 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-582c45ad-c665-454b-a90f-d1275999a01f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 16:11:08.793726 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-582c45ad-c665-454b-a90f-d1275999a01f admin admin] RBAC: Authorization granted {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 16:11:08.808277 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 884/1790] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Aug 22 16:11:08 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 22 16:11:08.809463 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-582c45ad-c665-454b-a90f-d1275999a01f admin admin] Authenticating user token {{(pid=81767) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 16:11:08.811173 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.fernet_utils [None req-582c45ad-c665-454b-a90f-d1275999a01f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81767) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 16:11:08.825650 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-582c45ad-c665-454b-a90f-d1275999a01f admin admin] Validating token access rules against request {{(pid=81767) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 16:11:08.830761 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09c4907a-9005-4c85-b6b4-b1cec5a21959 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a98b1e4431940ab8d1fc95df2858a6c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81767) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 16:11:08.831728 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c4907a-9005-4c85-b6b4-b1cec5a21959 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:11:08.832119 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c4907a-9005-4c85-b6b4-b1cec5a21959 admin admin] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:11:08.832498 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c4907a-9005-4c85-b6b4-b1cec5a21959 admin admin] PATH_INFO: `/v3/users/a52a3afe77094bde8222ca0101b8747e` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:11:08.834399 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09c4907a-9005-4c85-b6b4-b1cec5a21959 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a52a3afe77094bde8222ca0101b8747e)` {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 16:11:08.837015 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09c4907a-9005-4c85-b6b4-b1cec5a21959 admin admin] RBAC: Authorization granted {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 16:11:08.870715 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-a79607aa-29cb-44e6-95c4-c1abd70e0f25 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '102cf78148744c2aae7ba0c1aff3b3c7', 'request_id': 'req-a79607aa-29cb-44e6-95c4-c1abd70e0f25'} {{(pid=81768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 16:11:08.883432 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-a79607aa-29cb-44e6-95c4-c1abd70e0f25 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '102cf78148744c2aae7ba0c1aff3b3c7', 'request_id': 'req-a79607aa-29cb-44e6-95c4-c1abd70e0f25'} {{(pid=81768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 16:11:08.978010 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 16:11:08.979571 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-a79607aa-29cb-44e6-95c4-c1abd70e0f25 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '102cf78148744c2aae7ba0c1aff3b3c7', 'request_id': 'req-a79607aa-29cb-44e6-95c4-c1abd70e0f25'} {{(pid=81768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 16:11:08.993772 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-a79607aa-29cb-44e6-95c4-c1abd70e0f25 admin admin] The token cache is being invalidate because project 102cf78148744c2aae7ba0c1aff3b3c7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81768) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 22 16:11:08.994056 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-a79607aa-29cb-44e6-95c4-c1abd70e0f25 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 16:11:08.994724 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-a79607aa-29cb-44e6-95c4-c1abd70e0f25 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 16:11:08.996269 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 907/1791] 10.222.0.15 () {58 vars in 1233 bytes} [Mon Aug 22 16:11:08 2022] DELETE /identity/v3/projects/102cf78148744c2aae7ba0c1aff3b3c7 => generated 0 bytes in 294 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 16:11:08.997514 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0e4a7b2-4aa7-40cc-a274-b5e0bd00ba85 None None] REQUEST_METHOD: `POST` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:11:08.997890 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0e4a7b2-4aa7-40cc-a274-b5e0bd00ba85 None None] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:11:08.998067 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0e4a7b2-4aa7-40cc-a274-b5e0bd00ba85 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:11:09.021936 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.auth.core [None req-c0e4a7b2-4aa7-40cc-a274-b5e0bd00ba85 None None] MFA Rules not processed for user `6b2199ae0fb0480aa917ada391f5ec54`. Rule list: `[]` (Enabled: `True`). {{(pid=81768) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 16:11:09.023153 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-c0e4a7b2-4aa7-40cc-a274-b5e0bd00ba85 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 16:11:09.044744 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.notifications [None req-09c4907a-9005-4c85-b6b4-b1cec5a21959 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a52a3afe77094bde8222ca0101b8747e'} {{(pid=81767) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 16:11:09.045442 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.notifications [None req-09c4907a-9005-4c85-b6b4-b1cec5a21959 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a52a3afe77094bde8222ca0101b8747e'} {{(pid=81767) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 16:11:09.092725 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 908/1792] 10.222.0.15 () {58 vars in 958 bytes} [Mon Aug 22 16:11:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 16:11:09.096601 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0e4a7b2-4aa7-40cc-a274-b5e0bd00ba85 None None] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 16:11:09.098177 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-c0e4a7b2-4aa7-40cc-a274-b5e0bd00ba85 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 16:11:09.107837 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.notifications [None req-09c4907a-9005-4c85-b6b4-b1cec5a21959 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a52a3afe77094bde8222ca0101b8747e'} {{(pid=81767) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 16:11:09.112299 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0e4a7b2-4aa7-40cc-a274-b5e0bd00ba85 None None] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 16:11:09.116546 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.notifications [None req-09c4907a-9005-4c85-b6b4-b1cec5a21959 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a52a3afe77094bde8222ca0101b8747e'} {{(pid=81767) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 16:11:09.118990 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf8bee58-8b99-4b5f-9153-4b636ae7829a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a98b1e4431940ab8d1fc95df2858a6c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 16:11:09.119648 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf8bee58-8b99-4b5f-9153-4b636ae7829a admin admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:11:09.119929 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf8bee58-8b99-4b5f-9153-4b636ae7829a admin admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:11:09.120201 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf8bee58-8b99-4b5f-9153-4b636ae7829a admin admin] PATH_INFO: `/v3/roles` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:11:09.120851 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf8bee58-8b99-4b5f-9153-4b636ae7829a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 16:11:09.122473 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf8bee58-8b99-4b5f-9153-4b636ae7829a admin admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 16:11:09.133886 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 909/1793] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Aug 22 16:11:09 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 22 16:11:09.134557 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf8bee58-8b99-4b5f-9153-4b636ae7829a admin admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 16:11:09.135417 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.notifications [None req-09c4907a-9005-4c85-b6b4-b1cec5a21959 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a52a3afe77094bde8222ca0101b8747e'} {{(pid=81767) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 16:11:09.136671 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-cf8bee58-8b99-4b5f-9153-4b636ae7829a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 16:11:09.146364 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 885/1794] 10.222.0.15 () {58 vars in 1224 bytes} [Mon Aug 22 16:11:08 2022] DELETE /identity/v3/users/a52a3afe77094bde8222ca0101b8747e => generated 0 bytes in 335 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 16:11:09.146364 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-702a5fa5-9753-4d44-8336-c62ae6482c72 None None] REQUEST_METHOD: `POST` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:11:09.146364 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-702a5fa5-9753-4d44-8336-c62ae6482c72 None None] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:11:09.146364 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-702a5fa5-9753-4d44-8336-c62ae6482c72 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:11:09.149825 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf8bee58-8b99-4b5f-9153-4b636ae7829a admin admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 16:11:09.151773 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application [None req-702a5fa5-9753-4d44-8336-c62ae6482c72 None None] Expecting to find domain in project. The server could not comply with the request since it is either malformed or otherwise incorrect. The client is assumed to be in error.: keystone.exception.ValidationError: Expecting to find domain in project. The server could not comply with the request since it is either malformed or otherwise incorrect. The client is assumed to be in error. Aug 22 16:11:09.151773 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 16:11:09.151773 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 16:11:09.151773 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 16:11:09.151773 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 16:11:09.151773 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 16:11:09.151773 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 16:11:09.151773 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 16:11:09.151773 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 16:11:09.151773 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 16:11:09.151773 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 16:11:09.151773 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 16:11:09.151773 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/server/flask/common.py", line 1064, in wrapper Aug 22 16:11:09.152716 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return f(*args, **kwargs) Aug 22 16:11:09.152716 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/auth.py", line 315, in post Aug 22 16:11:09.152716 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application token = authentication.authenticate_for_token(auth_data) Aug 22 16:11:09.152716 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/_shared/authentication.py", line 182, in authenticate_for_token Aug 22 16:11:09.152716 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application auth_info = core.AuthInfo.create(auth=auth) Aug 22 16:11:09.152716 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/auth/core.py", line 126, in create Aug 22 16:11:09.152716 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application auth_info._validate_and_normalize_auth_data(scope_only) Aug 22 16:11:09.152716 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/auth/core.py", line 340, in _validate_and_normalize_auth_data Aug 22 16:11:09.152716 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application self._validate_and_normalize_scope_data() Aug 22 16:11:09.152716 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/auth/core.py", line 290, in _validate_and_normalize_scope_data Aug 22 16:11:09.152716 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application project_ref = self._lookup_project(self.auth['scope']['project']) Aug 22 16:11:09.152716 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/auth/core.py", line 194, in _lookup_project Aug 22 16:11:09.152716 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application raise exception.ValidationError(attribute='domain', Aug 22 16:11:09.152716 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application keystone.exception.ValidationError: Expecting to find domain in project. The server could not comply with the request since it is either malformed or otherwise incorrect. The client is assumed to be in error. Aug 22 16:11:09.152716 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application Aug 22 16:11:09.153547 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 886/1795] 10.222.0.15 () {58 vars in 1009 bytes} [Mon Aug 22 16:11:09 2022] POST /identity/v3/auth/tokens => generated 230 bytes in 9 msecs (HTTP/1.1 400) 5 headers in 186 bytes (1 switches on core 0) Aug 22 16:11:09.154117 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56088c33-fa3a-452f-a23e-d6a22897e112 None None] REQUEST_METHOD: `POST` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:11:09.154397 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56088c33-fa3a-452f-a23e-d6a22897e112 None None] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:11:09.154667 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56088c33-fa3a-452f-a23e-d6a22897e112 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:11:09.192376 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.auth.core [None req-56088c33-fa3a-452f-a23e-d6a22897e112 None None] MFA Rules not processed for user `6b2199ae0fb0480aa917ada391f5ec54`. Rule list: `[]` (Enabled: `True`). {{(pid=81767) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 16:11:09.193984 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.fernet_utils [None req-56088c33-fa3a-452f-a23e-d6a22897e112 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81767) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 16:11:09.229060 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b46f5ea-6137-4eb5-b6b5-50f4430693fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a98b1e4431940ab8d1fc95df2858a6c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 16:11:09.230219 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b46f5ea-6137-4eb5-b6b5-50f4430693fb admin admin] REQUEST_METHOD: `PUT` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:11:09.230586 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b46f5ea-6137-4eb5-b6b5-50f4430693fb admin admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:11:09.230970 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b46f5ea-6137-4eb5-b6b5-50f4430693fb admin admin] PATH_INFO: `/v3/projects/8145e6e07ef04a03b256894e82711ad4/users/6125dea54e5d4b579f16f7b66c0ed472/roles/9d9848cfd22e4b089db443c350e56670` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:11:09.247733 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b46f5ea-6137-4eb5-b6b5-50f4430693fb admin admin] RBAC: Authorizing `identity:create_grant(project_id=8145e6e07ef04a03b256894e82711ad4, user_id=6125dea54e5d4b579f16f7b66c0ed472, role_id=9d9848cfd22e4b089db443c350e56670)` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 16:11:09.250557 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b46f5ea-6137-4eb5-b6b5-50f4430693fb admin admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 16:11:09.270159 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 887/1796] 10.222.0.15 () {58 vars in 958 bytes} [Mon Aug 22 16:11:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 16:11:09.271757 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56088c33-fa3a-452f-a23e-d6a22897e112 None None] Authenticating user token {{(pid=81767) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 16:11:09.282948 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56088c33-fa3a-452f-a23e-d6a22897e112 None None] Validating token access rules against request {{(pid=81767) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 16:11:09.287617 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a5d70de-6ab7-47a1-abad-698c1dc973f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a98b1e4431940ab8d1fc95df2858a6c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81767) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 16:11:09.288476 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a5d70de-6ab7-47a1-abad-698c1dc973f6 admin admin] REQUEST_METHOD: `GET` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:11:09.288830 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a5d70de-6ab7-47a1-abad-698c1dc973f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:11:09.289229 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a5d70de-6ab7-47a1-abad-698c1dc973f6 admin admin] PATH_INFO: `/v3/domains` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:11:09.289991 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a5d70de-6ab7-47a1-abad-698c1dc973f6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81767) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 16:11:09.292001 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a5d70de-6ab7-47a1-abad-698c1dc973f6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 16:11:09.292211 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a5d70de-6ab7-47a1-abad-698c1dc973f6 admin admin] RBAC: Authorization granted {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 16:11:09.308497 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 888/1797] 10.222.0.15 () {58 vars in 1153 bytes} [Mon Aug 22 16:11:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 16:11:09.309339 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a5d70de-6ab7-47a1-abad-698c1dc973f6 admin admin] Authenticating user token {{(pid=81767) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 16:11:09.322440 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a5d70de-6ab7-47a1-abad-698c1dc973f6 admin admin] Validating token access rules against request {{(pid=81767) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 16:11:09.334992 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 910/1798] 10.222.0.15 () {60 vars in 1483 bytes} [Mon Aug 22 16:11:09 2022] PUT /identity/v3/projects/8145e6e07ef04a03b256894e82711ad4/users/6125dea54e5d4b579f16f7b66c0ed472/roles/9d9848cfd22e4b089db443c350e56670 => generated 0 bytes in 201 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 16:11:09.336340 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bc899fd-99a8-447c-a950-927147bf34f4 None None] REQUEST_METHOD: `POST` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:11:09.336625 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bc899fd-99a8-447c-a950-927147bf34f4 None None] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:11:09.336898 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bc899fd-99a8-447c-a950-927147bf34f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:11:09.379459 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.auth.core [None req-4bc899fd-99a8-447c-a950-927147bf34f4 None None] MFA Rules not processed for user `6b2199ae0fb0480aa917ada391f5ec54`. Rule list: `[]` (Enabled: `True`). {{(pid=81768) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 16:11:09.382156 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-4bc899fd-99a8-447c-a950-927147bf34f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 16:11:09.391983 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG dbcounter [-] [81767] Writing DB stats keystone:SELECT=82,keystone:DELETE=13,keystone:INSERT=5 {{(pid=81767) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 16:11:09.416106 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-456caac2-f928-409f-9ca7-0f945f10f026 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a98b1e4431940ab8d1fc95df2858a6c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81767) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 16:11:09.416938 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-456caac2-f928-409f-9ca7-0f945f10f026 admin admin] REQUEST_METHOD: `PUT` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:11:09.417321 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-456caac2-f928-409f-9ca7-0f945f10f026 admin admin] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:11:09.417696 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-456caac2-f928-409f-9ca7-0f945f10f026 admin admin] PATH_INFO: `/v3/projects/d07fcbf4f0e34ffdb541d32ee95b9a5c/users/7258e36c0b2f42198d87fff78f1978bb/roles/e6fc1ee3fb974125ba9b4fe55b12b45e` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:11:09.438447 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-456caac2-f928-409f-9ca7-0f945f10f026 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d07fcbf4f0e34ffdb541d32ee95b9a5c, user_id=7258e36c0b2f42198d87fff78f1978bb, role_id=e6fc1ee3fb974125ba9b4fe55b12b45e)` {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 16:11:09.441203 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-456caac2-f928-409f-9ca7-0f945f10f026 admin admin] RBAC: Authorization granted {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 16:11:09.468425 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 911/1799] 10.222.0.15 () {58 vars in 958 bytes} [Mon Aug 22 16:11:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 16:11:09.469066 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bc899fd-99a8-447c-a950-927147bf34f4 None None] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 16:11:09.479852 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bc899fd-99a8-447c-a950-927147bf34f4 None None] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 16:11:09.484518 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-496c9cdb-b620-4095-9d03-0667bc3694a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a98b1e4431940ab8d1fc95df2858a6c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 16:11:09.485171 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-496c9cdb-b620-4095-9d03-0667bc3694a7 admin admin] REQUEST_METHOD: `POST` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:11:09.490212 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-496c9cdb-b620-4095-9d03-0667bc3694a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:11:09.490212 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-496c9cdb-b620-4095-9d03-0667bc3694a7 admin admin] PATH_INFO: `/v3/projects` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:11:09.490212 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-496c9cdb-b620-4095-9d03-0667bc3694a7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 16:11:09.490212 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-496c9cdb-b620-4095-9d03-0667bc3694a7 admin admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 16:11:09.546285 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 889/1800] 10.222.0.15 () {60 vars in 1483 bytes} [Mon Aug 22 16:11:09 2022] PUT /identity/v3/projects/d07fcbf4f0e34ffdb541d32ee95b9a5c/users/7258e36c0b2f42198d87fff78f1978bb/roles/e6fc1ee3fb974125ba9b4fe55b12b45e => generated 0 bytes in 236 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 16:11:09.547162 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-456caac2-f928-409f-9ca7-0f945f10f026 admin admin] Authenticating user token {{(pid=81767) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 16:11:09.580818 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 912/1801] 10.222.0.15 () {60 vars in 1153 bytes} [Mon Aug 22 16:11:09 2022] POST /identity/v3/projects => generated 365 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 22 16:11:09.581782 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-496c9cdb-b620-4095-9d03-0667bc3694a7 admin admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 16:11:09.677516 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-456caac2-f928-409f-9ca7-0f945f10f026 admin admin] Validating token access rules against request {{(pid=81767) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 16:11:09.719151 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-496c9cdb-b620-4095-9d03-0667bc3694a7 admin admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 16:11:09.727192 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd1f1afb-5684-4510-a92b-58d8ff2446b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a98b1e4431940ab8d1fc95df2858a6c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 16:11:09.727959 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd1f1afb-5684-4510-a92b-58d8ff2446b9 admin admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:11:09.728302 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd1f1afb-5684-4510-a92b-58d8ff2446b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:11:09.728830 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd1f1afb-5684-4510-a92b-58d8ff2446b9 admin admin] PATH_INFO: `/v3/domains` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:11:09.729647 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd1f1afb-5684-4510-a92b-58d8ff2446b9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81768) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 16:11:09.730018 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd1f1afb-5684-4510-a92b-58d8ff2446b9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 16:11:09.736092 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd1f1afb-5684-4510-a92b-58d8ff2446b9 admin admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 16:11:09.757933 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 913/1802] 10.222.0.15 () {58 vars in 1153 bytes} [Mon Aug 22 16:11:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 16:11:09.759752 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c1e4b4a-5e5f-4206-90ff-b06fa9d397e1 None None] REQUEST_METHOD: `POST` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:11:09.760258 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c1e4b4a-5e5f-4206-90ff-b06fa9d397e1 None None] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:11:09.760600 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c1e4b4a-5e5f-4206-90ff-b06fa9d397e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:11:09.784067 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e677b82f-88d3-4e3a-9dc1-7345f6ca3674 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a98b1e4431940ab8d1fc95df2858a6c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81767) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 16:11:09.784699 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e677b82f-88d3-4e3a-9dc1-7345f6ca3674 admin admin] REQUEST_METHOD: `GET` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:11:09.784984 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e677b82f-88d3-4e3a-9dc1-7345f6ca3674 admin admin] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:11:09.785263 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e677b82f-88d3-4e3a-9dc1-7345f6ca3674 admin admin] PATH_INFO: `/v3/roles` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:11:09.786922 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e677b82f-88d3-4e3a-9dc1-7345f6ca3674 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 16:11:09.788879 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e677b82f-88d3-4e3a-9dc1-7345f6ca3674 admin admin] RBAC: Authorization granted {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 16:11:09.790431 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=93,keystone:INSERT=1,keystone:DELETE=6 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 16:11:09.809159 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 890/1803] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Aug 22 16:11:09 2022] GET /identity/v3/roles => generated 1420 bytes in 263 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 22 16:11:09.814644 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e677b82f-88d3-4e3a-9dc1-7345f6ca3674 admin admin] Authenticating user token {{(pid=81767) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 16:11:09.826749 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.auth.core [None req-5c1e4b4a-5e5f-4206-90ff-b06fa9d397e1 None None] MFA Rules not processed for user `7258e36c0b2f42198d87fff78f1978bb`. Rule list: `[]` (Enabled: `True`). {{(pid=81768) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 16:11:09.828939 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-5c1e4b4a-5e5f-4206-90ff-b06fa9d397e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 16:11:09.836155 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e677b82f-88d3-4e3a-9dc1-7345f6ca3674 admin admin] Validating token access rules against request {{(pid=81767) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 16:11:09.848320 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cf8164f-bc2b-4615-b081-7f1b9650a703 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a98b1e4431940ab8d1fc95df2858a6c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81767) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 16:11:09.849810 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cf8164f-bc2b-4615-b081-7f1b9650a703 admin admin] REQUEST_METHOD: `POST` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:11:09.850101 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cf8164f-bc2b-4615-b081-7f1b9650a703 admin admin] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:11:09.850370 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cf8164f-bc2b-4615-b081-7f1b9650a703 admin admin] PATH_INFO: `/v3/users` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:11:09.851162 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cf8164f-bc2b-4615-b081-7f1b9650a703 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 16:11:09.853950 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cf8164f-bc2b-4615-b081-7f1b9650a703 admin admin] RBAC: Authorization granted {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 16:11:09.920988 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 914/1804] 10.222.0.15 () {58 vars in 958 bytes} [Mon Aug 22 16:11:09 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 16:11:09.920988 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c1e4b4a-5e5f-4206-90ff-b06fa9d397e1 None None] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 16:11:09.937777 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c1e4b4a-5e5f-4206-90ff-b06fa9d397e1 None None] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 16:11:09.944193 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-087caa30-2687-4c1c-9db5-b659083384d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a98b1e4431940ab8d1fc95df2858a6c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 16:11:09.945496 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-087caa30-2687-4c1c-9db5-b659083384d1 admin admin] REQUEST_METHOD: `POST` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:11:09.945496 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-087caa30-2687-4c1c-9db5-b659083384d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:11:09.945733 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-087caa30-2687-4c1c-9db5-b659083384d1 admin admin] PATH_INFO: `/v3/projects` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:11:09.946678 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-087caa30-2687-4c1c-9db5-b659083384d1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 16:11:09.948650 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-087caa30-2687-4c1c-9db5-b659083384d1 admin admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 16:11:09.992701 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 891/1805] 10.222.0.15 () {60 vars in 1144 bytes} [Mon Aug 22 16:11:09 2022] POST /identity/v3/users => generated 283 bytes in 179 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 22 16:11:09.993959 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cf8164f-bc2b-4615-b081-7f1b9650a703 admin admin] Authenticating user token {{(pid=81767) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 16:11:10.011429 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cf8164f-bc2b-4615-b081-7f1b9650a703 admin admin] Validating token access rules against request {{(pid=81767) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 16:11:10.018661 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81b75c42-9449-4668-863e-b8123bd277e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a98b1e4431940ab8d1fc95df2858a6c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81767) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 16:11:10.020096 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81b75c42-9449-4668-863e-b8123bd277e4 admin admin] REQUEST_METHOD: `PUT` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:11:10.021036 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81b75c42-9449-4668-863e-b8123bd277e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:11:10.021457 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81b75c42-9449-4668-863e-b8123bd277e4 admin admin] PATH_INFO: `/v3/projects/8145e6e07ef04a03b256894e82711ad4/users/6125dea54e5d4b579f16f7b66c0ed472/roles/9d9848cfd22e4b089db443c350e56670` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:11:10.024249 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81b75c42-9449-4668-863e-b8123bd277e4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8145e6e07ef04a03b256894e82711ad4, user_id=6125dea54e5d4b579f16f7b66c0ed472, role_id=9d9848cfd22e4b089db443c350e56670)` {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 16:11:10.032930 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81b75c42-9449-4668-863e-b8123bd277e4 admin admin] RBAC: Authorization granted {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 16:11:10.043153 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 915/1806] 10.222.0.15 () {60 vars in 1153 bytes} [Mon Aug 22 16:11:09 2022] POST /identity/v3/projects => generated 385 bytes in 125 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 22 16:11:10.043924 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-087caa30-2687-4c1c-9db5-b659083384d1 admin admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 16:11:10.070564 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 892/1807] 10.222.0.15 () {60 vars in 1483 bytes} [Mon Aug 22 16:11:09 2022] PUT /identity/v3/projects/8145e6e07ef04a03b256894e82711ad4/users/6125dea54e5d4b579f16f7b66c0ed472/roles/9d9848cfd22e4b089db443c350e56670 => generated 0 bytes in 77 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 16:11:10.071594 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81b75c42-9449-4668-863e-b8123bd277e4 admin admin] Authenticating user token {{(pid=81767) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 16:11:10.073523 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.fernet_utils [None req-81b75c42-9449-4668-863e-b8123bd277e4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81767) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 16:11:10.124610 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-087caa30-2687-4c1c-9db5-b659083384d1 admin admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 16:11:10.130795 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5ff5b86-6d9a-483b-b24f-4d05eabbb86f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a98b1e4431940ab8d1fc95df2858a6c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 16:11:10.131464 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5ff5b86-6d9a-483b-b24f-4d05eabbb86f admin admin] REQUEST_METHOD: `POST` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:11:10.131730 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5ff5b86-6d9a-483b-b24f-4d05eabbb86f admin admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:11:10.131997 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5ff5b86-6d9a-483b-b24f-4d05eabbb86f admin admin] PATH_INFO: `/v3/projects` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:11:10.132625 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5ff5b86-6d9a-483b-b24f-4d05eabbb86f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 16:11:10.134634 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5ff5b86-6d9a-483b-b24f-4d05eabbb86f admin admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 16:11:10.186294 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG dbcounter [-] [81767] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81767) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 16:11:10.189626 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81b75c42-9449-4668-863e-b8123bd277e4 admin admin] Validating token access rules against request {{(pid=81767) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 16:11:10.196574 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61c46b31-af49-4271-923e-c5632474a38f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a98b1e4431940ab8d1fc95df2858a6c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81767) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 16:11:10.197448 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61c46b31-af49-4271-923e-c5632474a38f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:11:10.197558 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61c46b31-af49-4271-923e-c5632474a38f admin admin] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:11:10.197815 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61c46b31-af49-4271-923e-c5632474a38f admin admin] PATH_INFO: `/v3/projects/38c257ac94544f0bace83551b519d5bc` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:11:10.199212 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61c46b31-af49-4271-923e-c5632474a38f admin admin] RBAC: Authorizing `identity:delete_project(project_id=38c257ac94544f0bace83551b519d5bc)` {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 16:11:10.201261 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61c46b31-af49-4271-923e-c5632474a38f admin admin] RBAC: Authorization granted {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 16:11:10.235317 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 916/1808] 10.222.0.15 () {60 vars in 1153 bytes} [Mon Aug 22 16:11:10 2022] POST /identity/v3/projects => generated 377 bytes in 191 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 22 16:11:10.235317 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5ff5b86-6d9a-483b-b24f-4d05eabbb86f admin admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 16:11:10.301130 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5ff5b86-6d9a-483b-b24f-4d05eabbb86f admin admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 16:11:10.306347 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13354947-38b3-43c1-96be-318e35998d87 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a98b1e4431940ab8d1fc95df2858a6c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 16:11:10.306954 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13354947-38b3-43c1-96be-318e35998d87 admin admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:11:10.307320 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13354947-38b3-43c1-96be-318e35998d87 admin admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:11:10.307595 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13354947-38b3-43c1-96be-318e35998d87 admin admin] PATH_INFO: `/v3/roles` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:11:10.308261 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13354947-38b3-43c1-96be-318e35998d87 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 16:11:10.309856 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13354947-38b3-43c1-96be-318e35998d87 admin admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 16:11:10.338368 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 917/1809] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Aug 22 16:11:10 2022] GET /identity/v3/roles => generated 1420 bytes in 98 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 22 16:11:10.338368 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13354947-38b3-43c1-96be-318e35998d87 admin admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 16:11:10.345837 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13354947-38b3-43c1-96be-318e35998d87 admin admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 16:11:10.361278 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.notifications [None req-61c46b31-af49-4271-923e-c5632474a38f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '38c257ac94544f0bace83551b519d5bc', 'request_id': 'req-61c46b31-af49-4271-923e-c5632474a38f'} {{(pid=81767) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 16:11:10.379337 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.notifications [None req-61c46b31-af49-4271-923e-c5632474a38f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '38c257ac94544f0bace83551b519d5bc', 'request_id': 'req-61c46b31-af49-4271-923e-c5632474a38f'} {{(pid=81767) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 16:11:10.438903 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc68e9f7-dd34-4e25-9a47-a3426d978843 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a98b1e4431940ab8d1fc95df2858a6c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 16:11:10.439659 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc68e9f7-dd34-4e25-9a47-a3426d978843 admin admin] REQUEST_METHOD: `POST` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:11:10.440012 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc68e9f7-dd34-4e25-9a47-a3426d978843 admin admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:11:10.440351 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc68e9f7-dd34-4e25-9a47-a3426d978843 admin admin] PATH_INFO: `/v3/users` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:11:10.441024 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc68e9f7-dd34-4e25-9a47-a3426d978843 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 16:11:10.443049 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc68e9f7-dd34-4e25-9a47-a3426d978843 admin admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 16:11:10.456213 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.notifications [None req-61c46b31-af49-4271-923e-c5632474a38f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '38c257ac94544f0bace83551b519d5bc', 'request_id': 'req-61c46b31-af49-4271-923e-c5632474a38f'} {{(pid=81767) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 16:11:10.470411 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.notifications [None req-61c46b31-af49-4271-923e-c5632474a38f admin admin] The token cache is being invalidate because project 38c257ac94544f0bace83551b519d5bc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81767) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 22 16:11:10.470785 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.notifications [None req-61c46b31-af49-4271-923e-c5632474a38f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81767) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 16:11:10.471708 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.notifications [None req-61c46b31-af49-4271-923e-c5632474a38f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81767) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 16:11:10.474947 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 893/1810] 10.222.0.15 () {58 vars in 1233 bytes} [Mon Aug 22 16:11:10 2022] DELETE /identity/v3/projects/38c257ac94544f0bace83551b519d5bc => generated 0 bytes in 404 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 16:11:10.490713 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fb6cb6e-768f-45bd-9c90-7943cc1ae479 None None] REQUEST_METHOD: `POST` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:11:10.491253 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fb6cb6e-768f-45bd-9c90-7943cc1ae479 None None] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:11:10.492324 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fb6cb6e-768f-45bd-9c90-7943cc1ae479 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:11:10.520644 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 918/1811] 10.222.0.15 () {60 vars in 1144 bytes} [Mon Aug 22 16:11:10 2022] POST /identity/v3/users => generated 293 bytes in 186 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 22 16:11:10.521062 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc68e9f7-dd34-4e25-9a47-a3426d978843 admin admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 16:11:10.525926 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-dc68e9f7-dd34-4e25-9a47-a3426d978843 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 16:11:10.531570 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.auth.core [None req-3fb6cb6e-768f-45bd-9c90-7943cc1ae479 None None] MFA Rules not processed for user `6125dea54e5d4b579f16f7b66c0ed472`. Rule list: `[]` (Enabled: `True`). {{(pid=81767) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 16:11:10.533542 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.fernet_utils [None req-3fb6cb6e-768f-45bd-9c90-7943cc1ae479 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81767) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 16:11:10.546882 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc68e9f7-dd34-4e25-9a47-a3426d978843 admin admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 16:11:10.560955 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fa9696e-b6ab-416d-a5ae-66296ed86426 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a98b1e4431940ab8d1fc95df2858a6c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 16:11:10.561584 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fa9696e-b6ab-416d-a5ae-66296ed86426 admin admin] REQUEST_METHOD: `POST` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:11:10.561866 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fa9696e-b6ab-416d-a5ae-66296ed86426 admin admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:11:10.562139 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fa9696e-b6ab-416d-a5ae-66296ed86426 admin admin] PATH_INFO: `/v3/users` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:11:10.566248 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fa9696e-b6ab-416d-a5ae-66296ed86426 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 16:11:10.570813 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fa9696e-b6ab-416d-a5ae-66296ed86426 admin admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 16:11:10.637622 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 894/1812] 10.222.0.15 () {58 vars in 958 bytes} [Mon Aug 22 16:11:10 2022] POST /identity/v3/auth/tokens => generated 3349 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 16:11:10.637622 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fb6cb6e-768f-45bd-9c90-7943cc1ae479 None None] Authenticating user token {{(pid=81767) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 16:11:10.638236 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.fernet_utils [None req-3fb6cb6e-768f-45bd-9c90-7943cc1ae479 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81767) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 16:11:10.656508 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fb6cb6e-768f-45bd-9c90-7943cc1ae479 None None] Validating token access rules against request {{(pid=81767) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 16:11:10.664193 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e768dc1f-e0c1-4efb-9569-6200e86a61b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a98b1e4431940ab8d1fc95df2858a6c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81767) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 16:11:10.664876 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e768dc1f-e0c1-4efb-9569-6200e86a61b0 admin admin] REQUEST_METHOD: `GET` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:11:10.665158 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e768dc1f-e0c1-4efb-9569-6200e86a61b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:11:10.666346 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e768dc1f-e0c1-4efb-9569-6200e86a61b0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:11:10.667057 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e768dc1f-e0c1-4efb-9569-6200e86a61b0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 16:11:10.669888 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e768dc1f-e0c1-4efb-9569-6200e86a61b0 admin admin] RBAC: Authorization granted {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 16:11:10.688824 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 895/1813] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Aug 22 16:11:10 2022] GET /identity/v3/roles => generated 1420 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 22 16:11:10.688824 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a75223f7-78bb-46f6-bf70-a336ac560123 None None] REQUEST_METHOD: `POST` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:11:10.688824 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a75223f7-78bb-46f6-bf70-a336ac560123 None None] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:11:10.688824 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a75223f7-78bb-46f6-bf70-a336ac560123 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:11:10.690743 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:INSERT=2,keystone:DELETE=11,keystone:SELECT=87 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 16:11:10.695115 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 919/1814] 10.222.0.15 () {60 vars in 1144 bytes} [Mon Aug 22 16:11:10 2022] POST /identity/v3/users => generated 295 bytes in 174 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 22 16:11:10.695871 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fa9696e-b6ab-416d-a5ae-66296ed86426 admin admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 16:11:10.698817 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-5fa9696e-b6ab-416d-a5ae-66296ed86426 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 16:11:10.729886 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fa9696e-b6ab-416d-a5ae-66296ed86426 admin admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 16:11:10.730850 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.auth.core [None req-a75223f7-78bb-46f6-bf70-a336ac560123 None None] MFA Rules not processed for user `6b2199ae0fb0480aa917ada391f5ec54`. Rule list: `[]` (Enabled: `True`). {{(pid=81767) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 16:11:10.732124 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.fernet_utils [None req-a75223f7-78bb-46f6-bf70-a336ac560123 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81767) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 16:11:10.744743 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88383b97-fa6a-43b3-a295-8afad49d12ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a98b1e4431940ab8d1fc95df2858a6c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 16:11:10.745929 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88383b97-fa6a-43b3-a295-8afad49d12ec admin admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:11:10.748296 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88383b97-fa6a-43b3-a295-8afad49d12ec admin admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:11:10.748296 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88383b97-fa6a-43b3-a295-8afad49d12ec admin admin] PATH_INFO: `/v3/roles` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:11:10.748296 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88383b97-fa6a-43b3-a295-8afad49d12ec admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 16:11:10.750598 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88383b97-fa6a-43b3-a295-8afad49d12ec admin admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 16:11:10.752286 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 896/1815] 10.222.0.15 () {58 vars in 958 bytes} [Mon Aug 22 16:11:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 66 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 16:11:10.753365 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a75223f7-78bb-46f6-bf70-a336ac560123 None None] Authenticating user token {{(pid=81767) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 16:11:10.756135 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.fernet_utils [None req-a75223f7-78bb-46f6-bf70-a336ac560123 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81767) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 16:11:10.777379 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 920/1816] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Aug 22 16:11:10 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 22 16:11:10.779808 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88383b97-fa6a-43b3-a295-8afad49d12ec admin admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 16:11:10.793470 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88383b97-fa6a-43b3-a295-8afad49d12ec admin admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 16:11:10.800144 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2732f99-7d9b-410f-b155-ea3c1a1e5b56 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a98b1e4431940ab8d1fc95df2858a6c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 16:11:10.801448 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2732f99-7d9b-410f-b155-ea3c1a1e5b56 admin admin] REQUEST_METHOD: `PUT` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:11:10.802269 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2732f99-7d9b-410f-b155-ea3c1a1e5b56 admin admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:11:10.802546 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2732f99-7d9b-410f-b155-ea3c1a1e5b56 admin admin] PATH_INFO: `/v3/projects/44f75b12ecb24afe8ede2f93c9642ce3/users/faba300378174db08541de5f62b55ebe/roles/e6fc1ee3fb974125ba9b4fe55b12b45e` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:11:10.825888 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2732f99-7d9b-410f-b155-ea3c1a1e5b56 admin admin] RBAC: Authorizing `identity:create_grant(project_id=44f75b12ecb24afe8ede2f93c9642ce3, user_id=faba300378174db08541de5f62b55ebe, role_id=e6fc1ee3fb974125ba9b4fe55b12b45e)` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 16:11:10.828849 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2732f99-7d9b-410f-b155-ea3c1a1e5b56 admin admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 16:11:10.848721 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a75223f7-78bb-46f6-bf70-a336ac560123 None None] Validating token access rules against request {{(pid=81767) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 16:11:10.853069 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce4b3ade-ac46-45c1-ae00-8ec425368d8a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79706d480f864998b9f8594be407ceab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4ee0413f9ef4d4fb1f95a660babe090', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81767) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 16:11:10.853730 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce4b3ade-ac46-45c1-ae00-8ec425368d8a service cinder] REQUEST_METHOD: `GET` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:11:10.854068 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce4b3ade-ac46-45c1-ae00-8ec425368d8a service cinder] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:11:10.854347 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce4b3ade-ac46-45c1-ae00-8ec425368d8a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:11:10.865121 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce4b3ade-ac46-45c1-ae00-8ec425368d8a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 16:11:10.866560 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce4b3ade-ac46-45c1-ae00-8ec425368d8a service cinder] RBAC: Authorization granted {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 16:11:10.878128 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 897/1817] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Aug 22 16:11:10 2022] GET /identity/v3/auth/tokens => generated 3349 bytes in 125 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 16:11:10.879249 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce4b3ade-ac46-45c1-ae00-8ec425368d8a service cinder] Authenticating user token {{(pid=81767) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 16:11:10.895773 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce4b3ade-ac46-45c1-ae00-8ec425368d8a service cinder] Validating token access rules against request {{(pid=81767) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 16:11:10.904374 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b887f637-c93e-455f-af9d-77bca019d23a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a98b1e4431940ab8d1fc95df2858a6c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81767) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 16:11:10.905054 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b887f637-c93e-455f-af9d-77bca019d23a admin admin] REQUEST_METHOD: `GET` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:11:10.905333 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b887f637-c93e-455f-af9d-77bca019d23a admin admin] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:11:10.905631 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b887f637-c93e-455f-af9d-77bca019d23a admin admin] PATH_INFO: `/v3/roles` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:11:10.906332 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b887f637-c93e-455f-af9d-77bca019d23a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 16:11:10.907877 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 921/1818] 10.222.0.15 () {60 vars in 1483 bytes} [Mon Aug 22 16:11:10 2022] PUT /identity/v3/projects/44f75b12ecb24afe8ede2f93c9642ce3/users/faba300378174db08541de5f62b55ebe/roles/e6fc1ee3fb974125ba9b4fe55b12b45e => generated 0 bytes in 130 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 16:11:10.917568 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b76191c0-0cc1-421f-8513-0197ea8bd22f None None] REQUEST_METHOD: `POST` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:11:10.917568 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b76191c0-0cc1-421f-8513-0197ea8bd22f None None] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:11:10.917568 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b76191c0-0cc1-421f-8513-0197ea8bd22f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:11:10.917870 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b887f637-c93e-455f-af9d-77bca019d23a admin admin] RBAC: Authorization granted {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 16:11:10.936487 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 898/1819] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Aug 22 16:11:10 2022] GET /identity/v3/roles => generated 1420 bytes in 58 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 22 16:11:10.937637 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b887f637-c93e-455f-af9d-77bca019d23a admin admin] Authenticating user token {{(pid=81767) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 16:11:10.963356 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b887f637-c93e-455f-af9d-77bca019d23a admin admin] Validating token access rules against request {{(pid=81767) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 16:11:10.977632 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.auth.core [None req-b76191c0-0cc1-421f-8513-0197ea8bd22f None None] MFA Rules not processed for user `6b2199ae0fb0480aa917ada391f5ec54`. Rule list: `[]` (Enabled: `True`). {{(pid=81768) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 16:11:10.981007 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-b76191c0-0cc1-421f-8513-0197ea8bd22f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 16:11:11.033490 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG dbcounter [-] [81767] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=81767) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 16:11:11.126700 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 922/1820] 10.222.0.15 () {58 vars in 958 bytes} [Mon Aug 22 16:11:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 218 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 16:11:11.130517 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b428744-2c2e-478c-bbd2-1f4e30b17d64 None None] REQUEST_METHOD: `POST` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:11:11.130970 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b428744-2c2e-478c-bbd2-1f4e30b17d64 None None] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:11:11.131461 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b428744-2c2e-478c-bbd2-1f4e30b17d64 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:11:11.157729 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e164bb7b-698a-418e-8802-45cdf8ee06ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a98b1e4431940ab8d1fc95df2858a6c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81767) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 16:11:11.158415 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e164bb7b-698a-418e-8802-45cdf8ee06ab admin admin] REQUEST_METHOD: `GET` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:11:11.158613 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e164bb7b-698a-418e-8802-45cdf8ee06ab admin admin] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:11:11.158881 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e164bb7b-698a-418e-8802-45cdf8ee06ab admin admin] PATH_INFO: `/v3/roles` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:11:11.159553 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e164bb7b-698a-418e-8802-45cdf8ee06ab admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 16:11:11.161934 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e164bb7b-698a-418e-8802-45cdf8ee06ab admin admin] RBAC: Authorization granted {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 16:11:11.186507 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.auth.core [None req-8b428744-2c2e-478c-bbd2-1f4e30b17d64 None None] MFA Rules not processed for user `faba300378174db08541de5f62b55ebe`. Rule list: `[]` (Enabled: `True`). {{(pid=81768) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 16:11:11.192196 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 899/1821] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Aug 22 16:11:10 2022] GET /identity/v3/roles => generated 1420 bytes in 252 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 22 16:11:11.192196 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e164bb7b-698a-418e-8802-45cdf8ee06ab admin admin] Authenticating user token {{(pid=81767) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 16:11:11.193815 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-8b428744-2c2e-478c-bbd2-1f4e30b17d64 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 16:11:11.210239 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e164bb7b-698a-418e-8802-45cdf8ee06ab admin admin] Validating token access rules against request {{(pid=81767) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 16:11:11.222227 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac6efc63-b8ea-40ab-91c0-dfd7f68221b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a98b1e4431940ab8d1fc95df2858a6c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81767) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 16:11:11.222943 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac6efc63-b8ea-40ab-91c0-dfd7f68221b5 admin admin] REQUEST_METHOD: `PUT` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:11:11.223225 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac6efc63-b8ea-40ab-91c0-dfd7f68221b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:11:11.225275 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac6efc63-b8ea-40ab-91c0-dfd7f68221b5 admin admin] PATH_INFO: `/v3/projects/0fccbf3a45f24f2eaf8d24d6513c148a/users/0e52755517194970ac5a9652bf24827a/roles/9d9848cfd22e4b089db443c350e56670` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:11:11.255209 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac6efc63-b8ea-40ab-91c0-dfd7f68221b5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0fccbf3a45f24f2eaf8d24d6513c148a, user_id=0e52755517194970ac5a9652bf24827a, role_id=9d9848cfd22e4b089db443c350e56670)` {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 16:11:11.259449 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac6efc63-b8ea-40ab-91c0-dfd7f68221b5 admin admin] RBAC: Authorization granted {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 16:11:11.292571 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 923/1822] 10.222.0.15 () {58 vars in 958 bytes} [Mon Aug 22 16:11:11 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 16:11:11.293398 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b428744-2c2e-478c-bbd2-1f4e30b17d64 None None] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 16:11:11.304961 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b428744-2c2e-478c-bbd2-1f4e30b17d64 None None] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 16:11:11.311576 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f81bf035-349e-410c-8bdb-1f4a4f4d8dd4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a98b1e4431940ab8d1fc95df2858a6c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 16:11:11.312273 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f81bf035-349e-410c-8bdb-1f4a4f4d8dd4 admin admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:11:11.312761 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f81bf035-349e-410c-8bdb-1f4a4f4d8dd4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:11:11.313623 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f81bf035-349e-410c-8bdb-1f4a4f4d8dd4 admin admin] PATH_INFO: `/v3/domains` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:11:11.313892 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f81bf035-349e-410c-8bdb-1f4a4f4d8dd4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81768) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 16:11:11.314205 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f81bf035-349e-410c-8bdb-1f4a4f4d8dd4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 16:11:11.315859 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f81bf035-349e-410c-8bdb-1f4a4f4d8dd4 admin admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 16:11:11.329300 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 924/1823] 10.222.0.15 () {58 vars in 1153 bytes} [Mon Aug 22 16:11:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 16:11:11.330212 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f81bf035-349e-410c-8bdb-1f4a4f4d8dd4 admin admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 16:11:11.332383 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-f81bf035-349e-410c-8bdb-1f4a4f4d8dd4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 16:11:11.345773 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 900/1824] 10.222.0.15 () {60 vars in 1483 bytes} [Mon Aug 22 16:11:11 2022] PUT /identity/v3/projects/0fccbf3a45f24f2eaf8d24d6513c148a/users/0e52755517194970ac5a9652bf24827a/roles/9d9848cfd22e4b089db443c350e56670 => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 16:11:11.345773 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac6efc63-b8ea-40ab-91c0-dfd7f68221b5 admin admin] Authenticating user token {{(pid=81767) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 16:11:11.360525 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f81bf035-349e-410c-8bdb-1f4a4f4d8dd4 admin admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 16:11:11.368547 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=92,keystone:INSERT=1,keystone:DELETE=7 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 16:11:11.436422 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71542d5f-5ced-4eea-a763-c5821c591ab8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a98b1e4431940ab8d1fc95df2858a6c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 16:11:11.437185 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71542d5f-5ced-4eea-a763-c5821c591ab8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:11:11.437566 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71542d5f-5ced-4eea-a763-c5821c591ab8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:11:11.438102 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71542d5f-5ced-4eea-a763-c5821c591ab8 admin admin] PATH_INFO: `/v3/users/f4b5760081a2402fa5bdbe84433269b3` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:11:11.439449 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71542d5f-5ced-4eea-a763-c5821c591ab8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f4b5760081a2402fa5bdbe84433269b3)` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 16:11:11.441509 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71542d5f-5ced-4eea-a763-c5821c591ab8 admin admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 16:11:11.445793 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac6efc63-b8ea-40ab-91c0-dfd7f68221b5 admin admin] Validating token access rules against request {{(pid=81767) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 16:11:11.452367 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e223308-4a4e-4347-a1d9-f9d39e798036 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a98b1e4431940ab8d1fc95df2858a6c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81767) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 16:11:11.453047 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e223308-4a4e-4347-a1d9-f9d39e798036 admin admin] REQUEST_METHOD: `PUT` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:11:11.454453 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e223308-4a4e-4347-a1d9-f9d39e798036 admin admin] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:11:11.454453 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e223308-4a4e-4347-a1d9-f9d39e798036 admin admin] PATH_INFO: `/v3/projects/a9a5e391a11747809428d29256acf9bb/users/be4684a264444a52ac5488b5b2ea5ff1/roles/e6fc1ee3fb974125ba9b4fe55b12b45e` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:11:11.472055 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e223308-4a4e-4347-a1d9-f9d39e798036 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a9a5e391a11747809428d29256acf9bb, user_id=be4684a264444a52ac5488b5b2ea5ff1, role_id=e6fc1ee3fb974125ba9b4fe55b12b45e)` {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 16:11:11.474957 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e223308-4a4e-4347-a1d9-f9d39e798036 admin admin] RBAC: Authorization granted {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 16:11:11.559613 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 901/1825] 10.222.0.15 () {60 vars in 1483 bytes} [Mon Aug 22 16:11:11 2022] PUT /identity/v3/projects/a9a5e391a11747809428d29256acf9bb/users/be4684a264444a52ac5488b5b2ea5ff1/roles/e6fc1ee3fb974125ba9b4fe55b12b45e => generated 0 bytes in 211 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 16:11:11.560592 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e223308-4a4e-4347-a1d9-f9d39e798036 admin admin] Authenticating user token {{(pid=81767) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 16:11:11.629615 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e223308-4a4e-4347-a1d9-f9d39e798036 admin admin] Validating token access rules against request {{(pid=81767) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 16:11:11.635081 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5fdaa79-2f55-4d71-9b6b-b2c88748e953 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a98b1e4431940ab8d1fc95df2858a6c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81767) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 16:11:11.636395 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5fdaa79-2f55-4d71-9b6b-b2c88748e953 admin admin] REQUEST_METHOD: `POST` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:11:11.636395 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5fdaa79-2f55-4d71-9b6b-b2c88748e953 admin admin] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:11:11.636395 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5fdaa79-2f55-4d71-9b6b-b2c88748e953 admin admin] PATH_INFO: `/v3/projects` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:11:11.637040 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5fdaa79-2f55-4d71-9b6b-b2c88748e953 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 16:11:11.639068 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5fdaa79-2f55-4d71-9b6b-b2c88748e953 admin admin] RBAC: Authorization granted {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 16:11:11.641131 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-71542d5f-5ced-4eea-a763-c5821c591ab8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f4b5760081a2402fa5bdbe84433269b3'} {{(pid=81768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 16:11:11.641864 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-71542d5f-5ced-4eea-a763-c5821c591ab8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f4b5760081a2402fa5bdbe84433269b3'} {{(pid=81768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 16:11:11.642534 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-71542d5f-5ced-4eea-a763-c5821c591ab8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f4b5760081a2402fa5bdbe84433269b3'} {{(pid=81768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 16:11:11.654833 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-71542d5f-5ced-4eea-a763-c5821c591ab8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f4b5760081a2402fa5bdbe84433269b3'} {{(pid=81768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 16:11:11.821983 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-71542d5f-5ced-4eea-a763-c5821c591ab8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f4b5760081a2402fa5bdbe84433269b3'} {{(pid=81768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 16:11:11.831871 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 902/1826] 10.222.0.15 () {60 vars in 1153 bytes} [Mon Aug 22 16:11:11 2022] POST /identity/v3/projects => generated 365 bytes in 272 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 22 16:11:11.835046 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5fdaa79-2f55-4d71-9b6b-b2c88748e953 admin admin] Authenticating user token {{(pid=81767) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 16:11:11.835159 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.fernet_utils [None req-e5fdaa79-2f55-4d71-9b6b-b2c88748e953 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81767) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 16:11:11.868113 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 925/1827] 10.222.0.15 () {58 vars in 1224 bytes} [Mon Aug 22 16:11:11 2022] DELETE /identity/v3/users/f4b5760081a2402fa5bdbe84433269b3 => generated 0 bytes in 533 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 16:11:11.869009 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71542d5f-5ced-4eea-a763-c5821c591ab8 admin admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 16:11:11.874336 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-71542d5f-5ced-4eea-a763-c5821c591ab8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 16:11:11.998006 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5fdaa79-2f55-4d71-9b6b-b2c88748e953 admin admin] Validating token access rules against request {{(pid=81767) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 16:11:12.010467 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fdc8010-5ad6-41e1-b443-3101ad062cf6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a98b1e4431940ab8d1fc95df2858a6c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81767) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 16:11:12.011365 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fdc8010-5ad6-41e1-b443-3101ad062cf6 admin admin] REQUEST_METHOD: `POST` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:11:12.011785 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fdc8010-5ad6-41e1-b443-3101ad062cf6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:11:12.012234 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fdc8010-5ad6-41e1-b443-3101ad062cf6 admin admin] PATH_INFO: `/v3/projects` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:11:12.013006 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fdc8010-5ad6-41e1-b443-3101ad062cf6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 16:11:12.015856 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fdc8010-5ad6-41e1-b443-3101ad062cf6 admin admin] RBAC: Authorization granted {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 16:11:12.022190 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71542d5f-5ced-4eea-a763-c5821c591ab8 admin admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 16:11:12.032130 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-827c83fe-8677-4fba-baba-9ae4c15797e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a98b1e4431940ab8d1fc95df2858a6c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 16:11:12.035771 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-827c83fe-8677-4fba-baba-9ae4c15797e7 admin admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:11:12.036578 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-827c83fe-8677-4fba-baba-9ae4c15797e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:11:12.036858 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-827c83fe-8677-4fba-baba-9ae4c15797e7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:11:12.038766 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-827c83fe-8677-4fba-baba-9ae4c15797e7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 16:11:12.039901 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-827c83fe-8677-4fba-baba-9ae4c15797e7 admin admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 16:11:12.063573 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 926/1828] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Aug 22 16:11:11 2022] GET /identity/v3/roles => generated 1420 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 22 16:11:12.069572 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6d9b63e-c05e-4a7a-82ea-8e39ba81ff07 None None] REQUEST_METHOD: `POST` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:11:12.070011 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6d9b63e-c05e-4a7a-82ea-8e39ba81ff07 None None] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:11:12.073083 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6d9b63e-c05e-4a7a-82ea-8e39ba81ff07 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:11:12.102443 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.auth.core [None req-f6d9b63e-c05e-4a7a-82ea-8e39ba81ff07 None None] MFA Rules not processed for user `6b2199ae0fb0480aa917ada391f5ec54`. Rule list: `[]` (Enabled: `True`). {{(pid=81768) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 16:11:12.103854 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-f6d9b63e-c05e-4a7a-82ea-8e39ba81ff07 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 16:11:12.116345 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 927/1829] 10.222.0.15 () {58 vars in 958 bytes} [Mon Aug 22 16:11:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 49 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 16:11:12.116345 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6d9b63e-c05e-4a7a-82ea-8e39ba81ff07 None None] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 16:11:12.117646 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-f6d9b63e-c05e-4a7a-82ea-8e39ba81ff07 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 16:11:12.131602 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6d9b63e-c05e-4a7a-82ea-8e39ba81ff07 None None] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 16:11:12.137363 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf322bad-af1e-4165-86ab-8220882bbade admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a98b1e4431940ab8d1fc95df2858a6c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 16:11:12.138148 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf322bad-af1e-4165-86ab-8220882bbade admin admin] REQUEST_METHOD: `POST` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:11:12.138529 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf322bad-af1e-4165-86ab-8220882bbade admin admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:11:12.138963 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf322bad-af1e-4165-86ab-8220882bbade admin admin] PATH_INFO: `/v3/users` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:11:12.140116 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf322bad-af1e-4165-86ab-8220882bbade admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 16:11:12.144151 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf322bad-af1e-4165-86ab-8220882bbade admin admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 16:11:12.185141 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 903/1830] 10.222.0.15 () {60 vars in 1153 bytes} [Mon Aug 22 16:11:11 2022] POST /identity/v3/projects => generated 385 bytes in 353 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 22 16:11:12.187168 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-164564c6-3556-4dcf-a1c7-c73cee50e76d None None] REQUEST_METHOD: `POST` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:11:12.187590 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-164564c6-3556-4dcf-a1c7-c73cee50e76d None None] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:11:12.187975 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-164564c6-3556-4dcf-a1c7-c73cee50e76d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:11:12.197756 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG dbcounter [-] [81767] Writing DB stats keystone:SELECT=83,keystone:DELETE=15,keystone:INSERT=2 {{(pid=81767) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 16:11:12.215605 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.auth.core [None req-164564c6-3556-4dcf-a1c7-c73cee50e76d None None] MFA Rules not processed for user `6b2199ae0fb0480aa917ada391f5ec54`. Rule list: `[]` (Enabled: `True`). {{(pid=81767) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 16:11:12.217259 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.fernet_utils [None req-164564c6-3556-4dcf-a1c7-c73cee50e76d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81767) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 16:11:12.267642 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 928/1831] 10.222.0.15 () {60 vars in 1144 bytes} [Mon Aug 22 16:11:12 2022] POST /identity/v3/users => generated 289 bytes in 149 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 22 16:11:12.267642 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf322bad-af1e-4165-86ab-8220882bbade admin admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 16:11:12.280853 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf322bad-af1e-4165-86ab-8220882bbade admin admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 16:11:12.347495 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 904/1832] 10.222.0.15 () {58 vars in 958 bytes} [Mon Aug 22 16:11:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 16:11:12.348808 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-164564c6-3556-4dcf-a1c7-c73cee50e76d None None] Authenticating user token {{(pid=81767) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 16:11:12.350991 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.fernet_utils [None req-164564c6-3556-4dcf-a1c7-c73cee50e76d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81767) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 16:11:12.420471 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a81a3bde-8a7f-4130-8a6a-5fa0ba884c5c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a98b1e4431940ab8d1fc95df2858a6c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 16:11:12.421108 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a81a3bde-8a7f-4130-8a6a-5fa0ba884c5c admin admin] REQUEST_METHOD: `PUT` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:11:12.421401 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a81a3bde-8a7f-4130-8a6a-5fa0ba884c5c admin admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:11:12.421693 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a81a3bde-8a7f-4130-8a6a-5fa0ba884c5c admin admin] PATH_INFO: `/v3/projects/0fccbf3a45f24f2eaf8d24d6513c148a/users/0e52755517194970ac5a9652bf24827a/roles/9d9848cfd22e4b089db443c350e56670` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:11:12.423822 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a81a3bde-8a7f-4130-8a6a-5fa0ba884c5c admin admin] RBAC: Authorizing `identity:create_grant(project_id=0fccbf3a45f24f2eaf8d24d6513c148a, user_id=0e52755517194970ac5a9652bf24827a, role_id=9d9848cfd22e4b089db443c350e56670)` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 16:11:12.427126 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a81a3bde-8a7f-4130-8a6a-5fa0ba884c5c admin admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 16:11:12.431726 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-164564c6-3556-4dcf-a1c7-c73cee50e76d None None] Validating token access rules against request {{(pid=81767) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 16:11:12.446628 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 929/1833] 10.222.0.15 () {60 vars in 1483 bytes} [Mon Aug 22 16:11:12 2022] PUT /identity/v3/projects/0fccbf3a45f24f2eaf8d24d6513c148a/users/0e52755517194970ac5a9652bf24827a/roles/9d9848cfd22e4b089db443c350e56670 => generated 0 bytes in 183 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 16:11:12.447402 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a81a3bde-8a7f-4130-8a6a-5fa0ba884c5c admin admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 16:11:12.544949 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98f9ab2a-71e5-425f-9d02-41a8edb5f186 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '234687adad594f1aa45775a565ffbdc7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4ee0413f9ef4d4fb1f95a660babe090', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81767) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 16:11:12.545591 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98f9ab2a-71e5-425f-9d02-41a8edb5f186 service neutron] REQUEST_METHOD: `GET` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:11:12.546625 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98f9ab2a-71e5-425f-9d02-41a8edb5f186 service neutron] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:11:12.547034 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98f9ab2a-71e5-425f-9d02-41a8edb5f186 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:11:12.577867 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a81a3bde-8a7f-4130-8a6a-5fa0ba884c5c admin admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 16:11:12.594791 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bfb14da-7b84-4dfb-9904-e7c2c6e07f31 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a98b1e4431940ab8d1fc95df2858a6c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 16:11:12.595578 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bfb14da-7b84-4dfb-9904-e7c2c6e07f31 admin admin] REQUEST_METHOD: `POST` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:11:12.595804 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bfb14da-7b84-4dfb-9904-e7c2c6e07f31 admin admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:11:12.596079 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bfb14da-7b84-4dfb-9904-e7c2c6e07f31 admin admin] PATH_INFO: `/v3/users` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:11:12.596720 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bfb14da-7b84-4dfb-9904-e7c2c6e07f31 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 16:11:12.599706 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bfb14da-7b84-4dfb-9904-e7c2c6e07f31 admin admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 16:11:12.715600 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98f9ab2a-71e5-425f-9d02-41a8edb5f186 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 16:11:12.717967 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98f9ab2a-71e5-425f-9d02-41a8edb5f186 service neutron] RBAC: Authorization granted {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 16:11:12.733181 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 930/1834] 10.222.0.15 () {60 vars in 1144 bytes} [Mon Aug 22 16:11:12 2022] POST /identity/v3/users => generated 293 bytes in 286 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 22 16:11:12.737795 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bfb14da-7b84-4dfb-9904-e7c2c6e07f31 admin admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 16:11:12.751813 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:INSERT=7,keystone:DELETE=3,keystone:SELECT=90 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 16:11:12.763995 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bfb14da-7b84-4dfb-9904-e7c2c6e07f31 admin admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 16:11:12.771678 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 905/1835] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Aug 22 16:11:12 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 423 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 16:11:12.772868 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98f9ab2a-71e5-425f-9d02-41a8edb5f186 service neutron] Authenticating user token {{(pid=81767) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 16:11:12.780087 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08c1adc3-1bc5-4032-8869-0e5a1d4cde66 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a98b1e4431940ab8d1fc95df2858a6c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 16:11:12.780719 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c1adc3-1bc5-4032-8869-0e5a1d4cde66 admin admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:11:12.780996 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c1adc3-1bc5-4032-8869-0e5a1d4cde66 admin admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:11:12.781265 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c1adc3-1bc5-4032-8869-0e5a1d4cde66 admin admin] PATH_INFO: `/v3/roles` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:11:12.781915 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08c1adc3-1bc5-4032-8869-0e5a1d4cde66 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 16:11:12.783291 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98f9ab2a-71e5-425f-9d02-41a8edb5f186 service neutron] Validating token access rules against request {{(pid=81767) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 16:11:12.783554 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08c1adc3-1bc5-4032-8869-0e5a1d4cde66 admin admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 16:11:12.790902 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8ceb55f-e2f1-4b90-bf57-d0e08b395a5e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '234687adad594f1aa45775a565ffbdc7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4ee0413f9ef4d4fb1f95a660babe090', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81767) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 16:11:12.791619 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8ceb55f-e2f1-4b90-bf57-d0e08b395a5e service neutron] REQUEST_METHOD: `GET` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:11:12.792114 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8ceb55f-e2f1-4b90-bf57-d0e08b395a5e service neutron] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:11:12.795664 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8ceb55f-e2f1-4b90-bf57-d0e08b395a5e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:11:12.806524 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 931/1836] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Aug 22 16:11:12 2022] GET /identity/v3/roles => generated 1420 bytes in 69 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 22 16:11:12.810304 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a06184a0-e892-4b30-aebc-0f5c81a8f1ae None None] REQUEST_METHOD: `POST` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:11:12.810589 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a06184a0-e892-4b30-aebc-0f5c81a8f1ae None None] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:11:12.810717 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8ceb55f-e2f1-4b90-bf57-d0e08b395a5e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 16:11:12.810855 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a06184a0-e892-4b30-aebc-0f5c81a8f1ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:11:12.812633 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8ceb55f-e2f1-4b90-bf57-d0e08b395a5e service neutron] RBAC: Authorization granted {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 16:11:12.828142 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 906/1837] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Aug 22 16:11:12 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 55 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 16:11:12.828675 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8ceb55f-e2f1-4b90-bf57-d0e08b395a5e service neutron] Authenticating user token {{(pid=81767) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 16:11:12.876945 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.auth.core [None req-a06184a0-e892-4b30-aebc-0f5c81a8f1ae None None] MFA Rules not processed for user `0e52755517194970ac5a9652bf24827a`. Rule list: `[]` (Enabled: `True`). {{(pid=81768) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 16:11:12.878162 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8ceb55f-e2f1-4b90-bf57-d0e08b395a5e service neutron] Validating token access rules against request {{(pid=81767) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 16:11:12.878276 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-a06184a0-e892-4b30-aebc-0f5c81a8f1ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 16:11:12.890941 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d63444e3-902c-41f9-bad1-e1ac17ece545 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a98b1e4431940ab8d1fc95df2858a6c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81767) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 16:11:12.892732 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d63444e3-902c-41f9-bad1-e1ac17ece545 admin admin] REQUEST_METHOD: `GET` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:11:12.893071 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d63444e3-902c-41f9-bad1-e1ac17ece545 admin admin] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:11:12.893417 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d63444e3-902c-41f9-bad1-e1ac17ece545 admin admin] PATH_INFO: `/v3/roles` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:11:12.894167 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d63444e3-902c-41f9-bad1-e1ac17ece545 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 16:11:12.897415 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d63444e3-902c-41f9-bad1-e1ac17ece545 admin admin] RBAC: Authorization granted {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 16:11:12.921922 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 907/1838] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Aug 22 16:11:12 2022] GET /identity/v3/roles => generated 1420 bytes in 93 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 22 16:11:12.923477 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d63444e3-902c-41f9-bad1-e1ac17ece545 admin admin] Authenticating user token {{(pid=81767) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 16:11:12.938402 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d63444e3-902c-41f9-bad1-e1ac17ece545 admin admin] Validating token access rules against request {{(pid=81767) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 16:11:12.946920 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18a094d3-36cc-4855-b77a-bfacfb8e6fc2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a98b1e4431940ab8d1fc95df2858a6c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81767) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 16:11:12.947876 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18a094d3-36cc-4855-b77a-bfacfb8e6fc2 admin admin] REQUEST_METHOD: `PUT` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:11:12.948367 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18a094d3-36cc-4855-b77a-bfacfb8e6fc2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:11:12.948780 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18a094d3-36cc-4855-b77a-bfacfb8e6fc2 admin admin] PATH_INFO: `/v3/projects/c4b407f80c9b42acb2f9d51952895129/users/0ba0c0c596b345dbbb599ea9ba746ad2/roles/9d9848cfd22e4b089db443c350e56670` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:11:12.968961 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18a094d3-36cc-4855-b77a-bfacfb8e6fc2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c4b407f80c9b42acb2f9d51952895129, user_id=0ba0c0c596b345dbbb599ea9ba746ad2, role_id=9d9848cfd22e4b089db443c350e56670)` {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 16:11:12.972053 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18a094d3-36cc-4855-b77a-bfacfb8e6fc2 admin admin] RBAC: Authorization granted {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 16:11:13.007370 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 932/1839] 10.222.0.15 () {58 vars in 958 bytes} [Mon Aug 22 16:11:12 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 200 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 16:11:13.009124 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a06184a0-e892-4b30-aebc-0f5c81a8f1ae None None] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 16:11:13.023051 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a06184a0-e892-4b30-aebc-0f5c81a8f1ae None None] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 16:11:13.030429 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ae13ff7-567a-4af4-a41f-231a0d12e33f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a98b1e4431940ab8d1fc95df2858a6c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 16:11:13.031244 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae13ff7-567a-4af4-a41f-231a0d12e33f admin admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:11:13.031641 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae13ff7-567a-4af4-a41f-231a0d12e33f admin admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:11:13.032007 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae13ff7-567a-4af4-a41f-231a0d12e33f admin admin] PATH_INFO: `/v3/roles` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:11:13.032768 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ae13ff7-567a-4af4-a41f-231a0d12e33f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 16:11:13.034820 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ae13ff7-567a-4af4-a41f-231a0d12e33f admin admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 16:11:13.049993 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 933/1840] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Aug 22 16:11:13 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 22 16:11:13.054174 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ae13ff7-567a-4af4-a41f-231a0d12e33f admin admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 16:11:13.056103 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-5ae13ff7-567a-4af4-a41f-231a0d12e33f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 16:11:13.058573 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 908/1841] 10.222.0.15 () {60 vars in 1483 bytes} [Mon Aug 22 16:11:12 2022] PUT /identity/v3/projects/c4b407f80c9b42acb2f9d51952895129/users/0ba0c0c596b345dbbb599ea9ba746ad2/roles/9d9848cfd22e4b089db443c350e56670 => generated 0 bytes in 134 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 16:11:13.058573 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18a094d3-36cc-4855-b77a-bfacfb8e6fc2 admin admin] Authenticating user token {{(pid=81767) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 16:11:13.139822 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ae13ff7-567a-4af4-a41f-231a0d12e33f admin admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 16:11:13.206466 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18a094d3-36cc-4855-b77a-bfacfb8e6fc2 admin admin] Validating token access rules against request {{(pid=81767) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 16:11:13.209412 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-277c4fa9-2a3f-41ee-a637-33f2dd06605f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79706d480f864998b9f8594be407ceab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4ee0413f9ef4d4fb1f95a660babe090', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 16:11:13.211047 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-277c4fa9-2a3f-41ee-a637-33f2dd06605f service cinder] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:11:13.212869 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-277c4fa9-2a3f-41ee-a637-33f2dd06605f service cinder] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:11:13.213185 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44c4bd91-9322-4008-9357-cc650d8bd540 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a98b1e4431940ab8d1fc95df2858a6c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81767) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 16:11:13.213362 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-277c4fa9-2a3f-41ee-a637-33f2dd06605f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:11:13.214173 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44c4bd91-9322-4008-9357-cc650d8bd540 admin admin] REQUEST_METHOD: `PUT` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:11:13.214433 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44c4bd91-9322-4008-9357-cc650d8bd540 admin admin] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:11:13.214706 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44c4bd91-9322-4008-9357-cc650d8bd540 admin admin] PATH_INFO: `/v3/projects/7f63a07aba354511b3839295aaaf1c0e/users/9a95d57d9e7d4f8c81a2b6f038448784/roles/e6fc1ee3fb974125ba9b4fe55b12b45e` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:11:13.219060 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-277c4fa9-2a3f-41ee-a637-33f2dd06605f service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 16:11:13.235327 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44c4bd91-9322-4008-9357-cc650d8bd540 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7f63a07aba354511b3839295aaaf1c0e, user_id=9a95d57d9e7d4f8c81a2b6f038448784, role_id=e6fc1ee3fb974125ba9b4fe55b12b45e)` {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 16:11:13.242644 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44c4bd91-9322-4008-9357-cc650d8bd540 admin admin] RBAC: Authorization granted {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 16:11:13.307257 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-277c4fa9-2a3f-41ee-a637-33f2dd06605f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 16:11:13.309135 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-277c4fa9-2a3f-41ee-a637-33f2dd06605f service cinder] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 16:11:13.321216 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 934/1842] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Aug 22 16:11:13 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 271 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 16:11:13.324581 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-277c4fa9-2a3f-41ee-a637-33f2dd06605f service cinder] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 16:11:13.349470 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 909/1843] 10.222.0.15 () {60 vars in 1483 bytes} [Mon Aug 22 16:11:13 2022] PUT /identity/v3/projects/7f63a07aba354511b3839295aaaf1c0e/users/9a95d57d9e7d4f8c81a2b6f038448784/roles/e6fc1ee3fb974125ba9b4fe55b12b45e => generated 0 bytes in 292 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 16:11:13.351514 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-277c4fa9-2a3f-41ee-a637-33f2dd06605f service cinder] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 16:11:13.361168 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d48f84e6-c1bc-45ab-b041-5200f9b71f99 None None] REQUEST_METHOD: `POST` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:11:13.361485 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d48f84e6-c1bc-45ab-b041-5200f9b71f99 None None] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:11:13.361732 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d48f84e6-c1bc-45ab-b041-5200f9b71f99 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:11:13.428217 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.auth.core [None req-d48f84e6-c1bc-45ab-b041-5200f9b71f99 None None] MFA Rules not processed for user `9a95d57d9e7d4f8c81a2b6f038448784`. Rule list: `[]` (Enabled: `True`). {{(pid=81767) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 16:11:13.430208 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.fernet_utils [None req-d48f84e6-c1bc-45ab-b041-5200f9b71f99 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81767) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 16:11:13.472392 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8be2058-0f4d-4d82-9192-5657e4f2f33a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a98b1e4431940ab8d1fc95df2858a6c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 16:11:13.473311 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8be2058-0f4d-4d82-9192-5657e4f2f33a admin admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:11:13.474109 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8be2058-0f4d-4d82-9192-5657e4f2f33a admin admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:11:13.474381 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8be2058-0f4d-4d82-9192-5657e4f2f33a admin admin] PATH_INFO: `/v3/roles` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:11:13.477508 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8be2058-0f4d-4d82-9192-5657e4f2f33a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 16:11:13.483187 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8be2058-0f4d-4d82-9192-5657e4f2f33a admin admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 16:11:13.513231 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 935/1844] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Aug 22 16:11:13 2022] GET /identity/v3/roles => generated 1420 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 22 16:11:13.514381 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8be2058-0f4d-4d82-9192-5657e4f2f33a admin admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 16:11:13.519866 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-f8be2058-0f4d-4d82-9192-5657e4f2f33a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 16:11:13.544888 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG dbcounter [-] [81767] Writing DB stats keystone:SELECT=97,keystone:DELETE=2,keystone:INSERT=1 {{(pid=81767) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 16:11:13.548226 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8be2058-0f4d-4d82-9192-5657e4f2f33a admin admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 16:11:13.561783 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a0fc153-f498-4d5e-813d-73de83bedc2c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a98b1e4431940ab8d1fc95df2858a6c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 16:11:13.562441 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a0fc153-f498-4d5e-813d-73de83bedc2c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:11:13.562720 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a0fc153-f498-4d5e-813d-73de83bedc2c admin admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:11:13.563739 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a0fc153-f498-4d5e-813d-73de83bedc2c admin admin] PATH_INFO: `/v3/users/6ec6b84e494c481abe9c9772a10037c9` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:11:13.565566 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a0fc153-f498-4d5e-813d-73de83bedc2c admin admin] RBAC: Authorizing `identity:delete_user(user_id=6ec6b84e494c481abe9c9772a10037c9)` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 16:11:13.572009 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a0fc153-f498-4d5e-813d-73de83bedc2c admin admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 16:11:13.620510 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 910/1845] 10.222.0.15 () {58 vars in 958 bytes} [Mon Aug 22 16:11:13 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 256 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 16:11:13.620510 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d48f84e6-c1bc-45ab-b041-5200f9b71f99 None None] Authenticating user token {{(pid=81767) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 16:11:13.646394 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d48f84e6-c1bc-45ab-b041-5200f9b71f99 None None] Validating token access rules against request {{(pid=81767) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 16:11:13.659567 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c6832e8-e955-4bd5-a324-83e552ed0dd4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a98b1e4431940ab8d1fc95df2858a6c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81767) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 16:11:13.660256 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c6832e8-e955-4bd5-a324-83e552ed0dd4 admin admin] REQUEST_METHOD: `PUT` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:11:13.660743 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c6832e8-e955-4bd5-a324-83e552ed0dd4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:11:13.661496 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c6832e8-e955-4bd5-a324-83e552ed0dd4 admin admin] PATH_INFO: `/v3/projects/c4b407f80c9b42acb2f9d51952895129/users/0ba0c0c596b345dbbb599ea9ba746ad2/roles/9d9848cfd22e4b089db443c350e56670` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:11:13.667574 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c6832e8-e955-4bd5-a324-83e552ed0dd4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c4b407f80c9b42acb2f9d51952895129, user_id=0ba0c0c596b345dbbb599ea9ba746ad2, role_id=9d9848cfd22e4b089db443c350e56670)` {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 16:11:13.670335 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c6832e8-e955-4bd5-a324-83e552ed0dd4 admin admin] RBAC: Authorization granted {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 16:11:13.682890 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 911/1846] 10.222.0.15 () {60 vars in 1483 bytes} [Mon Aug 22 16:11:13 2022] PUT /identity/v3/projects/c4b407f80c9b42acb2f9d51952895129/users/0ba0c0c596b345dbbb599ea9ba746ad2/roles/9d9848cfd22e4b089db443c350e56670 => generated 0 bytes in 66 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 16:11:13.683756 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c6832e8-e955-4bd5-a324-83e552ed0dd4 admin admin] Authenticating user token {{(pid=81767) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 16:11:13.692358 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.fernet_utils [None req-7c6832e8-e955-4bd5-a324-83e552ed0dd4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81767) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 16:11:13.773728 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c6832e8-e955-4bd5-a324-83e552ed0dd4 admin admin] Validating token access rules against request {{(pid=81767) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 16:11:13.778280 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42e35677-d980-4c34-b8d6-f59a2a0ebb9f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79706d480f864998b9f8594be407ceab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4ee0413f9ef4d4fb1f95a660babe090', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81767) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 16:11:13.778531 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42e35677-d980-4c34-b8d6-f59a2a0ebb9f service cinder] REQUEST_METHOD: `GET` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:11:13.778805 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42e35677-d980-4c34-b8d6-f59a2a0ebb9f service cinder] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:11:13.779073 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42e35677-d980-4c34-b8d6-f59a2a0ebb9f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:11:13.841497 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42e35677-d980-4c34-b8d6-f59a2a0ebb9f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 16:11:13.843235 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42e35677-d980-4c34-b8d6-f59a2a0ebb9f service cinder] RBAC: Authorization granted {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 16:11:13.853665 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 912/1847] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Aug 22 16:11:13 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 170 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 16:11:13.854935 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd833745-b8bd-42bb-bd88-2aff86c5a4d1 None None] REQUEST_METHOD: `POST` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:11:13.855345 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd833745-b8bd-42bb-bd88-2aff86c5a4d1 None None] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:11:13.855515 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd833745-b8bd-42bb-bd88-2aff86c5a4d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:11:13.865165 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-1a0fc153-f498-4d5e-813d-73de83bedc2c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6ec6b84e494c481abe9c9772a10037c9'} {{(pid=81768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 16:11:13.865165 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-1a0fc153-f498-4d5e-813d-73de83bedc2c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6ec6b84e494c481abe9c9772a10037c9'} {{(pid=81768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 16:11:13.865476 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-1a0fc153-f498-4d5e-813d-73de83bedc2c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6ec6b84e494c481abe9c9772a10037c9'} {{(pid=81768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 16:11:13.874583 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-1a0fc153-f498-4d5e-813d-73de83bedc2c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6ec6b84e494c481abe9c9772a10037c9'} {{(pid=81768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 16:11:13.887492 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.auth.core [None req-fd833745-b8bd-42bb-bd88-2aff86c5a4d1 None None] MFA Rules not processed for user `0ba0c0c596b345dbbb599ea9ba746ad2`. Rule list: `[]` (Enabled: `True`). {{(pid=81767) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 16:11:13.888762 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.fernet_utils [None req-fd833745-b8bd-42bb-bd88-2aff86c5a4d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81767) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 16:11:13.970067 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-1a0fc153-f498-4d5e-813d-73de83bedc2c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6ec6b84e494c481abe9c9772a10037c9'} {{(pid=81768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 16:11:14.011213 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 936/1848] 10.222.0.15 () {58 vars in 1224 bytes} [Mon Aug 22 16:11:13 2022] DELETE /identity/v3/users/6ec6b84e494c481abe9c9772a10037c9 => generated 0 bytes in 497 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 16:11:14.012550 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a0fc153-f498-4d5e-813d-73de83bedc2c admin admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 16:11:14.017882 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-1a0fc153-f498-4d5e-813d-73de83bedc2c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 16:11:14.029199 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 913/1849] 10.222.0.15 () {58 vars in 958 bytes} [Mon Aug 22 16:11:13 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 16:11:14.157257 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a0fc153-f498-4d5e-813d-73de83bedc2c admin admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 16:11:14.176502 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bbe46cf-f4a7-4282-b57d-a5b607481759 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a98b1e4431940ab8d1fc95df2858a6c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 16:11:14.177170 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bbe46cf-f4a7-4282-b57d-a5b607481759 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:11:14.177444 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bbe46cf-f4a7-4282-b57d-a5b607481759 admin admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:11:14.177762 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bbe46cf-f4a7-4282-b57d-a5b607481759 admin admin] PATH_INFO: `/v3/projects/fa3302b8c19c41f481207eab12a8ef2c` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:11:14.178920 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bbe46cf-f4a7-4282-b57d-a5b607481759 admin admin] RBAC: Authorizing `identity:delete_project(project_id=fa3302b8c19c41f481207eab12a8ef2c)` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 16:11:14.180873 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bbe46cf-f4a7-4282-b57d-a5b607481759 admin admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 16:11:14.329119 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-9bbe46cf-f4a7-4282-b57d-a5b607481759 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fa3302b8c19c41f481207eab12a8ef2c', 'request_id': 'req-9bbe46cf-f4a7-4282-b57d-a5b607481759'} {{(pid=81768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 16:11:14.338416 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-9bbe46cf-f4a7-4282-b57d-a5b607481759 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fa3302b8c19c41f481207eab12a8ef2c', 'request_id': 'req-9bbe46cf-f4a7-4282-b57d-a5b607481759'} {{(pid=81768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 16:11:14.400348 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd833745-b8bd-42bb-bd88-2aff86c5a4d1 None None] Authenticating user token {{(pid=81767) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 16:11:14.401963 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.fernet_utils [None req-fd833745-b8bd-42bb-bd88-2aff86c5a4d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81767) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 16:11:14.416276 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-9bbe46cf-f4a7-4282-b57d-a5b607481759 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fa3302b8c19c41f481207eab12a8ef2c', 'request_id': 'req-9bbe46cf-f4a7-4282-b57d-a5b607481759'} {{(pid=81768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 16:11:14.421525 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd833745-b8bd-42bb-bd88-2aff86c5a4d1 None None] Validating token access rules against request {{(pid=81767) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 16:11:14.436031 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-9bbe46cf-f4a7-4282-b57d-a5b607481759 admin admin] The token cache is being invalidate because project fa3302b8c19c41f481207eab12a8ef2c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81768) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 22 16:11:14.436323 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-9bbe46cf-f4a7-4282-b57d-a5b607481759 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 16:11:14.437016 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-9bbe46cf-f4a7-4282-b57d-a5b607481759 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 16:11:14.438562 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 937/1850] 10.222.0.15 () {58 vars in 1233 bytes} [Mon Aug 22 16:11:14 2022] DELETE /identity/v3/projects/fa3302b8c19c41f481207eab12a8ef2c => generated 0 bytes in 426 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 16:11:14.443083 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bbe46cf-f4a7-4282-b57d-a5b607481759 admin admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 16:11:14.444508 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-9bbe46cf-f4a7-4282-b57d-a5b607481759 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 16:11:14.523683 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66d746a4-8443-4513-9f04-86a06888f3b0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79706d480f864998b9f8594be407ceab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4ee0413f9ef4d4fb1f95a660babe090', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81767) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 16:11:14.524935 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d746a4-8443-4513-9f04-86a06888f3b0 service cinder] REQUEST_METHOD: `GET` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:11:14.525246 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d746a4-8443-4513-9f04-86a06888f3b0 service cinder] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:11:14.525587 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d746a4-8443-4513-9f04-86a06888f3b0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:11:14.531957 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.fernet_utils [None req-66d746a4-8443-4513-9f04-86a06888f3b0 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81767) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 16:11:14.586678 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bbe46cf-f4a7-4282-b57d-a5b607481759 admin admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 16:11:14.595323 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5f81d9b-0887-4eae-9a44-5aa1789c6c7e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a98b1e4431940ab8d1fc95df2858a6c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 16:11:14.596771 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5f81d9b-0887-4eae-9a44-5aa1789c6c7e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:11:14.597199 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5f81d9b-0887-4eae-9a44-5aa1789c6c7e admin admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:11:14.597618 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5f81d9b-0887-4eae-9a44-5aa1789c6c7e admin admin] PATH_INFO: `/v3/users/6125dea54e5d4b579f16f7b66c0ed472` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:11:14.599999 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5f81d9b-0887-4eae-9a44-5aa1789c6c7e admin admin] RBAC: Authorizing `identity:delete_user(user_id=6125dea54e5d4b579f16f7b66c0ed472)` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 16:11:14.602353 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5f81d9b-0887-4eae-9a44-5aa1789c6c7e admin admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 16:11:14.680176 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66d746a4-8443-4513-9f04-86a06888f3b0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 16:11:14.682019 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66d746a4-8443-4513-9f04-86a06888f3b0 service cinder] RBAC: Authorization granted {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 16:11:14.694578 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 914/1851] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Aug 22 16:11:14 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 295 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 16:11:14.765757 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 16:11:14.828865 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66d746a4-8443-4513-9f04-86a06888f3b0 service cinder] Authenticating user token {{(pid=81767) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 16:11:14.830555 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.fernet_utils [None req-66d746a4-8443-4513-9f04-86a06888f3b0 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81767) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 16:11:14.856788 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66d746a4-8443-4513-9f04-86a06888f3b0 service cinder] Validating token access rules against request {{(pid=81767) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 16:11:14.881210 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ae069d9-9b00-4801-8241-5b8e52a73c23 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a98b1e4431940ab8d1fc95df2858a6c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81767) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 16:11:14.881885 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ae069d9-9b00-4801-8241-5b8e52a73c23 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:11:14.882168 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ae069d9-9b00-4801-8241-5b8e52a73c23 admin admin] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:11:14.882447 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ae069d9-9b00-4801-8241-5b8e52a73c23 admin admin] PATH_INFO: `/v3/projects/efef11be3fe24f61975e2b30f1c31a94` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:11:14.885015 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ae069d9-9b00-4801-8241-5b8e52a73c23 admin admin] RBAC: Authorizing `identity:delete_project(project_id=efef11be3fe24f61975e2b30f1c31a94)` {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 16:11:14.890299 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ae069d9-9b00-4801-8241-5b8e52a73c23 admin admin] RBAC: Authorization granted {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 16:11:14.961003 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-e5f81d9b-0887-4eae-9a44-5aa1789c6c7e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6125dea54e5d4b579f16f7b66c0ed472'} {{(pid=81768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 16:11:14.962151 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-e5f81d9b-0887-4eae-9a44-5aa1789c6c7e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6125dea54e5d4b579f16f7b66c0ed472'} {{(pid=81768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 16:11:14.962363 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-e5f81d9b-0887-4eae-9a44-5aa1789c6c7e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6125dea54e5d4b579f16f7b66c0ed472'} {{(pid=81768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 16:11:14.972619 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-e5f81d9b-0887-4eae-9a44-5aa1789c6c7e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6125dea54e5d4b579f16f7b66c0ed472'} {{(pid=81768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 16:11:15.030630 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.notifications [None req-3ae069d9-9b00-4801-8241-5b8e52a73c23 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'efef11be3fe24f61975e2b30f1c31a94', 'request_id': 'req-3ae069d9-9b00-4801-8241-5b8e52a73c23'} {{(pid=81767) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 16:11:15.039602 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.notifications [None req-3ae069d9-9b00-4801-8241-5b8e52a73c23 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'efef11be3fe24f61975e2b30f1c31a94', 'request_id': 'req-3ae069d9-9b00-4801-8241-5b8e52a73c23'} {{(pid=81767) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 16:11:15.091386 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-e5f81d9b-0887-4eae-9a44-5aa1789c6c7e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6125dea54e5d4b579f16f7b66c0ed472'} {{(pid=81768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 16:11:15.116758 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 938/1852] 10.222.0.15 () {58 vars in 1224 bytes} [Mon Aug 22 16:11:14 2022] DELETE /identity/v3/users/6125dea54e5d4b579f16f7b66c0ed472 => generated 0 bytes in 671 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 22 16:11:15.180370 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.notifications [None req-3ae069d9-9b00-4801-8241-5b8e52a73c23 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'efef11be3fe24f61975e2b30f1c31a94', 'request_id': 'req-3ae069d9-9b00-4801-8241-5b8e52a73c23'} {{(pid=81767) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 16:11:15.197746 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.notifications [None req-3ae069d9-9b00-4801-8241-5b8e52a73c23 admin admin] The token cache is being invalidate because project efef11be3fe24f61975e2b30f1c31a94 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81767) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 22 16:11:15.198101 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.notifications [None req-3ae069d9-9b00-4801-8241-5b8e52a73c23 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81767) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 16:11:15.198811 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.notifications [None req-3ae069d9-9b00-4801-8241-5b8e52a73c23 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81767) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 16:11:15.205420 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 915/1853] 10.222.0.15 () {58 vars in 1233 bytes} [Mon Aug 22 16:11:14 2022] DELETE /identity/v3/projects/efef11be3fe24f61975e2b30f1c31a94 => generated 0 bytes in 372 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 16:11:15.208411 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9be3b6e-6fb2-4692-8d7d-bfc0516d865e None None] REQUEST_METHOD: `POST` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:11:15.209158 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9be3b6e-6fb2-4692-8d7d-bfc0516d865e None None] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:11:15.209784 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9be3b6e-6fb2-4692-8d7d-bfc0516d865e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:11:15.257311 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.auth.core [None req-f9be3b6e-6fb2-4692-8d7d-bfc0516d865e None None] MFA Rules not processed for user `6b2199ae0fb0480aa917ada391f5ec54`. Rule list: `[]` (Enabled: `True`). {{(pid=81768) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 16:11:15.258938 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-f9be3b6e-6fb2-4692-8d7d-bfc0516d865e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 16:11:15.363854 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 939/1854] 10.222.0.15 () {58 vars in 958 bytes} [Mon Aug 22 16:11:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 16:11:15.364670 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11797973-b238-4124-89f4-71e95c5dc82b None None] REQUEST_METHOD: `POST` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:11:15.365141 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11797973-b238-4124-89f4-71e95c5dc82b None None] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:11:15.365784 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11797973-b238-4124-89f4-71e95c5dc82b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:11:15.416290 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.auth.core [None req-11797973-b238-4124-89f4-71e95c5dc82b None None] MFA Rules not processed for user `6b2199ae0fb0480aa917ada391f5ec54`. Rule list: `[]` (Enabled: `True`). {{(pid=81767) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 16:11:15.417394 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.fernet_utils [None req-11797973-b238-4124-89f4-71e95c5dc82b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81767) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 16:11:15.429404 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 916/1855] 10.222.0.15 () {58 vars in 958 bytes} [Mon Aug 22 16:11:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 67 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 16:11:15.439786 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9be3b6e-6fb2-4692-8d7d-bfc0516d865e None None] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 16:11:15.453917 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9be3b6e-6fb2-4692-8d7d-bfc0516d865e None None] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 16:11:15.460520 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-521babc5-d63d-4a63-a963-b90a88e78654 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a98b1e4431940ab8d1fc95df2858a6c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 16:11:15.461863 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-521babc5-d63d-4a63-a963-b90a88e78654 admin admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:11:15.462145 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-521babc5-d63d-4a63-a963-b90a88e78654 admin admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:11:15.462560 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-521babc5-d63d-4a63-a963-b90a88e78654 admin admin] PATH_INFO: `/v3/domains` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:11:15.463410 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-521babc5-d63d-4a63-a963-b90a88e78654 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81768) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 16:11:15.463855 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-521babc5-d63d-4a63-a963-b90a88e78654 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 16:11:15.466186 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-521babc5-d63d-4a63-a963-b90a88e78654 admin admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 16:11:15.483321 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 940/1856] 10.222.0.15 () {58 vars in 1153 bytes} [Mon Aug 22 16:11:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 16:11:15.485683 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11797973-b238-4124-89f4-71e95c5dc82b None None] Authenticating user token {{(pid=81767) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 16:11:15.497682 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11797973-b238-4124-89f4-71e95c5dc82b None None] Validating token access rules against request {{(pid=81767) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 16:11:15.502838 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5414203-a974-499b-a14a-ff753561acef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a98b1e4431940ab8d1fc95df2858a6c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81767) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 16:11:15.503595 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5414203-a974-499b-a14a-ff753561acef admin admin] REQUEST_METHOD: `POST` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:11:15.504099 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5414203-a974-499b-a14a-ff753561acef admin admin] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:11:15.504458 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5414203-a974-499b-a14a-ff753561acef admin admin] PATH_INFO: `/v3/projects` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:11:15.505937 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5414203-a974-499b-a14a-ff753561acef admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 16:11:15.508304 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5414203-a974-499b-a14a-ff753561acef admin admin] RBAC: Authorization granted {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 16:11:15.621016 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 917/1857] 10.222.0.15 () {60 vars in 1153 bytes} [Mon Aug 22 16:11:15 2022] POST /identity/v3/projects => generated 369 bytes in 134 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 22 16:11:15.622959 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-521babc5-d63d-4a63-a963-b90a88e78654 admin admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 16:11:15.634790 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-521babc5-d63d-4a63-a963-b90a88e78654 admin admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 16:11:15.719976 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG dbcounter [-] [81767] Writing DB stats keystone:SELECT=91,keystone:INSERT=4,keystone:DELETE=5 {{(pid=81767) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 16:11:15.734097 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c2a447c-2c3b-492d-8f03-71e1be4180dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a98b1e4431940ab8d1fc95df2858a6c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 16:11:15.734724 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c2a447c-2c3b-492d-8f03-71e1be4180dd admin admin] REQUEST_METHOD: `POST` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:11:15.735003 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c2a447c-2c3b-492d-8f03-71e1be4180dd admin admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:11:15.735296 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c2a447c-2c3b-492d-8f03-71e1be4180dd admin admin] PATH_INFO: `/v3/users` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:11:15.746469 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c2a447c-2c3b-492d-8f03-71e1be4180dd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 16:11:15.750329 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c2a447c-2c3b-492d-8f03-71e1be4180dd admin admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 16:11:15.819798 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5414203-a974-499b-a14a-ff753561acef admin admin] Authenticating user token {{(pid=81767) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 16:11:15.825363 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.fernet_utils [None req-b5414203-a974-499b-a14a-ff753561acef admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81767) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 16:11:15.858501 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 941/1858] 10.222.0.15 () {60 vars in 1144 bytes} [Mon Aug 22 16:11:15 2022] POST /identity/v3/users => generated 285 bytes in 236 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 22 16:11:15.860530 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5414203-a974-499b-a14a-ff753561acef admin admin] Validating token access rules against request {{(pid=81767) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 16:11:15.862402 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c2a447c-2c3b-492d-8f03-71e1be4180dd admin admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 16:11:15.881881 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16cbc3fb-348b-4198-b0d2-a09c32b87fb0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a98b1e4431940ab8d1fc95df2858a6c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81767) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 16:11:15.883016 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16cbc3fb-348b-4198-b0d2-a09c32b87fb0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:11:15.885950 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16cbc3fb-348b-4198-b0d2-a09c32b87fb0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:11:15.886638 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16cbc3fb-348b-4198-b0d2-a09c32b87fb0 admin admin] PATH_INFO: `/v3/projects/8145e6e07ef04a03b256894e82711ad4` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:11:15.890239 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16cbc3fb-348b-4198-b0d2-a09c32b87fb0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8145e6e07ef04a03b256894e82711ad4)` {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 16:11:15.892234 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16cbc3fb-348b-4198-b0d2-a09c32b87fb0 admin admin] RBAC: Authorization granted {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 16:11:15.895675 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c2a447c-2c3b-492d-8f03-71e1be4180dd admin admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 16:11:15.907677 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f172b959-0615-4d12-bb4b-e39885e15570 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a98b1e4431940ab8d1fc95df2858a6c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 16:11:15.908828 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f172b959-0615-4d12-bb4b-e39885e15570 admin admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:11:15.908828 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f172b959-0615-4d12-bb4b-e39885e15570 admin admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:11:15.909011 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f172b959-0615-4d12-bb4b-e39885e15570 admin admin] PATH_INFO: `/v3/roles` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:11:15.914409 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f172b959-0615-4d12-bb4b-e39885e15570 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 16:11:15.918302 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f172b959-0615-4d12-bb4b-e39885e15570 admin admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 16:11:15.953185 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 942/1859] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Aug 22 16:11:15 2022] GET /identity/v3/roles => generated 1420 bytes in 91 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 22 16:11:15.957315 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f172b959-0615-4d12-bb4b-e39885e15570 admin admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 16:11:15.981582 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f172b959-0615-4d12-bb4b-e39885e15570 admin admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 16:11:15.988615 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86df09fb-256e-4146-8c50-b1b1b3bec97f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a98b1e4431940ab8d1fc95df2858a6c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 16:11:15.990195 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86df09fb-256e-4146-8c50-b1b1b3bec97f admin admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:11:15.990479 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86df09fb-256e-4146-8c50-b1b1b3bec97f admin admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:11:15.990783 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86df09fb-256e-4146-8c50-b1b1b3bec97f admin admin] PATH_INFO: `/v3/roles` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:11:15.991798 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86df09fb-256e-4146-8c50-b1b1b3bec97f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 16:11:15.994902 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86df09fb-256e-4146-8c50-b1b1b3bec97f admin admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 16:11:16.020032 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 943/1860] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Aug 22 16:11:15 2022] GET /identity/v3/roles => generated 1420 bytes in 61 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 22 16:11:16.022003 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86df09fb-256e-4146-8c50-b1b1b3bec97f admin admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 16:11:16.039134 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86df09fb-256e-4146-8c50-b1b1b3bec97f admin admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 16:11:16.044647 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d348989-4fce-463b-95d5-e121ff268be7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a98b1e4431940ab8d1fc95df2858a6c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 16:11:16.045760 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d348989-4fce-463b-95d5-e121ff268be7 admin admin] REQUEST_METHOD: `PUT` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:11:16.051940 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d348989-4fce-463b-95d5-e121ff268be7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:11:16.051940 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d348989-4fce-463b-95d5-e121ff268be7 admin admin] PATH_INFO: `/v3/projects/715274740943419bb2989d6a2e4ccb43/users/50c24fc9331c41d6ae4aa1274c6c491d/roles/e6fc1ee3fb974125ba9b4fe55b12b45e` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:11:16.065857 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d348989-4fce-463b-95d5-e121ff268be7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=715274740943419bb2989d6a2e4ccb43, user_id=50c24fc9331c41d6ae4aa1274c6c491d, role_id=e6fc1ee3fb974125ba9b4fe55b12b45e)` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 16:11:16.068621 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d348989-4fce-463b-95d5-e121ff268be7 admin admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 16:11:16.189992 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.notifications [None req-16cbc3fb-348b-4198-b0d2-a09c32b87fb0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8145e6e07ef04a03b256894e82711ad4', 'request_id': 'req-16cbc3fb-348b-4198-b0d2-a09c32b87fb0'} {{(pid=81767) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 16:11:16.203690 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 944/1861] 10.222.0.15 () {60 vars in 1483 bytes} [Mon Aug 22 16:11:16 2022] PUT /identity/v3/projects/715274740943419bb2989d6a2e4ccb43/users/50c24fc9331c41d6ae4aa1274c6c491d/roles/e6fc1ee3fb974125ba9b4fe55b12b45e => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 22 16:11:16.203690 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d348989-4fce-463b-95d5-e121ff268be7 admin admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 16:11:16.203690 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-7d348989-4fce-463b-95d5-e121ff268be7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 16:11:16.206237 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.notifications [None req-16cbc3fb-348b-4198-b0d2-a09c32b87fb0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8145e6e07ef04a03b256894e82711ad4', 'request_id': 'req-16cbc3fb-348b-4198-b0d2-a09c32b87fb0'} {{(pid=81767) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 16:11:16.298489 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d348989-4fce-463b-95d5-e121ff268be7 admin admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 16:11:16.304399 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a2924ea-d73d-479b-ab15-b291ceea1bd9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a98b1e4431940ab8d1fc95df2858a6c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 16:11:16.305094 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a2924ea-d73d-479b-ab15-b291ceea1bd9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:11:16.305462 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a2924ea-d73d-479b-ab15-b291ceea1bd9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:11:16.305799 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a2924ea-d73d-479b-ab15-b291ceea1bd9 admin admin] PATH_INFO: `/v3/users/faba300378174db08541de5f62b55ebe` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:11:16.307142 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a2924ea-d73d-479b-ab15-b291ceea1bd9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=faba300378174db08541de5f62b55ebe)` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 16:11:16.309272 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a2924ea-d73d-479b-ab15-b291ceea1bd9 admin admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 16:11:16.313486 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.notifications [None req-16cbc3fb-348b-4198-b0d2-a09c32b87fb0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8145e6e07ef04a03b256894e82711ad4', 'request_id': 'req-16cbc3fb-348b-4198-b0d2-a09c32b87fb0'} {{(pid=81767) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 16:11:16.339374 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.notifications [None req-16cbc3fb-348b-4198-b0d2-a09c32b87fb0 admin admin] The token cache is being invalidate because project 8145e6e07ef04a03b256894e82711ad4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81767) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 22 16:11:16.339866 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.notifications [None req-16cbc3fb-348b-4198-b0d2-a09c32b87fb0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81767) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 16:11:16.340871 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.notifications [None req-16cbc3fb-348b-4198-b0d2-a09c32b87fb0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81767) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 16:11:16.346229 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 918/1862] 10.222.0.15 () {58 vars in 1233 bytes} [Mon Aug 22 16:11:15 2022] DELETE /identity/v3/projects/8145e6e07ef04a03b256894e82711ad4 => generated 0 bytes in 524 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 16:11:16.346229 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fe37c80-621b-41fb-8075-3a413e4d7825 None None] REQUEST_METHOD: `POST` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:11:16.346229 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fe37c80-621b-41fb-8075-3a413e4d7825 None None] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:11:16.347842 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fe37c80-621b-41fb-8075-3a413e4d7825 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:11:16.386895 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.auth.core [None req-9fe37c80-621b-41fb-8075-3a413e4d7825 None None] MFA Rules not processed for user `50c24fc9331c41d6ae4aa1274c6c491d`. Rule list: `[]` (Enabled: `True`). {{(pid=81767) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 16:11:16.388128 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.fernet_utils [None req-9fe37c80-621b-41fb-8075-3a413e4d7825 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81767) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 16:11:16.453889 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 919/1863] 10.222.0.15 () {58 vars in 958 bytes} [Mon Aug 22 16:11:16 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 16:11:16.455041 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-139f1c4f-41c5-4f7a-abfa-9d77113e942e None None] REQUEST_METHOD: `POST` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:11:16.455348 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-139f1c4f-41c5-4f7a-abfa-9d77113e942e None None] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:11:16.455622 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-139f1c4f-41c5-4f7a-abfa-9d77113e942e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:11:16.475231 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.auth.core [None req-139f1c4f-41c5-4f7a-abfa-9d77113e942e None None] MFA Rules not processed for user `6b2199ae0fb0480aa917ada391f5ec54`. Rule list: `[]` (Enabled: `True`). {{(pid=81767) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 16:11:16.476446 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.fernet_utils [None req-139f1c4f-41c5-4f7a-abfa-9d77113e942e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81767) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 16:11:16.490191 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 920/1864] 10.222.0.15 () {58 vars in 958 bytes} [Mon Aug 22 16:11:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 16:11:16.490191 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-139f1c4f-41c5-4f7a-abfa-9d77113e942e None None] Authenticating user token {{(pid=81767) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 16:11:16.490191 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.fernet_utils [None req-139f1c4f-41c5-4f7a-abfa-9d77113e942e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81767) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 16:11:16.500975 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-139f1c4f-41c5-4f7a-abfa-9d77113e942e None None] Validating token access rules against request {{(pid=81767) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 16:11:16.525882 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b46d983-dfe1-48b8-83b3-b514140bfcd9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a98b1e4431940ab8d1fc95df2858a6c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81767) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 16:11:16.527709 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b46d983-dfe1-48b8-83b3-b514140bfcd9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:11:16.527995 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b46d983-dfe1-48b8-83b3-b514140bfcd9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:11:16.528274 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b46d983-dfe1-48b8-83b3-b514140bfcd9 admin admin] PATH_INFO: `/v3/users/dbb8092b0c6d410e93a3f397e282157f` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:11:16.532197 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b46d983-dfe1-48b8-83b3-b514140bfcd9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=dbb8092b0c6d410e93a3f397e282157f)` {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 16:11:16.537745 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b46d983-dfe1-48b8-83b3-b514140bfcd9 admin admin] RBAC: Authorization granted {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 16:11:16.576375 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-2a2924ea-d73d-479b-ab15-b291ceea1bd9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'faba300378174db08541de5f62b55ebe'} {{(pid=81768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 16:11:16.578221 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-2a2924ea-d73d-479b-ab15-b291ceea1bd9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'faba300378174db08541de5f62b55ebe'} {{(pid=81768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 16:11:16.579160 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-2a2924ea-d73d-479b-ab15-b291ceea1bd9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'faba300378174db08541de5f62b55ebe'} {{(pid=81768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 16:11:16.594688 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-2a2924ea-d73d-479b-ab15-b291ceea1bd9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'faba300378174db08541de5f62b55ebe'} {{(pid=81768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 16:11:16.727341 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-2a2924ea-d73d-479b-ab15-b291ceea1bd9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'faba300378174db08541de5f62b55ebe'} {{(pid=81768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 16:11:16.752768 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 945/1865] 10.222.0.15 () {58 vars in 1224 bytes} [Mon Aug 22 16:11:16 2022] DELETE /identity/v3/users/faba300378174db08541de5f62b55ebe => generated 0 bytes in 557 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 16:11:16.753695 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a2924ea-d73d-479b-ab15-b291ceea1bd9 admin admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 16:11:16.756031 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-2a2924ea-d73d-479b-ab15-b291ceea1bd9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 16:11:16.833057 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.notifications [None req-7b46d983-dfe1-48b8-83b3-b514140bfcd9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'dbb8092b0c6d410e93a3f397e282157f'} {{(pid=81767) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 16:11:16.838906 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.notifications [None req-7b46d983-dfe1-48b8-83b3-b514140bfcd9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'dbb8092b0c6d410e93a3f397e282157f'} {{(pid=81767) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 16:11:16.846597 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a2924ea-d73d-479b-ab15-b291ceea1bd9 admin admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 16:11:16.851416 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-209e6984-5bbe-4fc6-93d9-c189c7dc8f40 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a98b1e4431940ab8d1fc95df2858a6c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 16:11:16.852058 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-209e6984-5bbe-4fc6-93d9-c189c7dc8f40 admin admin] REQUEST_METHOD: `POST` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:11:16.852342 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-209e6984-5bbe-4fc6-93d9-c189c7dc8f40 admin admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:11:16.852616 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-209e6984-5bbe-4fc6-93d9-c189c7dc8f40 admin admin] PATH_INFO: `/v3/projects` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:11:16.853244 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-209e6984-5bbe-4fc6-93d9-c189c7dc8f40 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 16:11:16.855155 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-209e6984-5bbe-4fc6-93d9-c189c7dc8f40 admin admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 16:11:16.937248 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.notifications [None req-7b46d983-dfe1-48b8-83b3-b514140bfcd9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'dbb8092b0c6d410e93a3f397e282157f'} {{(pid=81767) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 16:11:16.959188 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.notifications [None req-7b46d983-dfe1-48b8-83b3-b514140bfcd9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'dbb8092b0c6d410e93a3f397e282157f'} {{(pid=81767) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 16:11:16.962543 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.notifications [None req-7b46d983-dfe1-48b8-83b3-b514140bfcd9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'dbb8092b0c6d410e93a3f397e282157f'} {{(pid=81767) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 16:11:16.969502 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 921/1866] 10.222.0.15 () {58 vars in 1224 bytes} [Mon Aug 22 16:11:16 2022] DELETE /identity/v3/users/dbb8092b0c6d410e93a3f397e282157f => generated 0 bytes in 483 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 16:11:16.970944 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a387147-7079-483e-a28f-ab0e2855c872 None None] REQUEST_METHOD: `POST` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:11:16.971237 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a387147-7079-483e-a28f-ab0e2855c872 None None] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:11:16.971548 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a387147-7079-483e-a28f-ab0e2855c872 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:11:16.994026 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.auth.core [None req-8a387147-7079-483e-a28f-ab0e2855c872 None None] MFA Rules not processed for user `6b2199ae0fb0480aa917ada391f5ec54`. Rule list: `[]` (Enabled: `True`). {{(pid=81767) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 16:11:16.995220 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.fernet_utils [None req-8a387147-7079-483e-a28f-ab0e2855c872 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81767) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 16:11:17.096800 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 946/1867] 10.222.0.15 () {60 vars in 1153 bytes} [Mon Aug 22 16:11:16 2022] POST /identity/v3/projects => generated 369 bytes in 343 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 22 16:11:17.098362 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-778ecedb-284d-48cb-b002-8b7a9874b476 None None] REQUEST_METHOD: `POST` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:11:17.098813 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-778ecedb-284d-48cb-b002-8b7a9874b476 None None] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:11:17.099245 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-778ecedb-284d-48cb-b002-8b7a9874b476 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:11:17.101096 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 922/1868] 10.222.0.15 () {58 vars in 958 bytes} [Mon Aug 22 16:11:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 16:11:17.101994 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a387147-7079-483e-a28f-ab0e2855c872 None None] Authenticating user token {{(pid=81767) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 16:11:17.104625 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.fernet_utils [None req-8a387147-7079-483e-a28f-ab0e2855c872 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81767) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 16:11:17.128580 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a387147-7079-483e-a28f-ab0e2855c872 None None] Validating token access rules against request {{(pid=81767) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 16:11:17.135373 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.auth.core [None req-778ecedb-284d-48cb-b002-8b7a9874b476 None None] MFA Rules not processed for user `6b2199ae0fb0480aa917ada391f5ec54`. Rule list: `[]` (Enabled: `True`). {{(pid=81768) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 16:11:17.137061 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-778ecedb-284d-48cb-b002-8b7a9874b476 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 16:11:17.201395 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1be90c47-8a9e-4f20-aeaf-937ea6f234c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a98b1e4431940ab8d1fc95df2858a6c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81767) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 16:11:17.202129 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1be90c47-8a9e-4f20-aeaf-937ea6f234c5 admin admin] REQUEST_METHOD: `POST` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:11:17.202403 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1be90c47-8a9e-4f20-aeaf-937ea6f234c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:11:17.202717 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1be90c47-8a9e-4f20-aeaf-937ea6f234c5 admin admin] PATH_INFO: `/v3/users` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:11:17.204061 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1be90c47-8a9e-4f20-aeaf-937ea6f234c5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 16:11:17.206651 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1be90c47-8a9e-4f20-aeaf-937ea6f234c5 admin admin] RBAC: Authorization granted {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 16:11:17.251313 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 16:11:17.254342 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 947/1869] 10.222.0.15 () {58 vars in 958 bytes} [Mon Aug 22 16:11:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 16:11:17.254342 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-778ecedb-284d-48cb-b002-8b7a9874b476 None None] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 16:11:17.266108 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-778ecedb-284d-48cb-b002-8b7a9874b476 None None] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 16:11:17.271659 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fadde8d0-af20-42f4-949b-297aaf629e1a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a98b1e4431940ab8d1fc95df2858a6c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 16:11:17.272269 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fadde8d0-af20-42f4-949b-297aaf629e1a admin admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:11:17.272543 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fadde8d0-af20-42f4-949b-297aaf629e1a admin admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:11:17.272844 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fadde8d0-af20-42f4-949b-297aaf629e1a admin admin] PATH_INFO: `/v3/domains` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:11:17.273496 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fadde8d0-af20-42f4-949b-297aaf629e1a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81768) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 16:11:17.273786 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fadde8d0-af20-42f4-949b-297aaf629e1a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 16:11:17.275877 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fadde8d0-af20-42f4-949b-297aaf629e1a admin admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 16:11:17.288937 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 948/1870] 10.222.0.15 () {58 vars in 1153 bytes} [Mon Aug 22 16:11:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 16:11:17.289771 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fadde8d0-af20-42f4-949b-297aaf629e1a admin admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 16:11:17.291648 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-fadde8d0-af20-42f4-949b-297aaf629e1a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 16:11:17.348707 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 923/1871] 10.222.0.15 () {60 vars in 1144 bytes} [Mon Aug 22 16:11:17 2022] POST /identity/v3/users => generated 291 bytes in 247 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 22 16:11:17.349741 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1be90c47-8a9e-4f20-aeaf-937ea6f234c5 admin admin] Authenticating user token {{(pid=81767) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 16:11:17.357237 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fadde8d0-af20-42f4-949b-297aaf629e1a admin admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 16:11:17.361058 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8b6df3d-aa8c-453d-996b-f285c359f4ef service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '234687adad594f1aa45775a565ffbdc7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4ee0413f9ef4d4fb1f95a660babe090', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 16:11:17.361742 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8b6df3d-aa8c-453d-996b-f285c359f4ef service neutron] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:11:17.362094 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8b6df3d-aa8c-453d-996b-f285c359f4ef service neutron] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:11:17.362669 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8b6df3d-aa8c-453d-996b-f285c359f4ef service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:11:17.366351 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1be90c47-8a9e-4f20-aeaf-937ea6f234c5 admin admin] Validating token access rules against request {{(pid=81767) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 16:11:17.372392 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8b6df3d-aa8c-453d-996b-f285c359f4ef service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 16:11:17.381905 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-088220a4-7c14-4080-bf2d-bd1ac111cb71 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a98b1e4431940ab8d1fc95df2858a6c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81767) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 16:11:17.382286 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8b6df3d-aa8c-453d-996b-f285c359f4ef service neutron] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 16:11:17.382378 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-088220a4-7c14-4080-bf2d-bd1ac111cb71 admin admin] REQUEST_METHOD: `POST` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:11:17.382378 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-088220a4-7c14-4080-bf2d-bd1ac111cb71 admin admin] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:11:17.382378 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-088220a4-7c14-4080-bf2d-bd1ac111cb71 admin admin] PATH_INFO: `/v3/projects` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:11:17.382378 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-088220a4-7c14-4080-bf2d-bd1ac111cb71 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 16:11:17.382378 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-088220a4-7c14-4080-bf2d-bd1ac111cb71 admin admin] RBAC: Authorization granted {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 16:11:17.385184 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 949/1872] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Aug 22 16:11:17 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 16:11:17.386044 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8b6df3d-aa8c-453d-996b-f285c359f4ef service neutron] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 16:11:17.395989 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8b6df3d-aa8c-453d-996b-f285c359f4ef service neutron] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 16:11:17.400829 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f02ae836-d756-4ff9-861d-46a9e0f03929 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a98b1e4431940ab8d1fc95df2858a6c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 16:11:17.401408 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f02ae836-d756-4ff9-861d-46a9e0f03929 admin admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:11:17.401685 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f02ae836-d756-4ff9-861d-46a9e0f03929 admin admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:11:17.401957 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f02ae836-d756-4ff9-861d-46a9e0f03929 admin admin] PATH_INFO: `/v3/roles` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:11:17.402601 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f02ae836-d756-4ff9-861d-46a9e0f03929 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 16:11:17.404236 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f02ae836-d756-4ff9-861d-46a9e0f03929 admin admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 16:11:17.419082 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 950/1873] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Aug 22 16:11:17 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 22 16:11:17.419082 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f02ae836-d756-4ff9-861d-46a9e0f03929 admin admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 16:11:17.419082 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-f02ae836-d756-4ff9-861d-46a9e0f03929 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 16:11:17.438068 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f02ae836-d756-4ff9-861d-46a9e0f03929 admin admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 16:11:17.446293 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-251794c2-079c-4d97-bd2e-29c13e293916 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a98b1e4431940ab8d1fc95df2858a6c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 16:11:17.446293 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-251794c2-079c-4d97-bd2e-29c13e293916 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:11:17.446293 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-251794c2-079c-4d97-bd2e-29c13e293916 admin admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:11:17.446293 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-251794c2-079c-4d97-bd2e-29c13e293916 admin admin] PATH_INFO: `/v3/projects/efecd6320bd24fdf97f753c0230d3b5c` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:11:17.446638 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-251794c2-079c-4d97-bd2e-29c13e293916 admin admin] RBAC: Authorizing `identity:delete_project(project_id=efecd6320bd24fdf97f753c0230d3b5c)` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 16:11:17.448346 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-251794c2-079c-4d97-bd2e-29c13e293916 admin admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 16:11:17.512002 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 924/1874] 10.222.0.15 () {60 vars in 1153 bytes} [Mon Aug 22 16:11:17 2022] POST /identity/v3/projects => generated 371 bytes in 163 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 22 16:11:17.512819 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-088220a4-7c14-4080-bf2d-bd1ac111cb71 admin admin] Authenticating user token {{(pid=81767) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 16:11:17.579356 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-088220a4-7c14-4080-bf2d-bd1ac111cb71 admin admin] Validating token access rules against request {{(pid=81767) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 16:11:17.585206 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6075dab-db08-4216-bcf9-be729a8424eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a98b1e4431940ab8d1fc95df2858a6c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81767) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 16:11:17.585897 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6075dab-db08-4216-bcf9-be729a8424eb admin admin] REQUEST_METHOD: `PUT` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:11:17.586236 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6075dab-db08-4216-bcf9-be729a8424eb admin admin] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:11:17.586459 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6075dab-db08-4216-bcf9-be729a8424eb admin admin] PATH_INFO: `/v3/projects/aa9e894c11ae4185b56665c499399ea6/users/aacec4652582498185152be8d8ecb78e/roles/9d9848cfd22e4b089db443c350e56670` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:11:17.624702 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-251794c2-079c-4d97-bd2e-29c13e293916 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'efecd6320bd24fdf97f753c0230d3b5c', 'request_id': 'req-251794c2-079c-4d97-bd2e-29c13e293916'} {{(pid=81768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 16:11:17.628640 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6075dab-db08-4216-bcf9-be729a8424eb admin admin] RBAC: Authorizing `identity:create_grant(project_id=aa9e894c11ae4185b56665c499399ea6, user_id=aacec4652582498185152be8d8ecb78e, role_id=9d9848cfd22e4b089db443c350e56670)` {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 16:11:17.632466 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6075dab-db08-4216-bcf9-be729a8424eb admin admin] RBAC: Authorization granted {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 16:11:17.641173 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-251794c2-079c-4d97-bd2e-29c13e293916 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'efecd6320bd24fdf97f753c0230d3b5c', 'request_id': 'req-251794c2-079c-4d97-bd2e-29c13e293916'} {{(pid=81768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 16:11:17.742631 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-251794c2-079c-4d97-bd2e-29c13e293916 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'efecd6320bd24fdf97f753c0230d3b5c', 'request_id': 'req-251794c2-079c-4d97-bd2e-29c13e293916'} {{(pid=81768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 16:11:17.753981 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 925/1875] 10.222.0.15 () {60 vars in 1483 bytes} [Mon Aug 22 16:11:17 2022] PUT /identity/v3/projects/aa9e894c11ae4185b56665c499399ea6/users/aacec4652582498185152be8d8ecb78e/roles/9d9848cfd22e4b089db443c350e56670 => generated 0 bytes in 241 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 16:11:17.756501 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6075dab-db08-4216-bcf9-be729a8424eb admin admin] Authenticating user token {{(pid=81767) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 16:11:17.776632 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-251794c2-079c-4d97-bd2e-29c13e293916 admin admin] The token cache is being invalidate because project efecd6320bd24fdf97f753c0230d3b5c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81768) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 22 16:11:17.776948 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-251794c2-079c-4d97-bd2e-29c13e293916 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 16:11:17.782945 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-251794c2-079c-4d97-bd2e-29c13e293916 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 16:11:17.792054 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 951/1876] 10.222.0.15 () {58 vars in 1233 bytes} [Mon Aug 22 16:11:17 2022] DELETE /identity/v3/projects/efecd6320bd24fdf97f753c0230d3b5c => generated 0 bytes in 376 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 16:11:17.795921 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-251794c2-079c-4d97-bd2e-29c13e293916 admin admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 16:11:17.800068 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-251794c2-079c-4d97-bd2e-29c13e293916 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 16:11:17.910003 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-251794c2-079c-4d97-bd2e-29c13e293916 admin admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 16:11:17.978922 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6075dab-db08-4216-bcf9-be729a8424eb admin admin] Validating token access rules against request {{(pid=81767) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 16:11:17.988973 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2be8a36-3272-402c-a91f-b784ae9a5cbf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a98b1e4431940ab8d1fc95df2858a6c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81767) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 16:11:17.991850 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2be8a36-3272-402c-a91f-b784ae9a5cbf admin admin] REQUEST_METHOD: `POST` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:11:17.992251 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2be8a36-3272-402c-a91f-b784ae9a5cbf admin admin] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:11:17.992554 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2be8a36-3272-402c-a91f-b784ae9a5cbf admin admin] PATH_INFO: `/v3/users` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:11:17.993281 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2be8a36-3272-402c-a91f-b784ae9a5cbf admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 16:11:17.997038 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2be8a36-3272-402c-a91f-b784ae9a5cbf admin admin] RBAC: Authorization granted {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 16:11:18.052724 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b51a0027-d330-435f-90d1-90eccbea3074 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a98b1e4431940ab8d1fc95df2858a6c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 16:11:18.054778 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b51a0027-d330-435f-90d1-90eccbea3074 admin admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:11:18.054778 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b51a0027-d330-435f-90d1-90eccbea3074 admin admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:11:18.054778 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b51a0027-d330-435f-90d1-90eccbea3074 admin admin] PATH_INFO: `/v3/roles` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:11:18.055554 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b51a0027-d330-435f-90d1-90eccbea3074 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 16:11:18.058646 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b51a0027-d330-435f-90d1-90eccbea3074 admin admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 16:11:18.073952 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 952/1877] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Aug 22 16:11:17 2022] GET /identity/v3/roles => generated 1420 bytes in 278 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 22 16:11:18.076123 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6745e44-da5b-4320-bb54-c0b94bbf2d87 None None] REQUEST_METHOD: `POST` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:11:18.076539 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6745e44-da5b-4320-bb54-c0b94bbf2d87 None None] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:11:18.077141 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6745e44-da5b-4320-bb54-c0b94bbf2d87 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:11:18.101473 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.auth.core [None req-f6745e44-da5b-4320-bb54-c0b94bbf2d87 None None] MFA Rules not processed for user `6b2199ae0fb0480aa917ada391f5ec54`. Rule list: `[]` (Enabled: `True`). {{(pid=81768) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 16:11:18.102748 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-f6745e44-da5b-4320-bb54-c0b94bbf2d87 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 16:11:18.113257 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 953/1878] 10.222.0.15 () {58 vars in 958 bytes} [Mon Aug 22 16:11:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 16:11:18.120423 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6745e44-da5b-4320-bb54-c0b94bbf2d87 None None] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 16:11:18.122107 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-f6745e44-da5b-4320-bb54-c0b94bbf2d87 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 16:11:18.135131 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6745e44-da5b-4320-bb54-c0b94bbf2d87 None None] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 16:11:18.140274 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeab1832-0c63-42f8-8078-bb05fea1368a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a98b1e4431940ab8d1fc95df2858a6c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 16:11:18.141030 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeab1832-0c63-42f8-8078-bb05fea1368a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:11:18.141371 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeab1832-0c63-42f8-8078-bb05fea1368a admin admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:11:18.141714 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeab1832-0c63-42f8-8078-bb05fea1368a admin admin] PATH_INFO: `/v3/projects/44f75b12ecb24afe8ede2f93c9642ce3` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:11:18.142990 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeab1832-0c63-42f8-8078-bb05fea1368a admin admin] RBAC: Authorizing `identity:delete_project(project_id=44f75b12ecb24afe8ede2f93c9642ce3)` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 16:11:18.146118 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeab1832-0c63-42f8-8078-bb05fea1368a admin admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 16:11:18.236674 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 926/1879] 10.222.0.15 () {60 vars in 1144 bytes} [Mon Aug 22 16:11:17 2022] POST /identity/v3/users => generated 286 bytes in 477 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 22 16:11:18.236674 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2be8a36-3272-402c-a91f-b784ae9a5cbf admin admin] Authenticating user token {{(pid=81767) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 16:11:18.248272 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2be8a36-3272-402c-a91f-b784ae9a5cbf admin admin] Validating token access rules against request {{(pid=81767) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 16:11:18.254806 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a485aa2d-7d66-43f0-9dec-a0b400cbb86e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a98b1e4431940ab8d1fc95df2858a6c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81767) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 16:11:18.255608 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a485aa2d-7d66-43f0-9dec-a0b400cbb86e admin admin] REQUEST_METHOD: `PUT` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:11:18.255890 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a485aa2d-7d66-43f0-9dec-a0b400cbb86e admin admin] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:11:18.256330 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a485aa2d-7d66-43f0-9dec-a0b400cbb86e admin admin] PATH_INFO: `/v3/projects/aa9e894c11ae4185b56665c499399ea6/users/aacec4652582498185152be8d8ecb78e/roles/9d9848cfd22e4b089db443c350e56670` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:11:18.258704 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a485aa2d-7d66-43f0-9dec-a0b400cbb86e admin admin] RBAC: Authorizing `identity:create_grant(project_id=aa9e894c11ae4185b56665c499399ea6, user_id=aacec4652582498185152be8d8ecb78e, role_id=9d9848cfd22e4b089db443c350e56670)` {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 16:11:18.262515 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a485aa2d-7d66-43f0-9dec-a0b400cbb86e admin admin] RBAC: Authorization granted {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 16:11:18.277887 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 927/1880] 10.222.0.15 () {60 vars in 1483 bytes} [Mon Aug 22 16:11:18 2022] PUT /identity/v3/projects/aa9e894c11ae4185b56665c499399ea6/users/aacec4652582498185152be8d8ecb78e/roles/9d9848cfd22e4b089db443c350e56670 => generated 0 bytes in 46 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 16:11:18.278929 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95244ce5-6abd-4b7e-9fce-7b8dc27d45ec None None] REQUEST_METHOD: `POST` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:11:18.279217 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95244ce5-6abd-4b7e-9fce-7b8dc27d45ec None None] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:11:18.279505 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95244ce5-6abd-4b7e-9fce-7b8dc27d45ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:11:18.301136 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.auth.core [None req-95244ce5-6abd-4b7e-9fce-7b8dc27d45ec None None] MFA Rules not processed for user `6b2199ae0fb0480aa917ada391f5ec54`. Rule list: `[]` (Enabled: `True`). {{(pid=81767) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 16:11:18.302813 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.fernet_utils [None req-95244ce5-6abd-4b7e-9fce-7b8dc27d45ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81767) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 16:11:18.372445 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG dbcounter [-] [81767] Writing DB stats keystone:DELETE=2,keystone:SELECT=97,keystone:INSERT=1 {{(pid=81767) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 16:11:18.396366 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-eeab1832-0c63-42f8-8078-bb05fea1368a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '44f75b12ecb24afe8ede2f93c9642ce3', 'request_id': 'req-eeab1832-0c63-42f8-8078-bb05fea1368a'} {{(pid=81768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 16:11:18.406981 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-eeab1832-0c63-42f8-8078-bb05fea1368a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '44f75b12ecb24afe8ede2f93c9642ce3', 'request_id': 'req-eeab1832-0c63-42f8-8078-bb05fea1368a'} {{(pid=81768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 16:11:18.440886 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 928/1881] 10.222.0.15 () {58 vars in 958 bytes} [Mon Aug 22 16:11:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 16:11:18.440886 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95244ce5-6abd-4b7e-9fce-7b8dc27d45ec None None] Authenticating user token {{(pid=81767) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 16:11:18.443906 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.fernet_utils [None req-95244ce5-6abd-4b7e-9fce-7b8dc27d45ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81767) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 16:11:18.471371 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95244ce5-6abd-4b7e-9fce-7b8dc27d45ec None None] Validating token access rules against request {{(pid=81767) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 16:11:18.496456 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-eeab1832-0c63-42f8-8078-bb05fea1368a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '44f75b12ecb24afe8ede2f93c9642ce3', 'request_id': 'req-eeab1832-0c63-42f8-8078-bb05fea1368a'} {{(pid=81768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 16:11:18.512910 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-eeab1832-0c63-42f8-8078-bb05fea1368a admin admin] The token cache is being invalidate because project 44f75b12ecb24afe8ede2f93c9642ce3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81768) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 22 16:11:18.513608 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-eeab1832-0c63-42f8-8078-bb05fea1368a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 16:11:18.515016 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-eeab1832-0c63-42f8-8078-bb05fea1368a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 16:11:18.521922 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 954/1882] 10.222.0.15 () {58 vars in 1233 bytes} [Mon Aug 22 16:11:18 2022] DELETE /identity/v3/projects/44f75b12ecb24afe8ede2f93c9642ce3 => generated 0 bytes in 408 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 16:11:18.523307 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeab1832-0c63-42f8-8078-bb05fea1368a admin admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 16:11:18.525534 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-eeab1832-0c63-42f8-8078-bb05fea1368a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 16:11:18.597745 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20db024e-f3d3-4924-8f6d-2db28e40c678 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a98b1e4431940ab8d1fc95df2858a6c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81767) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 16:11:18.598931 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20db024e-f3d3-4924-8f6d-2db28e40c678 admin admin] REQUEST_METHOD: `GET` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:11:18.599214 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20db024e-f3d3-4924-8f6d-2db28e40c678 admin admin] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:11:18.599510 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20db024e-f3d3-4924-8f6d-2db28e40c678 admin admin] PATH_INFO: `/v3/roles` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:11:18.600163 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20db024e-f3d3-4924-8f6d-2db28e40c678 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 16:11:18.601793 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20db024e-f3d3-4924-8f6d-2db28e40c678 admin admin] RBAC: Authorization granted {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 16:11:18.616264 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 16:11:18.628688 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 929/1883] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Aug 22 16:11:18 2022] GET /identity/v3/roles => generated 1420 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 22 16:11:18.628688 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06813190-d68e-4586-bfb1-8223ce35b2c6 None None] REQUEST_METHOD: `POST` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:11:18.628688 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06813190-d68e-4586-bfb1-8223ce35b2c6 None None] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:11:18.628688 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06813190-d68e-4586-bfb1-8223ce35b2c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:11:18.664033 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeab1832-0c63-42f8-8078-bb05fea1368a admin admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 16:11:18.669381 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d2cda32-ac19-4d4a-8879-00b2d9d884a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a98b1e4431940ab8d1fc95df2858a6c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 16:11:18.670253 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d2cda32-ac19-4d4a-8879-00b2d9d884a2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:11:18.670683 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d2cda32-ac19-4d4a-8879-00b2d9d884a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:11:18.671386 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d2cda32-ac19-4d4a-8879-00b2d9d884a2 admin admin] PATH_INFO: `/v3/users/a9f3880cbd9e4918ac9deafa18911cb0` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:11:18.679234 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d2cda32-ac19-4d4a-8879-00b2d9d884a2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a9f3880cbd9e4918ac9deafa18911cb0)` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 16:11:18.679234 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d2cda32-ac19-4d4a-8879-00b2d9d884a2 admin admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 16:11:18.679492 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.auth.core [None req-06813190-d68e-4586-bfb1-8223ce35b2c6 None None] MFA Rules not processed for user `aacec4652582498185152be8d8ecb78e`. Rule list: `[]` (Enabled: `True`). {{(pid=81767) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 16:11:18.679492 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.fernet_utils [None req-06813190-d68e-4586-bfb1-8223ce35b2c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81767) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 16:11:18.789405 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 930/1884] 10.222.0.15 () {58 vars in 958 bytes} [Mon Aug 22 16:11:18 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 16:11:18.790275 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06813190-d68e-4586-bfb1-8223ce35b2c6 None None] Authenticating user token {{(pid=81767) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 16:11:18.791931 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.fernet_utils [None req-06813190-d68e-4586-bfb1-8223ce35b2c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81767) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 16:11:18.805905 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06813190-d68e-4586-bfb1-8223ce35b2c6 None None] Validating token access rules against request {{(pid=81767) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 16:11:18.811758 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6433864e-ee41-453f-8774-14e98a663f18 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a98b1e4431940ab8d1fc95df2858a6c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81767) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 16:11:18.812479 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6433864e-ee41-453f-8774-14e98a663f18 admin admin] REQUEST_METHOD: `GET` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:11:18.812824 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6433864e-ee41-453f-8774-14e98a663f18 admin admin] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:11:18.813167 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6433864e-ee41-453f-8774-14e98a663f18 admin admin] PATH_INFO: `/v3/domains` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:11:18.813884 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6433864e-ee41-453f-8774-14e98a663f18 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81767) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 16:11:18.814252 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6433864e-ee41-453f-8774-14e98a663f18 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 16:11:18.815940 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6433864e-ee41-453f-8774-14e98a663f18 admin admin] RBAC: Authorization granted {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 16:11:18.828200 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 931/1885] 10.222.0.15 () {58 vars in 1153 bytes} [Mon Aug 22 16:11:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 16:11:18.829013 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6433864e-ee41-453f-8774-14e98a663f18 admin admin] Authenticating user token {{(pid=81767) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 16:11:18.830629 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.fernet_utils [None req-6433864e-ee41-453f-8774-14e98a663f18 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81767) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 16:11:18.844789 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6433864e-ee41-453f-8774-14e98a663f18 admin admin] Validating token access rules against request {{(pid=81767) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 16:11:18.850326 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a29f2496-aa80-46dd-b817-f95dc3f2f560 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a98b1e4431940ab8d1fc95df2858a6c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81767) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 16:11:18.850991 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a29f2496-aa80-46dd-b817-f95dc3f2f560 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:11:18.851320 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a29f2496-aa80-46dd-b817-f95dc3f2f560 admin admin] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:11:18.851732 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a29f2496-aa80-46dd-b817-f95dc3f2f560 admin admin] PATH_INFO: `/v3/users/0ba0c0c596b345dbbb599ea9ba746ad2` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:11:18.852907 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a29f2496-aa80-46dd-b817-f95dc3f2f560 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0ba0c0c596b345dbbb599ea9ba746ad2)` {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 16:11:18.854903 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a29f2496-aa80-46dd-b817-f95dc3f2f560 admin admin] RBAC: Authorization granted {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 16:11:18.953318 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-7d2cda32-ac19-4d4a-8879-00b2d9d884a2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a9f3880cbd9e4918ac9deafa18911cb0'} {{(pid=81768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 16:11:18.953318 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-7d2cda32-ac19-4d4a-8879-00b2d9d884a2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a9f3880cbd9e4918ac9deafa18911cb0'} {{(pid=81768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 16:11:18.953318 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-7d2cda32-ac19-4d4a-8879-00b2d9d884a2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a9f3880cbd9e4918ac9deafa18911cb0'} {{(pid=81768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 16:11:18.970498 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-7d2cda32-ac19-4d4a-8879-00b2d9d884a2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a9f3880cbd9e4918ac9deafa18911cb0'} {{(pid=81768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 16:11:19.085694 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-7d2cda32-ac19-4d4a-8879-00b2d9d884a2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a9f3880cbd9e4918ac9deafa18911cb0'} {{(pid=81768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 16:11:19.108598 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.notifications [None req-a29f2496-aa80-46dd-b817-f95dc3f2f560 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0ba0c0c596b345dbbb599ea9ba746ad2'} {{(pid=81767) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 16:11:19.109921 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.notifications [None req-a29f2496-aa80-46dd-b817-f95dc3f2f560 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0ba0c0c596b345dbbb599ea9ba746ad2'} {{(pid=81767) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 16:11:19.116484 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 955/1886] 10.222.0.15 () {58 vars in 1224 bytes} [Mon Aug 22 16:11:18 2022] DELETE /identity/v3/users/a9f3880cbd9e4918ac9deafa18911cb0 => generated 0 bytes in 594 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 16:11:19.117809 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d2cda32-ac19-4d4a-8879-00b2d9d884a2 admin admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 16:11:19.119869 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-7d2cda32-ac19-4d4a-8879-00b2d9d884a2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 16:11:19.190444 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d2cda32-ac19-4d4a-8879-00b2d9d884a2 admin admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 16:11:19.195520 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41f6a3a7-f5fb-47c3-9377-d2ec8a2ae1c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a98b1e4431940ab8d1fc95df2858a6c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 16:11:19.196064 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41f6a3a7-f5fb-47c3-9377-d2ec8a2ae1c2 admin admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:11:19.196341 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41f6a3a7-f5fb-47c3-9377-d2ec8a2ae1c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:11:19.197098 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41f6a3a7-f5fb-47c3-9377-d2ec8a2ae1c2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:11:19.197751 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41f6a3a7-f5fb-47c3-9377-d2ec8a2ae1c2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 16:11:19.199388 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41f6a3a7-f5fb-47c3-9377-d2ec8a2ae1c2 admin admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 16:11:19.210326 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 956/1887] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Aug 22 16:11:19 2022] GET /identity/v3/roles => generated 1420 bytes in 93 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 22 16:11:19.211154 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41f6a3a7-f5fb-47c3-9377-d2ec8a2ae1c2 admin admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 16:11:19.212860 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-41f6a3a7-f5fb-47c3-9377-d2ec8a2ae1c2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 16:11:19.219919 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.notifications [None req-a29f2496-aa80-46dd-b817-f95dc3f2f560 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0ba0c0c596b345dbbb599ea9ba746ad2'} {{(pid=81767) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 16:11:19.229906 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.notifications [None req-a29f2496-aa80-46dd-b817-f95dc3f2f560 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0ba0c0c596b345dbbb599ea9ba746ad2'} {{(pid=81767) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 16:11:19.243956 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.notifications [None req-a29f2496-aa80-46dd-b817-f95dc3f2f560 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0ba0c0c596b345dbbb599ea9ba746ad2'} {{(pid=81767) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 16:11:19.252251 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 932/1888] 10.222.0.15 () {58 vars in 1224 bytes} [Mon Aug 22 16:11:18 2022] DELETE /identity/v3/users/0ba0c0c596b345dbbb599ea9ba746ad2 => generated 0 bytes in 423 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 16:11:19.253386 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a29f2496-aa80-46dd-b817-f95dc3f2f560 admin admin] Authenticating user token {{(pid=81767) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 16:11:19.267527 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.fernet_utils [None req-a29f2496-aa80-46dd-b817-f95dc3f2f560 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81767) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 16:11:19.277330 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41f6a3a7-f5fb-47c3-9377-d2ec8a2ae1c2 admin admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 16:11:19.343097 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20d59b51-679c-4b66-b5a7-6e9465dbbc78 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79706d480f864998b9f8594be407ceab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4ee0413f9ef4d4fb1f95a660babe090', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 16:11:19.344053 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20d59b51-679c-4b66-b5a7-6e9465dbbc78 service cinder] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:11:19.344493 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20d59b51-679c-4b66-b5a7-6e9465dbbc78 service cinder] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:11:19.344771 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20d59b51-679c-4b66-b5a7-6e9465dbbc78 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:11:19.347589 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-20d59b51-679c-4b66-b5a7-6e9465dbbc78 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 16:11:19.427479 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20d59b51-679c-4b66-b5a7-6e9465dbbc78 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 16:11:19.429051 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a29f2496-aa80-46dd-b817-f95dc3f2f560 admin admin] Validating token access rules against request {{(pid=81767) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 16:11:19.429208 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20d59b51-679c-4b66-b5a7-6e9465dbbc78 service cinder] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 16:11:19.440484 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 957/1889] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Aug 22 16:11:19 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 229 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 16:11:19.441369 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e1624d4-fedb-4203-ab4a-b1c762098a29 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a98b1e4431940ab8d1fc95df2858a6c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81767) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 16:11:19.446533 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db8e9ecf-8bc1-4225-8d5c-903249d016e7 None None] REQUEST_METHOD: `POST` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:11:19.446533 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db8e9ecf-8bc1-4225-8d5c-903249d016e7 None None] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:11:19.446533 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db8e9ecf-8bc1-4225-8d5c-903249d016e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:11:19.446882 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e1624d4-fedb-4203-ab4a-b1c762098a29 admin admin] REQUEST_METHOD: `POST` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:11:19.446882 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e1624d4-fedb-4203-ab4a-b1c762098a29 admin admin] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:11:19.446882 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e1624d4-fedb-4203-ab4a-b1c762098a29 admin admin] PATH_INFO: `/v3/projects` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:11:19.446882 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e1624d4-fedb-4203-ab4a-b1c762098a29 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 16:11:19.448640 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e1624d4-fedb-4203-ab4a-b1c762098a29 admin admin] RBAC: Authorization granted {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 16:11:19.472008 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.auth.core [None req-db8e9ecf-8bc1-4225-8d5c-903249d016e7 None None] MFA Rules not processed for user `6b2199ae0fb0480aa917ada391f5ec54`. Rule list: `[]` (Enabled: `True`). {{(pid=81768) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 16:11:19.474182 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-db8e9ecf-8bc1-4225-8d5c-903249d016e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 16:11:19.483409 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 958/1890] 10.222.0.15 () {58 vars in 958 bytes} [Mon Aug 22 16:11:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 16:11:19.484224 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db8e9ecf-8bc1-4225-8d5c-903249d016e7 None None] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 16:11:19.485831 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-db8e9ecf-8bc1-4225-8d5c-903249d016e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 16:11:19.499847 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db8e9ecf-8bc1-4225-8d5c-903249d016e7 None None] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 16:11:19.506008 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8ada5a1-82c6-484f-86bd-623334848322 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a98b1e4431940ab8d1fc95df2858a6c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 16:11:19.506691 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8ada5a1-82c6-484f-86bd-623334848322 admin admin] REQUEST_METHOD: `PUT` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:11:19.506965 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8ada5a1-82c6-484f-86bd-623334848322 admin admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:11:19.507240 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8ada5a1-82c6-484f-86bd-623334848322 admin admin] PATH_INFO: `/v3/projects/9752eece61444130b0b869263cc92305/users/514a45348e12415abaee3d2e29406689/roles/e6fc1ee3fb974125ba9b4fe55b12b45e` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:11:19.524360 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8ada5a1-82c6-484f-86bd-623334848322 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9752eece61444130b0b869263cc92305, user_id=514a45348e12415abaee3d2e29406689, role_id=e6fc1ee3fb974125ba9b4fe55b12b45e)` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 16:11:19.528124 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8ada5a1-82c6-484f-86bd-623334848322 admin admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 16:11:19.610234 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 933/1891] 10.222.0.15 () {60 vars in 1153 bytes} [Mon Aug 22 16:11:19 2022] POST /identity/v3/projects => generated 381 bytes in 357 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 22 16:11:19.612053 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e1624d4-fedb-4203-ab4a-b1c762098a29 admin admin] Authenticating user token {{(pid=81767) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 16:11:19.614121 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.fernet_utils [None req-0e1624d4-fedb-4203-ab4a-b1c762098a29 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81767) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 16:11:19.646265 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 959/1892] 10.222.0.15 () {60 vars in 1483 bytes} [Mon Aug 22 16:11:19 2022] PUT /identity/v3/projects/9752eece61444130b0b869263cc92305/users/514a45348e12415abaee3d2e29406689/roles/e6fc1ee3fb974125ba9b4fe55b12b45e => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 16:11:19.647451 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8ada5a1-82c6-484f-86bd-623334848322 admin admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 16:11:19.747460 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e1624d4-fedb-4203-ab4a-b1c762098a29 admin admin] Validating token access rules against request {{(pid=81767) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 16:11:19.862077 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8ada5a1-82c6-484f-86bd-623334848322 admin admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 16:11:19.873920 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aad104c-e0ab-4c6b-8228-9b20ca8d943a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a98b1e4431940ab8d1fc95df2858a6c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 16:11:19.874534 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aad104c-e0ab-4c6b-8228-9b20ca8d943a admin admin] REQUEST_METHOD: `POST` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:11:19.875033 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aad104c-e0ab-4c6b-8228-9b20ca8d943a admin admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:11:19.875363 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aad104c-e0ab-4c6b-8228-9b20ca8d943a admin admin] PATH_INFO: `/v3/users` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:11:19.876849 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aad104c-e0ab-4c6b-8228-9b20ca8d943a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 16:11:19.882610 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aad104c-e0ab-4c6b-8228-9b20ca8d943a admin admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 16:11:19.886266 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0a6b7c3-954a-427a-89cb-6dab1a19a20b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '234687adad594f1aa45775a565ffbdc7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4ee0413f9ef4d4fb1f95a660babe090', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81767) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 16:11:19.887261 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a6b7c3-954a-427a-89cb-6dab1a19a20b service neutron] REQUEST_METHOD: `GET` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:11:19.887913 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a6b7c3-954a-427a-89cb-6dab1a19a20b service neutron] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:11:19.888391 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a6b7c3-954a-427a-89cb-6dab1a19a20b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:11:19.909624 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0a6b7c3-954a-427a-89cb-6dab1a19a20b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 16:11:19.911998 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0a6b7c3-954a-427a-89cb-6dab1a19a20b service neutron] RBAC: Authorization granted {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 16:11:19.939397 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 934/1893] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Aug 22 16:11:19 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 328 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 16:11:19.941116 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0a6b7c3-954a-427a-89cb-6dab1a19a20b service neutron] Authenticating user token {{(pid=81767) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 16:11:19.943102 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.fernet_utils [None req-c0a6b7c3-954a-427a-89cb-6dab1a19a20b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81767) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 16:11:20.005642 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 960/1894] 10.222.0.15 () {60 vars in 1144 bytes} [Mon Aug 22 16:11:19 2022] POST /identity/v3/users => generated 291 bytes in 359 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 22 16:11:20.007389 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c66cca92-a7d5-4cfe-903e-7133cb13d0c0 None None] REQUEST_METHOD: `POST` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:11:20.007681 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c66cca92-a7d5-4cfe-903e-7133cb13d0c0 None None] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:11:20.008102 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c66cca92-a7d5-4cfe-903e-7133cb13d0c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:11:20.021551 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 16:11:20.068143 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0a6b7c3-954a-427a-89cb-6dab1a19a20b service neutron] Validating token access rules against request {{(pid=81767) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 16:11:20.077026 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-186ef4be-5143-4df2-8bfb-a4bb1e3dc7e9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79706d480f864998b9f8594be407ceab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4ee0413f9ef4d4fb1f95a660babe090', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81767) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 16:11:20.077659 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-186ef4be-5143-4df2-8bfb-a4bb1e3dc7e9 service cinder] REQUEST_METHOD: `GET` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:11:20.078627 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-186ef4be-5143-4df2-8bfb-a4bb1e3dc7e9 service cinder] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:11:20.078908 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-186ef4be-5143-4df2-8bfb-a4bb1e3dc7e9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:11:20.084408 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.fernet_utils [None req-186ef4be-5143-4df2-8bfb-a4bb1e3dc7e9 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81767) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 16:11:20.097450 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.auth.core [None req-c66cca92-a7d5-4cfe-903e-7133cb13d0c0 None None] MFA Rules not processed for user `514a45348e12415abaee3d2e29406689`. Rule list: `[]` (Enabled: `True`). {{(pid=81768) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 16:11:20.104030 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-c66cca92-a7d5-4cfe-903e-7133cb13d0c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 16:11:20.202087 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-186ef4be-5143-4df2-8bfb-a4bb1e3dc7e9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 16:11:20.203745 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-186ef4be-5143-4df2-8bfb-a4bb1e3dc7e9 service cinder] RBAC: Authorization granted {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 16:11:20.219952 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 935/1895] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Aug 22 16:11:19 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 279 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 16:11:20.220979 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-186ef4be-5143-4df2-8bfb-a4bb1e3dc7e9 service cinder] Authenticating user token {{(pid=81767) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 16:11:20.224927 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 961/1896] 10.222.0.15 () {58 vars in 958 bytes} [Mon Aug 22 16:11:20 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 219 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 16:11:20.225785 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c66cca92-a7d5-4cfe-903e-7133cb13d0c0 None None] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 16:11:20.227191 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-c66cca92-a7d5-4cfe-903e-7133cb13d0c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 16:11:20.239556 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-186ef4be-5143-4df2-8bfb-a4bb1e3dc7e9 service cinder] Validating token access rules against request {{(pid=81767) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 16:11:20.244947 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5162149f-d4f4-4990-ba15-7fcd3f0ced3a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a98b1e4431940ab8d1fc95df2858a6c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81767) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 16:11:20.245691 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5162149f-d4f4-4990-ba15-7fcd3f0ced3a admin admin] REQUEST_METHOD: `GET` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:11:20.246022 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5162149f-d4f4-4990-ba15-7fcd3f0ced3a admin admin] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:11:20.246320 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5162149f-d4f4-4990-ba15-7fcd3f0ced3a admin admin] PATH_INFO: `/v3/roles` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:11:20.247019 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5162149f-d4f4-4990-ba15-7fcd3f0ced3a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 16:11:20.249163 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5162149f-d4f4-4990-ba15-7fcd3f0ced3a admin admin] RBAC: Authorization granted {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 16:11:20.269411 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 936/1897] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Aug 22 16:11:20 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 22 16:11:20.269411 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5162149f-d4f4-4990-ba15-7fcd3f0ced3a admin admin] Authenticating user token {{(pid=81767) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 16:11:20.276785 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c66cca92-a7d5-4cfe-903e-7133cb13d0c0 None None] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 16:11:20.282609 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5162149f-d4f4-4990-ba15-7fcd3f0ced3a admin admin] Validating token access rules against request {{(pid=81767) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 16:11:20.288643 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3a9276c-8ef6-45b2-a6db-5ac29092f77c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a98b1e4431940ab8d1fc95df2858a6c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81767) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 16:11:20.289483 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3a9276c-8ef6-45b2-a6db-5ac29092f77c admin admin] REQUEST_METHOD: `POST` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:11:20.290443 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3a9276c-8ef6-45b2-a6db-5ac29092f77c admin admin] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:11:20.290443 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3a9276c-8ef6-45b2-a6db-5ac29092f77c admin admin] PATH_INFO: `/v3/projects` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:11:20.290832 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3a9276c-8ef6-45b2-a6db-5ac29092f77c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 16:11:20.292795 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8201eee-c8bd-44d8-aa3b-0bcde44d5188 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a98b1e4431940ab8d1fc95df2858a6c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 16:11:20.293603 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8201eee-c8bd-44d8-aa3b-0bcde44d5188 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:11:20.297014 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8201eee-c8bd-44d8-aa3b-0bcde44d5188 admin admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:11:20.297366 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8201eee-c8bd-44d8-aa3b-0bcde44d5188 admin admin] PATH_INFO: `/v3/projects/c4b407f80c9b42acb2f9d51952895129` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:11:20.298044 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3a9276c-8ef6-45b2-a6db-5ac29092f77c admin admin] RBAC: Authorization granted {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 16:11:20.298695 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8201eee-c8bd-44d8-aa3b-0bcde44d5188 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c4b407f80c9b42acb2f9d51952895129)` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 16:11:20.305488 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8201eee-c8bd-44d8-aa3b-0bcde44d5188 admin admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 16:11:20.401699 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 937/1898] 10.222.0.15 () {60 vars in 1153 bytes} [Mon Aug 22 16:11:20 2022] POST /identity/v3/projects => generated 371 bytes in 136 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 22 16:11:20.402896 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3a9276c-8ef6-45b2-a6db-5ac29092f77c admin admin] Authenticating user token {{(pid=81767) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 16:11:20.497212 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-c8201eee-c8bd-44d8-aa3b-0bcde44d5188 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c4b407f80c9b42acb2f9d51952895129', 'request_id': 'req-c8201eee-c8bd-44d8-aa3b-0bcde44d5188'} {{(pid=81768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 16:11:20.513477 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-c8201eee-c8bd-44d8-aa3b-0bcde44d5188 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c4b407f80c9b42acb2f9d51952895129', 'request_id': 'req-c8201eee-c8bd-44d8-aa3b-0bcde44d5188'} {{(pid=81768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 16:11:20.534611 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3a9276c-8ef6-45b2-a6db-5ac29092f77c admin admin] Validating token access rules against request {{(pid=81767) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 16:11:20.601230 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG dbcounter [-] [81767] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81767) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 16:11:20.606164 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da7953d0-2232-4351-90b5-dcab5a799c78 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a98b1e4431940ab8d1fc95df2858a6c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81767) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 16:11:20.610026 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da7953d0-2232-4351-90b5-dcab5a799c78 admin admin] REQUEST_METHOD: `GET` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:11:20.610026 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da7953d0-2232-4351-90b5-dcab5a799c78 admin admin] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:11:20.610026 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da7953d0-2232-4351-90b5-dcab5a799c78 admin admin] PATH_INFO: `/v3/roles` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:11:20.610026 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da7953d0-2232-4351-90b5-dcab5a799c78 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 16:11:20.610890 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da7953d0-2232-4351-90b5-dcab5a799c78 admin admin] RBAC: Authorization granted {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 16:11:20.623071 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 938/1899] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Aug 22 16:11:20 2022] GET /identity/v3/roles => generated 1420 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 22 16:11:20.623406 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da7953d0-2232-4351-90b5-dcab5a799c78 admin admin] Authenticating user token {{(pid=81767) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 16:11:20.628954 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-c8201eee-c8bd-44d8-aa3b-0bcde44d5188 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c4b407f80c9b42acb2f9d51952895129', 'request_id': 'req-c8201eee-c8bd-44d8-aa3b-0bcde44d5188'} {{(pid=81768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 16:11:20.645485 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da7953d0-2232-4351-90b5-dcab5a799c78 admin admin] Validating token access rules against request {{(pid=81767) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 16:11:20.646887 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-c8201eee-c8bd-44d8-aa3b-0bcde44d5188 admin admin] The token cache is being invalidate because project c4b407f80c9b42acb2f9d51952895129 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81768) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 22 16:11:20.647180 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-c8201eee-c8bd-44d8-aa3b-0bcde44d5188 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 16:11:20.651605 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-c8201eee-c8bd-44d8-aa3b-0bcde44d5188 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 16:11:20.652008 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24f00aee-a50b-4c53-9873-a5d2f47c1655 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a98b1e4431940ab8d1fc95df2858a6c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81767) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 16:11:20.652834 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24f00aee-a50b-4c53-9873-a5d2f47c1655 admin admin] REQUEST_METHOD: `POST` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:11:20.654702 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24f00aee-a50b-4c53-9873-a5d2f47c1655 admin admin] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:11:20.655022 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24f00aee-a50b-4c53-9873-a5d2f47c1655 admin admin] PATH_INFO: `/v3/users` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:11:20.655582 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 962/1900] 10.222.0.15 () {58 vars in 1233 bytes} [Mon Aug 22 16:11:20 2022] DELETE /identity/v3/projects/c4b407f80c9b42acb2f9d51952895129 => generated 0 bytes in 428 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 16:11:20.658225 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24f00aee-a50b-4c53-9873-a5d2f47c1655 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 16:11:20.658341 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8201eee-c8bd-44d8-aa3b-0bcde44d5188 admin admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 16:11:20.658418 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24f00aee-a50b-4c53-9873-a5d2f47c1655 admin admin] RBAC: Authorization granted {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 16:11:20.660424 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-c8201eee-c8bd-44d8-aa3b-0bcde44d5188 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 16:11:20.678379 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8201eee-c8bd-44d8-aa3b-0bcde44d5188 admin admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 16:11:20.686897 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cfbd813-6cfe-449f-9136-5d22dd497514 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a98b1e4431940ab8d1fc95df2858a6c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 16:11:20.686897 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cfbd813-6cfe-449f-9136-5d22dd497514 admin admin] REQUEST_METHOD: `PUT` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:11:20.686897 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cfbd813-6cfe-449f-9136-5d22dd497514 admin admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:11:20.686897 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cfbd813-6cfe-449f-9136-5d22dd497514 admin admin] PATH_INFO: `/v3/projects/1cb989001cdd4a40b26c33a399448ee7/users/5c9035bfd8c4475fae8a4895804e94a8/roles/e6fc1ee3fb974125ba9b4fe55b12b45e` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:11:20.705712 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cfbd813-6cfe-449f-9136-5d22dd497514 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1cb989001cdd4a40b26c33a399448ee7, user_id=5c9035bfd8c4475fae8a4895804e94a8, role_id=e6fc1ee3fb974125ba9b4fe55b12b45e)` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 16:11:20.709307 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cfbd813-6cfe-449f-9136-5d22dd497514 admin admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 16:11:20.772708 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 939/1901] 10.222.0.15 () {60 vars in 1144 bytes} [Mon Aug 22 16:11:20 2022] POST /identity/v3/users => generated 292 bytes in 149 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 22 16:11:20.774824 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24f00aee-a50b-4c53-9873-a5d2f47c1655 admin admin] Authenticating user token {{(pid=81767) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 16:11:20.776864 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.fernet_utils [None req-24f00aee-a50b-4c53-9873-a5d2f47c1655 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81767) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 16:11:20.790091 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24f00aee-a50b-4c53-9873-a5d2f47c1655 admin admin] Validating token access rules against request {{(pid=81767) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 16:11:20.795722 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31f9ebcf-0b21-40b6-a5d8-4488647c027d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a98b1e4431940ab8d1fc95df2858a6c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81767) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 16:11:20.796385 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31f9ebcf-0b21-40b6-a5d8-4488647c027d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:11:20.796668 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31f9ebcf-0b21-40b6-a5d8-4488647c027d admin admin] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:11:20.796975 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31f9ebcf-0b21-40b6-a5d8-4488647c027d admin admin] PATH_INFO: `/v3/projects/d7dea9a491404d7d90e4811f75cfe30a` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:11:20.798227 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31f9ebcf-0b21-40b6-a5d8-4488647c027d admin admin] RBAC: Authorizing `identity:delete_project(project_id=d7dea9a491404d7d90e4811f75cfe30a)` {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 16:11:20.800448 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31f9ebcf-0b21-40b6-a5d8-4488647c027d admin admin] RBAC: Authorization granted {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 16:11:20.839821 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 963/1902] 10.222.0.15 () {60 vars in 1483 bytes} [Mon Aug 22 16:11:20 2022] PUT /identity/v3/projects/1cb989001cdd4a40b26c33a399448ee7/users/5c9035bfd8c4475fae8a4895804e94a8/roles/e6fc1ee3fb974125ba9b4fe55b12b45e => generated 0 bytes in 183 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 16:11:20.852876 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb72c7cd-87dc-45bc-a0fe-395577ccdc44 None None] REQUEST_METHOD: `POST` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:11:20.855096 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb72c7cd-87dc-45bc-a0fe-395577ccdc44 None None] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:11:20.855096 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb72c7cd-87dc-45bc-a0fe-395577ccdc44 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:11:20.893718 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.auth.core [None req-bb72c7cd-87dc-45bc-a0fe-395577ccdc44 None None] MFA Rules not processed for user `6b2199ae0fb0480aa917ada391f5ec54`. Rule list: `[]` (Enabled: `True`). {{(pid=81768) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 16:11:20.894920 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-bb72c7cd-87dc-45bc-a0fe-395577ccdc44 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 16:11:20.975898 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 964/1903] 10.222.0.15 () {58 vars in 958 bytes} [Mon Aug 22 16:11:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 16:11:20.984820 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb72c7cd-87dc-45bc-a0fe-395577ccdc44 None None] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 16:11:20.987353 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-bb72c7cd-87dc-45bc-a0fe-395577ccdc44 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 16:11:21.000589 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb72c7cd-87dc-45bc-a0fe-395577ccdc44 None None] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 16:11:21.006476 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79f2db0a-f8dd-4066-b259-f800253d2060 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a98b1e4431940ab8d1fc95df2858a6c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 16:11:21.007104 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79f2db0a-f8dd-4066-b259-f800253d2060 admin admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:11:21.007406 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79f2db0a-f8dd-4066-b259-f800253d2060 admin admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:11:21.007676 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79f2db0a-f8dd-4066-b259-f800253d2060 admin admin] PATH_INFO: `/v3/roles` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:11:21.008343 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79f2db0a-f8dd-4066-b259-f800253d2060 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 16:11:21.011127 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79f2db0a-f8dd-4066-b259-f800253d2060 admin admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 16:11:21.030121 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 965/1904] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Aug 22 16:11:20 2022] GET /identity/v3/roles => generated 1420 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 22 16:11:21.030911 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.notifications [None req-31f9ebcf-0b21-40b6-a5d8-4488647c027d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd7dea9a491404d7d90e4811f75cfe30a', 'request_id': 'req-31f9ebcf-0b21-40b6-a5d8-4488647c027d'} {{(pid=81767) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 16:11:21.034898 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb57937-eb46-45db-be32-19fad3754da3 None None] REQUEST_METHOD: `POST` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:11:21.035457 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb57937-eb46-45db-be32-19fad3754da3 None None] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:11:21.035864 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb57937-eb46-45db-be32-19fad3754da3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:11:21.038791 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.notifications [None req-31f9ebcf-0b21-40b6-a5d8-4488647c027d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd7dea9a491404d7d90e4811f75cfe30a', 'request_id': 'req-31f9ebcf-0b21-40b6-a5d8-4488647c027d'} {{(pid=81767) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 16:11:21.076148 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.auth.core [None req-bbb57937-eb46-45db-be32-19fad3754da3 None None] MFA Rules not processed for user `5c9035bfd8c4475fae8a4895804e94a8`. Rule list: `[]` (Enabled: `True`). {{(pid=81768) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 16:11:21.077552 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-bbb57937-eb46-45db-be32-19fad3754da3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 16:11:21.147830 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.notifications [None req-31f9ebcf-0b21-40b6-a5d8-4488647c027d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd7dea9a491404d7d90e4811f75cfe30a', 'request_id': 'req-31f9ebcf-0b21-40b6-a5d8-4488647c027d'} {{(pid=81767) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 16:11:21.177672 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.notifications [None req-31f9ebcf-0b21-40b6-a5d8-4488647c027d admin admin] The token cache is being invalidate because project d7dea9a491404d7d90e4811f75cfe30a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81767) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 22 16:11:21.177943 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.notifications [None req-31f9ebcf-0b21-40b6-a5d8-4488647c027d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81767) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 16:11:21.179413 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.notifications [None req-31f9ebcf-0b21-40b6-a5d8-4488647c027d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81767) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 16:11:21.185149 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 940/1905] 10.222.0.15 () {58 vars in 1233 bytes} [Mon Aug 22 16:11:20 2022] DELETE /identity/v3/projects/d7dea9a491404d7d90e4811f75cfe30a => generated 0 bytes in 409 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 16:11:21.186079 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31f9ebcf-0b21-40b6-a5d8-4488647c027d admin admin] Authenticating user token {{(pid=81767) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 16:11:21.190140 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.fernet_utils [None req-31f9ebcf-0b21-40b6-a5d8-4488647c027d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81767) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 16:11:21.191239 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 966/1906] 10.222.0.15 () {58 vars in 958 bytes} [Mon Aug 22 16:11:21 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 16:11:21.193085 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30190e5d-40a2-4218-a961-a278ac5fbf53 None None] REQUEST_METHOD: `POST` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:11:21.195945 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30190e5d-40a2-4218-a961-a278ac5fbf53 None None] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:11:21.196260 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30190e5d-40a2-4218-a961-a278ac5fbf53 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:11:21.237324 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.auth.core [None req-30190e5d-40a2-4218-a961-a278ac5fbf53 None None] MFA Rules not processed for user `6b2199ae0fb0480aa917ada391f5ec54`. Rule list: `[]` (Enabled: `True`). {{(pid=81768) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 16:11:21.239850 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-30190e5d-40a2-4218-a961-a278ac5fbf53 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 16:11:21.272417 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31f9ebcf-0b21-40b6-a5d8-4488647c027d admin admin] Validating token access rules against request {{(pid=81767) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 16:11:21.276106 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3752d06-1b38-46dd-8465-1e95541232f4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79706d480f864998b9f8594be407ceab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4ee0413f9ef4d4fb1f95a660babe090', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81767) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 16:11:21.276725 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3752d06-1b38-46dd-8465-1e95541232f4 service cinder] REQUEST_METHOD: `GET` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:11:21.277004 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3752d06-1b38-46dd-8465-1e95541232f4 service cinder] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:11:21.277292 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3752d06-1b38-46dd-8465-1e95541232f4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:11:21.279129 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.fernet_utils [None req-e3752d06-1b38-46dd-8465-1e95541232f4 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81767) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 16:11:21.317690 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 967/1907] 10.222.0.15 () {58 vars in 958 bytes} [Mon Aug 22 16:11:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 16:11:21.318850 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30190e5d-40a2-4218-a961-a278ac5fbf53 None None] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 16:11:21.319836 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-30190e5d-40a2-4218-a961-a278ac5fbf53 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 16:11:21.334785 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30190e5d-40a2-4218-a961-a278ac5fbf53 None None] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 16:11:21.344510 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71fb3d56-c976-464b-ba14-ae36a7aeb60a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a98b1e4431940ab8d1fc95df2858a6c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 16:11:21.345180 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71fb3d56-c976-464b-ba14-ae36a7aeb60a admin admin] REQUEST_METHOD: `PUT` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:11:21.345467 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71fb3d56-c976-464b-ba14-ae36a7aeb60a admin admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:11:21.345726 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71fb3d56-c976-464b-ba14-ae36a7aeb60a admin admin] PATH_INFO: `/v3/projects/6d12cfa870464db1be34b222751fd7c4/users/c1c56586b608403b8a063a222e07995b/roles/9d9848cfd22e4b089db443c350e56670` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:11:21.358031 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3752d06-1b38-46dd-8465-1e95541232f4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 16:11:21.359749 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3752d06-1b38-46dd-8465-1e95541232f4 service cinder] RBAC: Authorization granted {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 16:11:21.367328 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71fb3d56-c976-464b-ba14-ae36a7aeb60a admin admin] RBAC: Authorizing `identity:create_grant(project_id=6d12cfa870464db1be34b222751fd7c4, user_id=c1c56586b608403b8a063a222e07995b, role_id=9d9848cfd22e4b089db443c350e56670)` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 16:11:21.370055 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71fb3d56-c976-464b-ba14-ae36a7aeb60a admin admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 16:11:21.371640 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 941/1908] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Aug 22 16:11:21 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 186 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 16:11:21.373382 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10ec1504-8a27-4d95-8049-97e0a6ce49fe None None] REQUEST_METHOD: `POST` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:11:21.373518 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10ec1504-8a27-4d95-8049-97e0a6ce49fe None None] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:11:21.373924 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10ec1504-8a27-4d95-8049-97e0a6ce49fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:11:21.406281 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.auth.core [None req-10ec1504-8a27-4d95-8049-97e0a6ce49fe None None] MFA Rules not processed for user `be4684a264444a52ac5488b5b2ea5ff1`. Rule list: `[]` (Enabled: `True`). {{(pid=81767) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 16:11:21.407521 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.fernet_utils [None req-10ec1504-8a27-4d95-8049-97e0a6ce49fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81767) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 16:11:21.450703 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 968/1909] 10.222.0.15 () {60 vars in 1483 bytes} [Mon Aug 22 16:11:21 2022] PUT /identity/v3/projects/6d12cfa870464db1be34b222751fd7c4/users/c1c56586b608403b8a063a222e07995b/roles/9d9848cfd22e4b089db443c350e56670 => generated 0 bytes in 132 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 16:11:21.454562 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71fb3d56-c976-464b-ba14-ae36a7aeb60a admin admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 16:11:21.456300 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-71fb3d56-c976-464b-ba14-ae36a7aeb60a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 16:11:21.479036 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 942/1910] 10.222.0.15 () {58 vars in 958 bytes} [Mon Aug 22 16:11:21 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 16:11:21.479948 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10ec1504-8a27-4d95-8049-97e0a6ce49fe None None] Authenticating user token {{(pid=81767) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 16:11:21.481245 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.fernet_utils [None req-10ec1504-8a27-4d95-8049-97e0a6ce49fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81767) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 16:11:21.505714 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10ec1504-8a27-4d95-8049-97e0a6ce49fe None None] Validating token access rules against request {{(pid=81767) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 16:11:21.536697 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71fb3d56-c976-464b-ba14-ae36a7aeb60a admin admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 16:11:21.543574 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33885431-a5b3-4be2-9cfb-7bbb9d76f0be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a98b1e4431940ab8d1fc95df2858a6c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 16:11:21.544425 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33885431-a5b3-4be2-9cfb-7bbb9d76f0be admin admin] REQUEST_METHOD: `DELETE` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:11:21.544666 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33885431-a5b3-4be2-9cfb-7bbb9d76f0be admin admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:11:21.544960 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33885431-a5b3-4be2-9cfb-7bbb9d76f0be admin admin] PATH_INFO: `/v3/users/7867ce27391c4ef2847d61fece719937` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:11:21.546120 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33885431-a5b3-4be2-9cfb-7bbb9d76f0be admin admin] RBAC: Authorizing `identity:delete_user(user_id=7867ce27391c4ef2847d61fece719937)` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 16:11:21.548239 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33885431-a5b3-4be2-9cfb-7bbb9d76f0be admin admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 16:11:21.567819 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64f095d9-62cb-43b4-9b94-def5f0af2868 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a98b1e4431940ab8d1fc95df2858a6c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81767) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 16:11:21.567819 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f095d9-62cb-43b4-9b94-def5f0af2868 admin admin] REQUEST_METHOD: `POST` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:11:21.567819 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f095d9-62cb-43b4-9b94-def5f0af2868 admin admin] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:11:21.567819 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f095d9-62cb-43b4-9b94-def5f0af2868 admin admin] PATH_INFO: `/v3/projects` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:11:21.568162 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64f095d9-62cb-43b4-9b94-def5f0af2868 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 16:11:21.570201 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64f095d9-62cb-43b4-9b94-def5f0af2868 admin admin] RBAC: Authorization granted {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 16:11:21.719859 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=87,keystone:DELETE=10,keystone:INSERT=3 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 16:11:21.727691 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 943/1911] 10.222.0.15 () {60 vars in 1153 bytes} [Mon Aug 22 16:11:21 2022] POST /identity/v3/projects => generated 381 bytes in 248 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 22 16:11:21.731668 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64f095d9-62cb-43b4-9b94-def5f0af2868 admin admin] Authenticating user token {{(pid=81767) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 16:11:21.804721 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64f095d9-62cb-43b4-9b94-def5f0af2868 admin admin] Validating token access rules against request {{(pid=81767) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 16:11:21.809660 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8967e56-869c-484a-948c-a7f5b28b9079 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a98b1e4431940ab8d1fc95df2858a6c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81767) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 16:11:21.810283 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8967e56-869c-484a-948c-a7f5b28b9079 admin admin] REQUEST_METHOD: `GET` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:11:21.810559 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8967e56-869c-484a-948c-a7f5b28b9079 admin admin] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:11:21.810823 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8967e56-869c-484a-948c-a7f5b28b9079 admin admin] PATH_INFO: `/v3/domains` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:11:21.811504 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8967e56-869c-484a-948c-a7f5b28b9079 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81767) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 16:11:21.811856 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8967e56-869c-484a-948c-a7f5b28b9079 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 16:11:21.813396 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8967e56-869c-484a-948c-a7f5b28b9079 admin admin] RBAC: Authorization granted {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 16:11:21.828215 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 944/1912] 10.222.0.15 () {58 vars in 1153 bytes} [Mon Aug 22 16:11:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 97 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 16:11:21.829131 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8967e56-869c-484a-948c-a7f5b28b9079 admin admin] Authenticating user token {{(pid=81767) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 16:11:21.833819 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-33885431-a5b3-4be2-9cfb-7bbb9d76f0be admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7867ce27391c4ef2847d61fece719937'} {{(pid=81768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 16:11:21.834810 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-33885431-a5b3-4be2-9cfb-7bbb9d76f0be admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7867ce27391c4ef2847d61fece719937'} {{(pid=81768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 16:11:21.835643 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-33885431-a5b3-4be2-9cfb-7bbb9d76f0be admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7867ce27391c4ef2847d61fece719937'} {{(pid=81768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 16:11:21.842436 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8967e56-869c-484a-948c-a7f5b28b9079 admin admin] Validating token access rules against request {{(pid=81767) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 16:11:21.845618 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-33885431-a5b3-4be2-9cfb-7bbb9d76f0be admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7867ce27391c4ef2847d61fece719937'} {{(pid=81768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 16:11:21.847258 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0e59f5b-6cd8-4d19-8384-ce37386c0456 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a98b1e4431940ab8d1fc95df2858a6c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81767) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 16:11:21.849103 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e59f5b-6cd8-4d19-8384-ce37386c0456 admin admin] REQUEST_METHOD: `GET` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:11:21.849103 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e59f5b-6cd8-4d19-8384-ce37386c0456 admin admin] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:11:21.849103 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e59f5b-6cd8-4d19-8384-ce37386c0456 admin admin] PATH_INFO: `/v3/roles` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:11:21.849559 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0e59f5b-6cd8-4d19-8384-ce37386c0456 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 16:11:21.851147 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0e59f5b-6cd8-4d19-8384-ce37386c0456 admin admin] RBAC: Authorization granted {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 16:11:21.865856 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 945/1913] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Aug 22 16:11:21 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 22 16:11:21.865856 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0e59f5b-6cd8-4d19-8384-ce37386c0456 admin admin] Authenticating user token {{(pid=81767) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 16:11:21.865856 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.fernet_utils [None req-e0e59f5b-6cd8-4d19-8384-ce37386c0456 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81767) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 16:11:21.941362 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-33885431-a5b3-4be2-9cfb-7bbb9d76f0be admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7867ce27391c4ef2847d61fece719937'} {{(pid=81768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 16:11:21.945720 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0e59f5b-6cd8-4d19-8384-ce37386c0456 admin admin] Validating token access rules against request {{(pid=81767) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 16:11:21.956481 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d78270b-77bf-42cf-be3a-1f889d4de1d4 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4ef569e41fb46d0b721bb1c7663616f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4ee0413f9ef4d4fb1f95a660babe090', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81767) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 16:11:21.957252 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d78270b-77bf-42cf-be3a-1f889d4de1d4 service nova] REQUEST_METHOD: `GET` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:11:21.958317 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d78270b-77bf-42cf-be3a-1f889d4de1d4 service nova] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:11:21.958424 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d78270b-77bf-42cf-be3a-1f889d4de1d4 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:11:21.960698 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.fernet_utils [None req-6d78270b-77bf-42cf-be3a-1f889d4de1d4 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81767) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 16:11:21.962885 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 969/1914] 10.222.0.15 () {58 vars in 1224 bytes} [Mon Aug 22 16:11:21 2022] DELETE /identity/v3/users/7867ce27391c4ef2847d61fece719937 => generated 0 bytes in 511 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 16:11:21.963763 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33885431-a5b3-4be2-9cfb-7bbb9d76f0be admin admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 16:11:21.985643 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-33885431-a5b3-4be2-9cfb-7bbb9d76f0be admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 16:11:22.030303 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d78270b-77bf-42cf-be3a-1f889d4de1d4 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 16:11:22.032819 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d78270b-77bf-42cf-be3a-1f889d4de1d4 service nova] RBAC: Authorization granted {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 16:11:22.044354 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 946/1915] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Aug 22 16:11:21 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 181 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 16:11:22.045331 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d78270b-77bf-42cf-be3a-1f889d4de1d4 service nova] Authenticating user token {{(pid=81767) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 16:11:22.046757 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.fernet_utils [None req-6d78270b-77bf-42cf-be3a-1f889d4de1d4 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81767) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 16:11:22.062905 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d78270b-77bf-42cf-be3a-1f889d4de1d4 service nova] Validating token access rules against request {{(pid=81767) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 16:11:22.069384 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb2cc082-73ad-47e8-b35e-b2652ff2b8fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a98b1e4431940ab8d1fc95df2858a6c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81767) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 16:11:22.070107 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb2cc082-73ad-47e8-b35e-b2652ff2b8fc admin admin] REQUEST_METHOD: `POST` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:11:22.070457 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb2cc082-73ad-47e8-b35e-b2652ff2b8fc admin admin] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:11:22.070802 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb2cc082-73ad-47e8-b35e-b2652ff2b8fc admin admin] PATH_INFO: `/v3/projects` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:11:22.071567 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb2cc082-73ad-47e8-b35e-b2652ff2b8fc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 16:11:22.074197 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb2cc082-73ad-47e8-b35e-b2652ff2b8fc admin admin] RBAC: Authorization granted {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 16:11:22.089373 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33885431-a5b3-4be2-9cfb-7bbb9d76f0be admin admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 16:11:22.102347 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a050870-aec6-44e5-a4c9-0229990f2c7e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a98b1e4431940ab8d1fc95df2858a6c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 16:11:22.103001 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a050870-aec6-44e5-a4c9-0229990f2c7e admin admin] REQUEST_METHOD: `POST` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:11:22.103298 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a050870-aec6-44e5-a4c9-0229990f2c7e admin admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:11:22.103582 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a050870-aec6-44e5-a4c9-0229990f2c7e admin admin] PATH_INFO: `/v3/users` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:11:22.104222 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a050870-aec6-44e5-a4c9-0229990f2c7e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 16:11:22.110915 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a050870-aec6-44e5-a4c9-0229990f2c7e admin admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 16:11:22.111579 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG dbcounter [-] [81767] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81767) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 16:11:22.180275 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 947/1916] 10.222.0.15 () {60 vars in 1153 bytes} [Mon Aug 22 16:11:22 2022] POST /identity/v3/projects => generated 367 bytes in 132 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 22 16:11:22.180275 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb2cc082-73ad-47e8-b35e-b2652ff2b8fc admin admin] Authenticating user token {{(pid=81767) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 16:11:22.180275 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.fernet_utils [None req-fb2cc082-73ad-47e8-b35e-b2652ff2b8fc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81767) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 16:11:22.254150 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 970/1917] 10.222.0.15 () {60 vars in 1144 bytes} [Mon Aug 22 16:11:21 2022] POST /identity/v3/users => generated 297 bytes in 288 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 22 16:11:22.254150 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a050870-aec6-44e5-a4c9-0229990f2c7e admin admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 16:11:22.308968 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a050870-aec6-44e5-a4c9-0229990f2c7e admin admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 16:11:22.417153 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb2cc082-73ad-47e8-b35e-b2652ff2b8fc admin admin] Validating token access rules against request {{(pid=81767) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 16:11:22.428031 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13b3ff12-fe2c-4a94-ab65-98735b7f29ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a98b1e4431940ab8d1fc95df2858a6c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81767) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 16:11:22.428866 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13b3ff12-fe2c-4a94-ab65-98735b7f29ec admin admin] REQUEST_METHOD: `PUT` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:11:22.429265 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13b3ff12-fe2c-4a94-ab65-98735b7f29ec admin admin] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:11:22.429419 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13b3ff12-fe2c-4a94-ab65-98735b7f29ec admin admin] PATH_INFO: `/v3/projects/6d12cfa870464db1be34b222751fd7c4/users/c1c56586b608403b8a063a222e07995b/roles/9d9848cfd22e4b089db443c350e56670` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:11:22.447369 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13b3ff12-fe2c-4a94-ab65-98735b7f29ec admin admin] RBAC: Authorizing `identity:create_grant(project_id=6d12cfa870464db1be34b222751fd7c4, user_id=c1c56586b608403b8a063a222e07995b, role_id=9d9848cfd22e4b089db443c350e56670)` {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 16:11:22.447488 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d0bb75a-d438-4511-95ae-fa8ae21f748d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a98b1e4431940ab8d1fc95df2858a6c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 16:11:22.448467 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d0bb75a-d438-4511-95ae-fa8ae21f748d admin admin] REQUEST_METHOD: `POST` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:11:22.448734 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d0bb75a-d438-4511-95ae-fa8ae21f748d admin admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:11:22.449004 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d0bb75a-d438-4511-95ae-fa8ae21f748d admin admin] PATH_INFO: `/v3/users` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:11:22.449742 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d0bb75a-d438-4511-95ae-fa8ae21f748d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 16:11:22.450965 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13b3ff12-fe2c-4a94-ab65-98735b7f29ec admin admin] RBAC: Authorization granted {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 16:11:22.455302 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d0bb75a-d438-4511-95ae-fa8ae21f748d admin admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 16:11:22.478347 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 948/1918] 10.222.0.15 () {60 vars in 1483 bytes} [Mon Aug 22 16:11:22 2022] PUT /identity/v3/projects/6d12cfa870464db1be34b222751fd7c4/users/c1c56586b608403b8a063a222e07995b/roles/9d9848cfd22e4b089db443c350e56670 => generated 0 bytes in 300 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 16:11:22.479572 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13b3ff12-fe2c-4a94-ab65-98735b7f29ec admin admin] Authenticating user token {{(pid=81767) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 16:11:22.550365 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13b3ff12-fe2c-4a94-ab65-98735b7f29ec admin admin] Validating token access rules against request {{(pid=81767) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 16:11:22.555869 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd89b6c7-6c02-4d97-a2b1-8c0865206a18 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a98b1e4431940ab8d1fc95df2858a6c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81767) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 16:11:22.556750 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd89b6c7-6c02-4d97-a2b1-8c0865206a18 admin admin] REQUEST_METHOD: `GET` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:11:22.568072 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd89b6c7-6c02-4d97-a2b1-8c0865206a18 admin admin] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:11:22.568072 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd89b6c7-6c02-4d97-a2b1-8c0865206a18 admin admin] PATH_INFO: `/v3/roles` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:11:22.568072 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd89b6c7-6c02-4d97-a2b1-8c0865206a18 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 16:11:22.568072 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd89b6c7-6c02-4d97-a2b1-8c0865206a18 admin admin] RBAC: Authorization granted {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 16:11:22.569662 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 949/1919] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Aug 22 16:11:22 2022] GET /identity/v3/roles => generated 1420 bytes in 91 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 22 16:11:22.570933 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c02a86-5939-4005-a24d-81918326af2a None None] REQUEST_METHOD: `POST` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:11:22.571220 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c02a86-5939-4005-a24d-81918326af2a None None] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:11:22.571486 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c02a86-5939-4005-a24d-81918326af2a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:11:22.605966 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 971/1920] 10.222.0.15 () {60 vars in 1144 bytes} [Mon Aug 22 16:11:22 2022] POST /identity/v3/users => generated 284 bytes in 354 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 22 16:11:22.606850 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d0bb75a-d438-4511-95ae-fa8ae21f748d admin admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 16:11:22.630203 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d0bb75a-d438-4511-95ae-fa8ae21f748d admin admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 16:11:22.637924 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-432ff1e4-5f5e-4e71-ab06-50cc8c74acc1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a98b1e4431940ab8d1fc95df2858a6c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 16:11:22.638907 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-432ff1e4-5f5e-4e71-ab06-50cc8c74acc1 admin admin] REQUEST_METHOD: `PUT` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:11:22.639802 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-432ff1e4-5f5e-4e71-ab06-50cc8c74acc1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:11:22.640018 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.auth.core [None req-75c02a86-5939-4005-a24d-81918326af2a None None] MFA Rules not processed for user `c1c56586b608403b8a063a222e07995b`. Rule list: `[]` (Enabled: `True`). {{(pid=81767) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 16:11:22.640123 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-432ff1e4-5f5e-4e71-ab06-50cc8c74acc1 admin admin] PATH_INFO: `/v3/projects/c95c499d811044d6b57903b4f764e5a0/users/5ec894c74be14c469e3458fc0f53603d/roles/9d9848cfd22e4b089db443c350e56670` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:11:22.641305 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.fernet_utils [None req-75c02a86-5939-4005-a24d-81918326af2a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81767) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 16:11:22.671658 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-432ff1e4-5f5e-4e71-ab06-50cc8c74acc1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c95c499d811044d6b57903b4f764e5a0, user_id=5ec894c74be14c469e3458fc0f53603d, role_id=9d9848cfd22e4b089db443c350e56670)` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 16:11:22.675453 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-432ff1e4-5f5e-4e71-ab06-50cc8c74acc1 admin admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 16:11:22.739999 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 950/1921] 10.222.0.15 () {58 vars in 958 bytes} [Mon Aug 22 16:11:22 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 16:11:22.741590 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75c02a86-5939-4005-a24d-81918326af2a None None] Authenticating user token {{(pid=81767) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 16:11:22.743176 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.fernet_utils [None req-75c02a86-5939-4005-a24d-81918326af2a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81767) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 16:11:22.757154 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 972/1922] 10.222.0.15 () {60 vars in 1483 bytes} [Mon Aug 22 16:11:22 2022] PUT /identity/v3/projects/c95c499d811044d6b57903b4f764e5a0/users/5ec894c74be14c469e3458fc0f53603d/roles/9d9848cfd22e4b089db443c350e56670 => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 16:11:22.758481 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-432ff1e4-5f5e-4e71-ab06-50cc8c74acc1 admin admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 16:11:22.773177 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75c02a86-5939-4005-a24d-81918326af2a None None] Validating token access rules against request {{(pid=81767) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 16:11:22.847866 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caa28e64-f7f7-4372-a7c3-3cd27b32f7c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a98b1e4431940ab8d1fc95df2858a6c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81767) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 16:11:22.849251 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caa28e64-f7f7-4372-a7c3-3cd27b32f7c2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:11:22.849544 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caa28e64-f7f7-4372-a7c3-3cd27b32f7c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:11:22.849820 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caa28e64-f7f7-4372-a7c3-3cd27b32f7c2 admin admin] PATH_INFO: `/v3/projects/c34f017c0c144e37a7b9e7cfd21860e1` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:11:22.851030 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caa28e64-f7f7-4372-a7c3-3cd27b32f7c2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c34f017c0c144e37a7b9e7cfd21860e1)` {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 16:11:22.853251 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caa28e64-f7f7-4372-a7c3-3cd27b32f7c2 admin admin] RBAC: Authorization granted {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 16:11:22.862841 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-432ff1e4-5f5e-4e71-ab06-50cc8c74acc1 admin admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 16:11:22.874264 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d505cf9a-dcba-44a0-934a-4b4582f20274 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a98b1e4431940ab8d1fc95df2858a6c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 16:11:22.875171 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d505cf9a-dcba-44a0-934a-4b4582f20274 admin admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:11:22.875582 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d505cf9a-dcba-44a0-934a-4b4582f20274 admin admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:11:22.875919 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d505cf9a-dcba-44a0-934a-4b4582f20274 admin admin] PATH_INFO: `/v3/roles` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:11:22.876665 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d505cf9a-dcba-44a0-934a-4b4582f20274 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 16:11:22.878611 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d505cf9a-dcba-44a0-934a-4b4582f20274 admin admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 16:11:22.892633 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 973/1923] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Aug 22 16:11:22 2022] GET /identity/v3/roles => generated 1420 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 22 16:11:22.896409 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d505cf9a-dcba-44a0-934a-4b4582f20274 admin admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 16:11:22.898503 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-d505cf9a-dcba-44a0-934a-4b4582f20274 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 16:11:22.965093 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d505cf9a-dcba-44a0-934a-4b4582f20274 admin admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 16:11:22.969157 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86648d83-c4f6-411a-a0d0-998d55742084 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79706d480f864998b9f8594be407ceab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4ee0413f9ef4d4fb1f95a660babe090', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 16:11:22.969769 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86648d83-c4f6-411a-a0d0-998d55742084 service cinder] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:11:22.970049 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86648d83-c4f6-411a-a0d0-998d55742084 service cinder] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:11:22.970322 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86648d83-c4f6-411a-a0d0-998d55742084 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:11:23.004725 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.notifications [None req-caa28e64-f7f7-4372-a7c3-3cd27b32f7c2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c34f017c0c144e37a7b9e7cfd21860e1', 'request_id': 'req-caa28e64-f7f7-4372-a7c3-3cd27b32f7c2'} {{(pid=81767) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 16:11:23.010653 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.notifications [None req-caa28e64-f7f7-4372-a7c3-3cd27b32f7c2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c34f017c0c144e37a7b9e7cfd21860e1', 'request_id': 'req-caa28e64-f7f7-4372-a7c3-3cd27b32f7c2'} {{(pid=81767) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 16:11:23.048519 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86648d83-c4f6-411a-a0d0-998d55742084 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 16:11:23.050245 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86648d83-c4f6-411a-a0d0-998d55742084 service cinder] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 16:11:23.060941 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 974/1924] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Aug 22 16:11:22 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 168 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 16:11:23.061681 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86648d83-c4f6-411a-a0d0-998d55742084 service cinder] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 16:11:23.072869 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86648d83-c4f6-411a-a0d0-998d55742084 service cinder] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 16:11:23.083922 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.notifications [None req-caa28e64-f7f7-4372-a7c3-3cd27b32f7c2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c34f017c0c144e37a7b9e7cfd21860e1', 'request_id': 'req-caa28e64-f7f7-4372-a7c3-3cd27b32f7c2'} {{(pid=81767) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 16:11:23.097200 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.notifications [None req-caa28e64-f7f7-4372-a7c3-3cd27b32f7c2 admin admin] The token cache is being invalidate because project c34f017c0c144e37a7b9e7cfd21860e1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81767) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 22 16:11:23.097575 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.notifications [None req-caa28e64-f7f7-4372-a7c3-3cd27b32f7c2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81767) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 16:11:23.098953 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.notifications [None req-caa28e64-f7f7-4372-a7c3-3cd27b32f7c2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81767) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 16:11:23.103842 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 951/1925] 10.222.0.15 () {58 vars in 1233 bytes} [Mon Aug 22 16:11:22 2022] DELETE /identity/v3/projects/c34f017c0c144e37a7b9e7cfd21860e1 => generated 0 bytes in 359 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 16:11:23.103842 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caa28e64-f7f7-4372-a7c3-3cd27b32f7c2 admin admin] Authenticating user token {{(pid=81767) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 16:11:23.103842 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.fernet_utils [None req-caa28e64-f7f7-4372-a7c3-3cd27b32f7c2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81767) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 16:11:23.144449 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8404f99f-7343-45e5-8d72-00d0a8cba978 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a98b1e4431940ab8d1fc95df2858a6c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 16:11:23.145920 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8404f99f-7343-45e5-8d72-00d0a8cba978 admin admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:11:23.146290 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8404f99f-7343-45e5-8d72-00d0a8cba978 admin admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:11:23.146637 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8404f99f-7343-45e5-8d72-00d0a8cba978 admin admin] PATH_INFO: `/v3/roles` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:11:23.147686 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8404f99f-7343-45e5-8d72-00d0a8cba978 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 16:11:23.150101 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8404f99f-7343-45e5-8d72-00d0a8cba978 admin admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 16:11:23.171731 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 975/1926] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Aug 22 16:11:23 2022] GET /identity/v3/roles => generated 1420 bytes in 102 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 22 16:11:23.171731 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa51d792-1fa3-4e2e-a4ef-31de963e6be3 None None] REQUEST_METHOD: `POST` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:11:23.171731 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa51d792-1fa3-4e2e-a4ef-31de963e6be3 None None] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:11:23.171731 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa51d792-1fa3-4e2e-a4ef-31de963e6be3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:11:23.174608 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caa28e64-f7f7-4372-a7c3-3cd27b32f7c2 admin admin] Validating token access rules against request {{(pid=81767) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 16:11:23.179346 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14195b88-6af9-4f0d-8747-c62e6fdf0eb9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a98b1e4431940ab8d1fc95df2858a6c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81767) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 16:11:23.179953 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14195b88-6af9-4f0d-8747-c62e6fdf0eb9 admin admin] REQUEST_METHOD: `GET` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:11:23.180231 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14195b88-6af9-4f0d-8747-c62e6fdf0eb9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:11:23.180500 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14195b88-6af9-4f0d-8747-c62e6fdf0eb9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:11:23.181195 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14195b88-6af9-4f0d-8747-c62e6fdf0eb9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 16:11:23.182825 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14195b88-6af9-4f0d-8747-c62e6fdf0eb9 admin admin] RBAC: Authorization granted {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 16:11:23.186001 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.auth.core [None req-aa51d792-1fa3-4e2e-a4ef-31de963e6be3 None None] MFA Rules not processed for user `6b2199ae0fb0480aa917ada391f5ec54`. Rule list: `[]` (Enabled: `True`). {{(pid=81768) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 16:11:23.187219 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-aa51d792-1fa3-4e2e-a4ef-31de963e6be3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 16:11:23.193689 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 952/1927] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Aug 22 16:11:23 2022] GET /identity/v3/roles => generated 1420 bytes in 92 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 22 16:11:23.194595 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14195b88-6af9-4f0d-8747-c62e6fdf0eb9 admin admin] Authenticating user token {{(pid=81767) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 16:11:23.197328 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.fernet_utils [None req-14195b88-6af9-4f0d-8747-c62e6fdf0eb9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81767) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 16:11:23.217837 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 976/1928] 10.222.0.15 () {58 vars in 958 bytes} [Mon Aug 22 16:11:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 16:11:23.217837 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa51d792-1fa3-4e2e-a4ef-31de963e6be3 None None] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 16:11:23.217837 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa51d792-1fa3-4e2e-a4ef-31de963e6be3 None None] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 16:11:23.218326 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14195b88-6af9-4f0d-8747-c62e6fdf0eb9 admin admin] Validating token access rules against request {{(pid=81767) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 16:11:23.218326 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e18d69b5-a00e-4445-9ec8-770b5f7b1617 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a98b1e4431940ab8d1fc95df2858a6c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81767) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 16:11:23.218326 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e18d69b5-a00e-4445-9ec8-770b5f7b1617 admin admin] REQUEST_METHOD: `PUT` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:11:23.218326 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e18d69b5-a00e-4445-9ec8-770b5f7b1617 admin admin] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:11:23.218590 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e18d69b5-a00e-4445-9ec8-770b5f7b1617 admin admin] PATH_INFO: `/v3/projects/c95c499d811044d6b57903b4f764e5a0/users/5ec894c74be14c469e3458fc0f53603d/roles/9d9848cfd22e4b089db443c350e56670` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:11:23.220376 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b422510d-cc8d-4e1c-8d34-7a05317d8722 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a98b1e4431940ab8d1fc95df2858a6c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 16:11:23.220615 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e18d69b5-a00e-4445-9ec8-770b5f7b1617 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c95c499d811044d6b57903b4f764e5a0, user_id=5ec894c74be14c469e3458fc0f53603d, role_id=9d9848cfd22e4b089db443c350e56670)` {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 16:11:23.221036 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b422510d-cc8d-4e1c-8d34-7a05317d8722 admin admin] REQUEST_METHOD: `PUT` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:11:23.221303 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b422510d-cc8d-4e1c-8d34-7a05317d8722 admin admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:11:23.221626 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b422510d-cc8d-4e1c-8d34-7a05317d8722 admin admin] PATH_INFO: `/v3/projects/253d69d423624a3aae7a58f08aabb941/users/28a2242d4d894a87922d4e0a3fac6d11/roles/e6fc1ee3fb974125ba9b4fe55b12b45e` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:11:23.223351 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e18d69b5-a00e-4445-9ec8-770b5f7b1617 admin admin] RBAC: Authorization granted {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 16:11:23.238886 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 953/1929] 10.222.0.15 () {60 vars in 1483 bytes} [Mon Aug 22 16:11:23 2022] PUT /identity/v3/projects/c95c499d811044d6b57903b4f764e5a0/users/5ec894c74be14c469e3458fc0f53603d/roles/9d9848cfd22e4b089db443c350e56670 => generated 0 bytes in 45 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 16:11:23.249827 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ada8fa1-5d48-4101-a839-55f1b0aa5cda None None] REQUEST_METHOD: `POST` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:11:23.249827 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ada8fa1-5d48-4101-a839-55f1b0aa5cda None None] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:11:23.249827 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ada8fa1-5d48-4101-a839-55f1b0aa5cda None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:11:23.256906 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b422510d-cc8d-4e1c-8d34-7a05317d8722 admin admin] RBAC: Authorizing `identity:create_grant(project_id=253d69d423624a3aae7a58f08aabb941, user_id=28a2242d4d894a87922d4e0a3fac6d11, role_id=e6fc1ee3fb974125ba9b4fe55b12b45e)` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 16:11:23.259609 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b422510d-cc8d-4e1c-8d34-7a05317d8722 admin admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 16:11:23.267877 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.auth.core [None req-1ada8fa1-5d48-4101-a839-55f1b0aa5cda None None] MFA Rules not processed for user `6b2199ae0fb0480aa917ada391f5ec54`. Rule list: `[]` (Enabled: `True`). {{(pid=81767) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 16:11:23.269053 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.fernet_utils [None req-1ada8fa1-5d48-4101-a839-55f1b0aa5cda None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81767) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 16:11:23.367253 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 954/1930] 10.222.0.15 () {58 vars in 958 bytes} [Mon Aug 22 16:11:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 16:11:23.367253 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-470f41d6-cd06-4524-99d8-ac111437d2b1 None None] REQUEST_METHOD: `POST` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:11:23.367253 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-470f41d6-cd06-4524-99d8-ac111437d2b1 None None] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:11:23.367253 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-470f41d6-cd06-4524-99d8-ac111437d2b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:11:23.375669 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 977/1931] 10.222.0.15 () {60 vars in 1483 bytes} [Mon Aug 22 16:11:23 2022] PUT /identity/v3/projects/253d69d423624a3aae7a58f08aabb941/users/28a2242d4d894a87922d4e0a3fac6d11/roles/e6fc1ee3fb974125ba9b4fe55b12b45e => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 16:11:23.387564 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b422510d-cc8d-4e1c-8d34-7a05317d8722 admin admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 16:11:23.400012 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.auth.core [None req-470f41d6-cd06-4524-99d8-ac111437d2b1 None None] MFA Rules not processed for user `5ec894c74be14c469e3458fc0f53603d`. Rule list: `[]` (Enabled: `True`). {{(pid=81767) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 16:11:23.401307 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.fernet_utils [None req-470f41d6-cd06-4524-99d8-ac111437d2b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81767) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 16:11:23.514799 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b422510d-cc8d-4e1c-8d34-7a05317d8722 admin admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 16:11:23.520518 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6031e62-f8c8-4105-bf6f-a24d98d7f7a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a98b1e4431940ab8d1fc95df2858a6c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 16:11:23.521477 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6031e62-f8c8-4105-bf6f-a24d98d7f7a8 admin admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:11:23.522090 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6031e62-f8c8-4105-bf6f-a24d98d7f7a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:11:23.522660 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6031e62-f8c8-4105-bf6f-a24d98d7f7a8 admin admin] PATH_INFO: `/v3/domains` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:11:23.523633 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6031e62-f8c8-4105-bf6f-a24d98d7f7a8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81768) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 16:11:23.524098 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6031e62-f8c8-4105-bf6f-a24d98d7f7a8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 16:11:23.527770 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6031e62-f8c8-4105-bf6f-a24d98d7f7a8 admin admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 16:11:23.533985 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 955/1932] 10.222.0.15 () {58 vars in 958 bytes} [Mon Aug 22 16:11:23 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 16:11:23.537311 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a54bf6dc-c037-4971-a319-9c3d8a57e494 None None] REQUEST_METHOD: `POST` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:11:23.539208 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a54bf6dc-c037-4971-a319-9c3d8a57e494 None None] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:11:23.545146 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a54bf6dc-c037-4971-a319-9c3d8a57e494 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:11:23.551617 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 978/1933] 10.222.0.15 () {58 vars in 1153 bytes} [Mon Aug 22 16:11:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 16:11:23.552543 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6031e62-f8c8-4105-bf6f-a24d98d7f7a8 admin admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 16:11:23.554372 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-d6031e62-f8c8-4105-bf6f-a24d98d7f7a8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 16:11:23.568933 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG dbcounter [-] [81767] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81767) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 16:11:23.597515 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.auth.core [None req-a54bf6dc-c037-4971-a319-9c3d8a57e494 None None] MFA Rules not processed for user `28a2242d4d894a87922d4e0a3fac6d11`. Rule list: `[]` (Enabled: `True`). {{(pid=81767) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 16:11:23.600208 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.fernet_utils [None req-a54bf6dc-c037-4971-a319-9c3d8a57e494 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81767) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 16:11:23.621811 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6031e62-f8c8-4105-bf6f-a24d98d7f7a8 admin admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 16:11:23.626090 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78f10c49-788b-4c45-9e52-a967f4ea6484 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79706d480f864998b9f8594be407ceab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4ee0413f9ef4d4fb1f95a660babe090', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 16:11:23.626863 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f10c49-788b-4c45-9e52-a967f4ea6484 service cinder] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:11:23.627110 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f10c49-788b-4c45-9e52-a967f4ea6484 service cinder] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:11:23.627526 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f10c49-788b-4c45-9e52-a967f4ea6484 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:11:23.630151 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-78f10c49-788b-4c45-9e52-a967f4ea6484 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 16:11:23.689044 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 956/1934] 10.222.0.15 () {58 vars in 958 bytes} [Mon Aug 22 16:11:23 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 16:11:23.689044 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a54bf6dc-c037-4971-a319-9c3d8a57e494 None None] Authenticating user token {{(pid=81767) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 16:11:23.699951 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a54bf6dc-c037-4971-a319-9c3d8a57e494 None None] Validating token access rules against request {{(pid=81767) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 16:11:23.705345 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a5fba93-8849-4cdd-ac50-3ad8ab3d1951 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a98b1e4431940ab8d1fc95df2858a6c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81767) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 16:11:23.706697 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a5fba93-8849-4cdd-ac50-3ad8ab3d1951 admin admin] REQUEST_METHOD: `POST` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:11:23.706697 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a5fba93-8849-4cdd-ac50-3ad8ab3d1951 admin admin] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:11:23.706959 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a5fba93-8849-4cdd-ac50-3ad8ab3d1951 admin admin] PATH_INFO: `/v3/projects` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:11:23.708227 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a5fba93-8849-4cdd-ac50-3ad8ab3d1951 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 16:11:23.710499 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a5fba93-8849-4cdd-ac50-3ad8ab3d1951 admin admin] RBAC: Authorization granted {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 16:11:23.717859 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78f10c49-788b-4c45-9e52-a967f4ea6484 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 16:11:23.719577 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78f10c49-788b-4c45-9e52-a967f4ea6484 service cinder] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 16:11:23.730633 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 979/1935] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Aug 22 16:11:23 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 178 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 16:11:23.731519 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78f10c49-788b-4c45-9e52-a967f4ea6484 service cinder] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 16:11:23.742945 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-78f10c49-788b-4c45-9e52-a967f4ea6484 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 16:11:23.754232 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78f10c49-788b-4c45-9e52-a967f4ea6484 service cinder] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 16:11:23.759929 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12a127bb-ea1a-4d05-9329-03abb4913ce6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79706d480f864998b9f8594be407ceab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4ee0413f9ef4d4fb1f95a660babe090', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 16:11:23.760556 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12a127bb-ea1a-4d05-9329-03abb4913ce6 service cinder] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:11:23.760836 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12a127bb-ea1a-4d05-9329-03abb4913ce6 service cinder] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:11:23.761115 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12a127bb-ea1a-4d05-9329-03abb4913ce6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:11:23.773084 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12a127bb-ea1a-4d05-9329-03abb4913ce6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 16:11:23.774516 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12a127bb-ea1a-4d05-9329-03abb4913ce6 service cinder] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 16:11:23.788666 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 980/1936] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Aug 22 16:11:23 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 57 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 16:11:23.789614 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12a127bb-ea1a-4d05-9329-03abb4913ce6 service cinder] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 16:11:23.822704 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12a127bb-ea1a-4d05-9329-03abb4913ce6 service cinder] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 16:11:23.828059 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1272f51-df26-42cd-92c1-110e75fd48b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a98b1e4431940ab8d1fc95df2858a6c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 16:11:23.828828 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1272f51-df26-42cd-92c1-110e75fd48b3 admin admin] REQUEST_METHOD: `POST` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:11:23.828969 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1272f51-df26-42cd-92c1-110e75fd48b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:11:23.829186 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1272f51-df26-42cd-92c1-110e75fd48b3 admin admin] PATH_INFO: `/v3/projects` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:11:23.829842 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1272f51-df26-42cd-92c1-110e75fd48b3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 16:11:23.831865 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1272f51-df26-42cd-92c1-110e75fd48b3 admin admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 16:11:23.867464 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 957/1937] 10.222.0.15 () {60 vars in 1153 bytes} [Mon Aug 22 16:11:23 2022] POST /identity/v3/projects => generated 365 bytes in 176 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 22 16:11:23.868562 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a5fba93-8849-4cdd-ac50-3ad8ab3d1951 admin admin] Authenticating user token {{(pid=81767) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 16:11:23.932270 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a5fba93-8849-4cdd-ac50-3ad8ab3d1951 admin admin] Validating token access rules against request {{(pid=81767) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 16:11:23.942676 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d386e42-dd7f-4b39-9cac-1dd2f7a66498 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a98b1e4431940ab8d1fc95df2858a6c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81767) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 16:11:23.942676 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d386e42-dd7f-4b39-9cac-1dd2f7a66498 admin admin] REQUEST_METHOD: `POST` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:11:23.942676 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d386e42-dd7f-4b39-9cac-1dd2f7a66498 admin admin] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:11:23.942676 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d386e42-dd7f-4b39-9cac-1dd2f7a66498 admin admin] PATH_INFO: `/v3/users` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:11:23.943020 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d386e42-dd7f-4b39-9cac-1dd2f7a66498 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 16:11:23.943522 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d386e42-dd7f-4b39-9cac-1dd2f7a66498 admin admin] RBAC: Authorization granted {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 16:11:23.963321 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 981/1938] 10.222.0.15 () {60 vars in 1153 bytes} [Mon Aug 22 16:11:23 2022] POST /identity/v3/projects => generated 369 bytes in 174 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 22 16:11:23.981799 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1272f51-df26-42cd-92c1-110e75fd48b3 admin admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 16:11:23.995478 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:INSERT=5,keystone:SELECT=95 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 16:11:24.026366 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 958/1939] 10.222.0.15 () {60 vars in 1144 bytes} [Mon Aug 22 16:11:23 2022] POST /identity/v3/users => generated 283 bytes in 158 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 22 16:11:24.031018 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d386e42-dd7f-4b39-9cac-1dd2f7a66498 admin admin] Authenticating user token {{(pid=81767) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 16:11:24.058984 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1272f51-df26-42cd-92c1-110e75fd48b3 admin admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 16:11:24.061789 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d386e42-dd7f-4b39-9cac-1dd2f7a66498 admin admin] Validating token access rules against request {{(pid=81767) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 16:11:24.066145 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d80f6824-f419-4e2c-b20f-5dbef27b911b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a98b1e4431940ab8d1fc95df2858a6c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 16:11:24.066849 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d80f6824-f419-4e2c-b20f-5dbef27b911b admin admin] REQUEST_METHOD: `POST` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:11:24.068284 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d80f6824-f419-4e2c-b20f-5dbef27b911b admin admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:11:24.068284 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d80f6824-f419-4e2c-b20f-5dbef27b911b admin admin] PATH_INFO: `/v3/users` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:11:24.068994 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d80f6824-f419-4e2c-b20f-5dbef27b911b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 16:11:24.071109 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d80f6824-f419-4e2c-b20f-5dbef27b911b admin admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 16:11:24.082809 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2077d288-d049-4eb7-b5c2-c3fff269795f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a98b1e4431940ab8d1fc95df2858a6c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81767) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 16:11:24.085000 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2077d288-d049-4eb7-b5c2-c3fff269795f admin admin] REQUEST_METHOD: `GET` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:11:24.085835 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2077d288-d049-4eb7-b5c2-c3fff269795f admin admin] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:11:24.086636 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2077d288-d049-4eb7-b5c2-c3fff269795f admin admin] PATH_INFO: `/v3/roles` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:11:24.091517 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2077d288-d049-4eb7-b5c2-c3fff269795f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 16:11:24.093679 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2077d288-d049-4eb7-b5c2-c3fff269795f admin admin] RBAC: Authorization granted {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 16:11:24.108701 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 959/1940] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Aug 22 16:11:24 2022] GET /identity/v3/roles => generated 1420 bytes in 79 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 22 16:11:24.109789 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2077d288-d049-4eb7-b5c2-c3fff269795f admin admin] Authenticating user token {{(pid=81767) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 16:11:24.180835 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2077d288-d049-4eb7-b5c2-c3fff269795f admin admin] Validating token access rules against request {{(pid=81767) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 16:11:24.185512 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdde8ea0-1b0f-43ca-b3bd-4d7165d0b065 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79706d480f864998b9f8594be407ceab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4ee0413f9ef4d4fb1f95a660babe090', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81767) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 16:11:24.186207 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdde8ea0-1b0f-43ca-b3bd-4d7165d0b065 service cinder] REQUEST_METHOD: `GET` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:11:24.186549 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdde8ea0-1b0f-43ca-b3bd-4d7165d0b065 service cinder] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:11:24.186885 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdde8ea0-1b0f-43ca-b3bd-4d7165d0b065 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:11:24.189018 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.fernet_utils [None req-cdde8ea0-1b0f-43ca-b3bd-4d7165d0b065 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81767) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 16:11:24.202012 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 982/1941] 10.222.0.15 () {60 vars in 1144 bytes} [Mon Aug 22 16:11:23 2022] POST /identity/v3/users => generated 291 bytes in 218 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 22 16:11:24.202454 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d80f6824-f419-4e2c-b20f-5dbef27b911b admin admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 16:11:24.248358 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d80f6824-f419-4e2c-b20f-5dbef27b911b admin admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 16:11:24.253065 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e79664ac-f76b-437e-877b-2126ab929853 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a98b1e4431940ab8d1fc95df2858a6c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 16:11:24.253701 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e79664ac-f76b-437e-877b-2126ab929853 admin admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:11:24.254030 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e79664ac-f76b-437e-877b-2126ab929853 admin admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:11:24.254528 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e79664ac-f76b-437e-877b-2126ab929853 admin admin] PATH_INFO: `/v3/roles` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:11:24.255116 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e79664ac-f76b-437e-877b-2126ab929853 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 16:11:24.256925 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e79664ac-f76b-437e-877b-2126ab929853 admin admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 16:11:24.267836 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 983/1942] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Aug 22 16:11:24 2022] GET /identity/v3/roles => generated 1420 bytes in 68 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 22 16:11:24.268736 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e79664ac-f76b-437e-877b-2126ab929853 admin admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 16:11:24.295133 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e79664ac-f76b-437e-877b-2126ab929853 admin admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 16:11:24.302565 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92e163a9-dbbd-4670-99fe-dae71b249ebc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a98b1e4431940ab8d1fc95df2858a6c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 16:11:24.303186 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92e163a9-dbbd-4670-99fe-dae71b249ebc admin admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:11:24.303536 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92e163a9-dbbd-4670-99fe-dae71b249ebc admin admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:11:24.303808 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92e163a9-dbbd-4670-99fe-dae71b249ebc admin admin] PATH_INFO: `/v3/roles` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:11:24.304464 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92e163a9-dbbd-4670-99fe-dae71b249ebc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 16:11:24.306137 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92e163a9-dbbd-4670-99fe-dae71b249ebc admin admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 16:11:24.312657 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdde8ea0-1b0f-43ca-b3bd-4d7165d0b065 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 16:11:24.314286 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdde8ea0-1b0f-43ca-b3bd-4d7165d0b065 service cinder] RBAC: Authorization granted {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 16:11:24.323934 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 984/1943] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Aug 22 16:11:24 2022] GET /identity/v3/roles => generated 1420 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 22 16:11:24.324875 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92e163a9-dbbd-4670-99fe-dae71b249ebc admin admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 16:11:24.339143 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 960/1944] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Aug 22 16:11:24 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 224 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 16:11:24.339143 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdde8ea0-1b0f-43ca-b3bd-4d7165d0b065 service cinder] Authenticating user token {{(pid=81767) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 16:11:24.351813 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdde8ea0-1b0f-43ca-b3bd-4d7165d0b065 service cinder] Validating token access rules against request {{(pid=81767) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 16:11:24.353011 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92e163a9-dbbd-4670-99fe-dae71b249ebc admin admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 16:11:24.370563 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e13b5a27-d409-4022-892b-3912f7476b36 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a98b1e4431940ab8d1fc95df2858a6c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81767) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 16:11:24.370724 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d56f3ff-8073-4999-a90f-3716aba8dbdd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a98b1e4431940ab8d1fc95df2858a6c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 16:11:24.370724 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d56f3ff-8073-4999-a90f-3716aba8dbdd admin admin] REQUEST_METHOD: `PUT` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:11:24.370724 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d56f3ff-8073-4999-a90f-3716aba8dbdd admin admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:11:24.370947 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e13b5a27-d409-4022-892b-3912f7476b36 admin admin] REQUEST_METHOD: `PUT` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:11:24.371021 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d56f3ff-8073-4999-a90f-3716aba8dbdd admin admin] PATH_INFO: `/v3/projects/9404a43483c44564a0dcf7d5895c3b08/users/dae6669681b540e9be606226ff80dadd/roles/e6fc1ee3fb974125ba9b4fe55b12b45e` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:11:24.371089 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e13b5a27-d409-4022-892b-3912f7476b36 admin admin] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:11:24.371089 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e13b5a27-d409-4022-892b-3912f7476b36 admin admin] PATH_INFO: `/v3/projects/4c1815c448c740ca9657c6dd6920c887/users/e4caeae7c3f840858460d8243e706210/roles/9d9848cfd22e4b089db443c350e56670` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:11:24.385210 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d56f3ff-8073-4999-a90f-3716aba8dbdd admin admin] RBAC: Authorizing `identity:create_grant(project_id=9404a43483c44564a0dcf7d5895c3b08, user_id=dae6669681b540e9be606226ff80dadd, role_id=e6fc1ee3fb974125ba9b4fe55b12b45e)` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 16:11:24.388005 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d56f3ff-8073-4999-a90f-3716aba8dbdd admin admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 16:11:24.412800 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e13b5a27-d409-4022-892b-3912f7476b36 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4c1815c448c740ca9657c6dd6920c887, user_id=e4caeae7c3f840858460d8243e706210, role_id=9d9848cfd22e4b089db443c350e56670)` {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 16:11:24.415773 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e13b5a27-d409-4022-892b-3912f7476b36 admin admin] RBAC: Authorization granted {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 16:11:24.480486 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 985/1945] 10.222.0.15 () {60 vars in 1483 bytes} [Mon Aug 22 16:11:24 2022] PUT /identity/v3/projects/9404a43483c44564a0dcf7d5895c3b08/users/dae6669681b540e9be606226ff80dadd/roles/e6fc1ee3fb974125ba9b4fe55b12b45e => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 16:11:24.491145 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c78dbb35-9613-42de-a7dd-6c5ebc703491 None None] REQUEST_METHOD: `POST` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:11:24.491500 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c78dbb35-9613-42de-a7dd-6c5ebc703491 None None] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:11:24.491773 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c78dbb35-9613-42de-a7dd-6c5ebc703491 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:11:24.503679 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 961/1946] 10.222.0.15 () {60 vars in 1483 bytes} [Mon Aug 22 16:11:24 2022] PUT /identity/v3/projects/4c1815c448c740ca9657c6dd6920c887/users/e4caeae7c3f840858460d8243e706210/roles/9d9848cfd22e4b089db443c350e56670 => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 16:11:24.507734 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e13b5a27-d409-4022-892b-3912f7476b36 admin admin] Authenticating user token {{(pid=81767) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 16:11:24.533025 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.auth.core [None req-c78dbb35-9613-42de-a7dd-6c5ebc703491 None None] MFA Rules not processed for user `dae6669681b540e9be606226ff80dadd`. Rule list: `[]` (Enabled: `True`). {{(pid=81768) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 16:11:24.536710 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-c78dbb35-9613-42de-a7dd-6c5ebc703491 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 16:11:24.608695 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e13b5a27-d409-4022-892b-3912f7476b36 admin admin] Validating token access rules against request {{(pid=81767) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 16:11:24.608695 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a355dcac-d5f8-4fd6-84e7-562e7a5e7d4e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a98b1e4431940ab8d1fc95df2858a6c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81767) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 16:11:24.608695 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a355dcac-d5f8-4fd6-84e7-562e7a5e7d4e admin admin] REQUEST_METHOD: `GET` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:11:24.608695 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a355dcac-d5f8-4fd6-84e7-562e7a5e7d4e admin admin] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:11:24.609088 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a355dcac-d5f8-4fd6-84e7-562e7a5e7d4e admin admin] PATH_INFO: `/v3/roles` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:11:24.609088 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a355dcac-d5f8-4fd6-84e7-562e7a5e7d4e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 16:11:24.609088 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a355dcac-d5f8-4fd6-84e7-562e7a5e7d4e admin admin] RBAC: Authorization granted {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 16:11:24.626356 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 962/1947] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Aug 22 16:11:24 2022] GET /identity/v3/roles => generated 1420 bytes in 116 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 22 16:11:24.627900 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a355dcac-d5f8-4fd6-84e7-562e7a5e7d4e admin admin] Authenticating user token {{(pid=81767) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 16:11:24.647119 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a355dcac-d5f8-4fd6-84e7-562e7a5e7d4e admin admin] Validating token access rules against request {{(pid=81767) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 16:11:24.655237 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0013a32-c862-4fd1-b3d5-78d3a6cb401f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a98b1e4431940ab8d1fc95df2858a6c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81767) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 16:11:24.659001 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 986/1948] 10.222.0.15 () {58 vars in 958 bytes} [Mon Aug 22 16:11:24 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 22 16:11:24.659470 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0013a32-c862-4fd1-b3d5-78d3a6cb401f admin admin] REQUEST_METHOD: `PUT` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:11:24.659470 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0013a32-c862-4fd1-b3d5-78d3a6cb401f admin admin] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:11:24.659470 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0013a32-c862-4fd1-b3d5-78d3a6cb401f admin admin] PATH_INFO: `/v3/projects/4c1815c448c740ca9657c6dd6920c887/users/e4caeae7c3f840858460d8243e706210/roles/9d9848cfd22e4b089db443c350e56670` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:11:24.661563 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0013a32-c862-4fd1-b3d5-78d3a6cb401f admin admin] RBAC: Authorizing `identity:create_grant(project_id=4c1815c448c740ca9657c6dd6920c887, user_id=e4caeae7c3f840858460d8243e706210, role_id=9d9848cfd22e4b089db443c350e56670)` {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 16:11:24.661683 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c78dbb35-9613-42de-a7dd-6c5ebc703491 None None] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 16:11:24.665086 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0013a32-c862-4fd1-b3d5-78d3a6cb401f admin admin] RBAC: Authorization granted {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 16:11:24.681811 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 963/1949] 10.222.0.15 () {60 vars in 1483 bytes} [Mon Aug 22 16:11:24 2022] PUT /identity/v3/projects/4c1815c448c740ca9657c6dd6920c887/users/e4caeae7c3f840858460d8243e706210/roles/9d9848cfd22e4b089db443c350e56670 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 16:11:24.695256 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1457e98b-09c8-4a37-892f-43a56db8e39b None None] REQUEST_METHOD: `POST` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:11:24.695564 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1457e98b-09c8-4a37-892f-43a56db8e39b None None] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:11:24.695841 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1457e98b-09c8-4a37-892f-43a56db8e39b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:11:24.700201 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c78dbb35-9613-42de-a7dd-6c5ebc703491 None None] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 16:11:24.747559 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.auth.core [None req-1457e98b-09c8-4a37-892f-43a56db8e39b None None] MFA Rules not processed for user `e4caeae7c3f840858460d8243e706210`. Rule list: `[]` (Enabled: `True`). {{(pid=81767) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 16:11:24.749165 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.fernet_utils [None req-1457e98b-09c8-4a37-892f-43a56db8e39b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81767) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 16:11:24.803299 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be7e58ec-1880-4d20-ac76-1c848274e20f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a98b1e4431940ab8d1fc95df2858a6c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 16:11:24.803299 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be7e58ec-1880-4d20-ac76-1c848274e20f admin admin] REQUEST_METHOD: `POST` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:11:24.803299 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be7e58ec-1880-4d20-ac76-1c848274e20f admin admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:11:24.803299 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be7e58ec-1880-4d20-ac76-1c848274e20f admin admin] PATH_INFO: `/v3/projects` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:11:24.804091 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be7e58ec-1880-4d20-ac76-1c848274e20f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 16:11:24.806014 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be7e58ec-1880-4d20-ac76-1c848274e20f admin admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 16:11:24.842832 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 964/1950] 10.222.0.15 () {58 vars in 958 bytes} [Mon Aug 22 16:11:24 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 22 16:11:24.905491 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1457e98b-09c8-4a37-892f-43a56db8e39b None None] Authenticating user token {{(pid=81767) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 16:11:24.964076 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 987/1951] 10.222.0.15 () {60 vars in 1153 bytes} [Mon Aug 22 16:11:24 2022] POST /identity/v3/projects => generated 363 bytes in 303 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 22 16:11:24.965023 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be7e58ec-1880-4d20-ac76-1c848274e20f admin admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 16:11:24.993306 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1457e98b-09c8-4a37-892f-43a56db8e39b None None] Validating token access rules against request {{(pid=81767) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 16:11:25.049411 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be7e58ec-1880-4d20-ac76-1c848274e20f admin admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 16:11:25.055572 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcad80c2-2b22-46b1-abc0-0de08d3556cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a98b1e4431940ab8d1fc95df2858a6c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 16:11:25.056227 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcad80c2-2b22-46b1-abc0-0de08d3556cf admin admin] REQUEST_METHOD: `POST` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:11:25.056925 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcad80c2-2b22-46b1-abc0-0de08d3556cf admin admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:11:25.057205 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcad80c2-2b22-46b1-abc0-0de08d3556cf admin admin] PATH_INFO: `/v3/users` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:11:25.057864 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcad80c2-2b22-46b1-abc0-0de08d3556cf admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 16:11:25.064528 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcad80c2-2b22-46b1-abc0-0de08d3556cf admin admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 16:11:25.069043 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6ca769f-8b0d-4f68-9d73-9653b9557a60 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79706d480f864998b9f8594be407ceab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4ee0413f9ef4d4fb1f95a660babe090', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81767) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 16:11:25.070004 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6ca769f-8b0d-4f68-9d73-9653b9557a60 service cinder] REQUEST_METHOD: `GET` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:11:25.070473 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6ca769f-8b0d-4f68-9d73-9653b9557a60 service cinder] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:11:25.071429 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6ca769f-8b0d-4f68-9d73-9653b9557a60 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:11:25.153004 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6ca769f-8b0d-4f68-9d73-9653b9557a60 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 16:11:25.154557 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6ca769f-8b0d-4f68-9d73-9653b9557a60 service cinder] RBAC: Authorization granted {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 16:11:25.168457 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 965/1952] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Aug 22 16:11:24 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 263 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 22 16:11:25.177786 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 988/1953] 10.222.0.15 () {60 vars in 1144 bytes} [Mon Aug 22 16:11:24 2022] POST /identity/v3/users => generated 288 bytes in 213 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 22 16:11:25.186918 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6ca769f-8b0d-4f68-9d73-9653b9557a60 service cinder] Authenticating user token {{(pid=81767) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 16:11:25.195190 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6ca769f-8b0d-4f68-9d73-9653b9557a60 service cinder] Validating token access rules against request {{(pid=81767) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 16:11:25.200694 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-118507f9-b82c-490d-95e4-a8e6608d2e1f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a98b1e4431940ab8d1fc95df2858a6c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81767) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 16:11:25.201312 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-118507f9-b82c-490d-95e4-a8e6608d2e1f admin admin] REQUEST_METHOD: `GET` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:11:25.201674 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-118507f9-b82c-490d-95e4-a8e6608d2e1f admin admin] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:11:25.201941 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-118507f9-b82c-490d-95e4-a8e6608d2e1f admin admin] PATH_INFO: `/v3/roles` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:11:25.202609 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-118507f9-b82c-490d-95e4-a8e6608d2e1f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 16:11:25.206088 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-118507f9-b82c-490d-95e4-a8e6608d2e1f admin admin] RBAC: Authorization granted {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 16:11:25.217871 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 966/1954] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Aug 22 16:11:25 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 22 16:11:25.223766 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcad80c2-2b22-46b1-abc0-0de08d3556cf admin admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 16:11:25.234107 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcad80c2-2b22-46b1-abc0-0de08d3556cf admin admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 16:11:25.239508 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91fa91d0-04c6-4dc7-a7da-c28efe4f2f2c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a98b1e4431940ab8d1fc95df2858a6c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 16:11:25.240181 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91fa91d0-04c6-4dc7-a7da-c28efe4f2f2c admin admin] REQUEST_METHOD: `PUT` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:11:25.240461 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91fa91d0-04c6-4dc7-a7da-c28efe4f2f2c admin admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:11:25.240781 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91fa91d0-04c6-4dc7-a7da-c28efe4f2f2c admin admin] PATH_INFO: `/v3/projects/7c86b0d40f2847ad83c93fbc8de8cf44/users/8834d2789562476e902ba236b381a580/roles/9d9848cfd22e4b089db443c350e56670` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:11:25.259072 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91fa91d0-04c6-4dc7-a7da-c28efe4f2f2c admin admin] RBAC: Authorizing `identity:create_grant(project_id=7c86b0d40f2847ad83c93fbc8de8cf44, user_id=8834d2789562476e902ba236b381a580, role_id=9d9848cfd22e4b089db443c350e56670)` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 16:11:25.262046 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91fa91d0-04c6-4dc7-a7da-c28efe4f2f2c admin admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 16:11:25.342726 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 989/1955] 10.222.0.15 () {60 vars in 1483 bytes} [Mon Aug 22 16:11:25 2022] PUT /identity/v3/projects/7c86b0d40f2847ad83c93fbc8de8cf44/users/8834d2789562476e902ba236b381a580/roles/9d9848cfd22e4b089db443c350e56670 => generated 0 bytes in 116 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 16:11:25.343888 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-118507f9-b82c-490d-95e4-a8e6608d2e1f admin admin] Authenticating user token {{(pid=81767) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 16:11:25.364416 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-118507f9-b82c-490d-95e4-a8e6608d2e1f admin admin] Validating token access rules against request {{(pid=81767) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 16:11:25.404485 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG dbcounter [-] [81767] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81767) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 16:11:25.450056 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a68bf56-017f-4744-8791-4fcf3ce91814 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a98b1e4431940ab8d1fc95df2858a6c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81767) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 16:11:25.450684 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a68bf56-017f-4744-8791-4fcf3ce91814 admin admin] REQUEST_METHOD: `GET` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:11:25.451084 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a68bf56-017f-4744-8791-4fcf3ce91814 admin admin] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:11:25.451254 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a68bf56-017f-4744-8791-4fcf3ce91814 admin admin] PATH_INFO: `/v3/roles` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:11:25.451938 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a68bf56-017f-4744-8791-4fcf3ce91814 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 16:11:25.453589 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a68bf56-017f-4744-8791-4fcf3ce91814 admin admin] RBAC: Authorization granted {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 16:11:25.466959 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 967/1956] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Aug 22 16:11:25 2022] GET /identity/v3/roles => generated 1420 bytes in 124 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 22 16:11:25.476450 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91fa91d0-04c6-4dc7-a7da-c28efe4f2f2c admin admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 16:11:25.478370 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-840cd4ac-b46c-4742-b420-6512213c8813 None None] REQUEST_METHOD: `POST` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:11:25.478734 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-840cd4ac-b46c-4742-b420-6512213c8813 None None] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:11:25.479011 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-840cd4ac-b46c-4742-b420-6512213c8813 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:11:25.496956 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91fa91d0-04c6-4dc7-a7da-c28efe4f2f2c admin admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 16:11:25.508491 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bceeccf9-93cc-4fc0-8c2b-d32c52b4290b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a98b1e4431940ab8d1fc95df2858a6c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 16:11:25.509523 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.auth.core [None req-840cd4ac-b46c-4742-b420-6512213c8813 None None] MFA Rules not processed for user `be4684a264444a52ac5488b5b2ea5ff1`. Rule list: `[]` (Enabled: `True`). {{(pid=81767) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 16:11:25.509652 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bceeccf9-93cc-4fc0-8c2b-d32c52b4290b admin admin] REQUEST_METHOD: `PUT` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:11:25.509816 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bceeccf9-93cc-4fc0-8c2b-d32c52b4290b admin admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:11:25.510750 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.fernet_utils [None req-840cd4ac-b46c-4742-b420-6512213c8813 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81767) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 16:11:25.511329 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bceeccf9-93cc-4fc0-8c2b-d32c52b4290b admin admin] PATH_INFO: `/v3/projects/7c86b0d40f2847ad83c93fbc8de8cf44/users/8834d2789562476e902ba236b381a580/roles/9d9848cfd22e4b089db443c350e56670` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:11:25.513448 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bceeccf9-93cc-4fc0-8c2b-d32c52b4290b admin admin] RBAC: Authorizing `identity:create_grant(project_id=7c86b0d40f2847ad83c93fbc8de8cf44, user_id=8834d2789562476e902ba236b381a580, role_id=9d9848cfd22e4b089db443c350e56670)` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 16:11:25.516638 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bceeccf9-93cc-4fc0-8c2b-d32c52b4290b admin admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 16:11:25.531757 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 990/1957] 10.222.0.15 () {60 vars in 1483 bytes} [Mon Aug 22 16:11:25 2022] PUT /identity/v3/projects/7c86b0d40f2847ad83c93fbc8de8cf44/users/8834d2789562476e902ba236b381a580/roles/9d9848cfd22e4b089db443c350e56670 => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 16:11:25.543886 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73162960-fac9-4d0d-954a-f0aa9465c106 None None] REQUEST_METHOD: `POST` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:11:25.544175 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73162960-fac9-4d0d-954a-f0aa9465c106 None None] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:11:25.544453 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73162960-fac9-4d0d-954a-f0aa9465c106 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:11:25.570644 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 968/1958] 10.222.0.15 () {58 vars in 958 bytes} [Mon Aug 22 16:11:25 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 16:11:25.579395 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.auth.core [None req-73162960-fac9-4d0d-954a-f0aa9465c106 None None] MFA Rules not processed for user `8834d2789562476e902ba236b381a580`. Rule list: `[]` (Enabled: `True`). {{(pid=81768) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 16:11:25.580615 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-73162960-fac9-4d0d-954a-f0aa9465c106 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 16:11:25.699615 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 991/1959] 10.222.0.15 () {58 vars in 958 bytes} [Mon Aug 22 16:11:25 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 16:11:25.873333 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=100 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 16:11:25.932573 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68211d68-e2df-41df-8f0d-2f0831ee8a37 None None] REQUEST_METHOD: `POST` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:11:25.933041 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68211d68-e2df-41df-8f0d-2f0831ee8a37 None None] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:11:25.933232 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68211d68-e2df-41df-8f0d-2f0831ee8a37 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:11:25.939652 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application [None req-68211d68-e2df-41df-8f0d-2f0831ee8a37 None None] Expecting to find domain in project. The server could not comply with the request since it is either malformed or otherwise incorrect. The client is assumed to be in error.: keystone.exception.ValidationError: Expecting to find domain in project. The server could not comply with the request since it is either malformed or otherwise incorrect. The client is assumed to be in error. Aug 22 16:11:25.939652 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 16:11:25.939652 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 16:11:25.939652 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 16:11:25.939652 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 16:11:25.939652 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 16:11:25.939652 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 16:11:25.939652 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 16:11:25.939652 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 16:11:25.939652 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 16:11:25.939652 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 16:11:25.939652 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 16:11:25.939652 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/server/flask/common.py", line 1064, in wrapper Aug 22 16:11:25.940618 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return f(*args, **kwargs) Aug 22 16:11:25.940618 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/auth.py", line 315, in post Aug 22 16:11:25.940618 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application token = authentication.authenticate_for_token(auth_data) Aug 22 16:11:25.940618 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/_shared/authentication.py", line 182, in authenticate_for_token Aug 22 16:11:25.940618 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application auth_info = core.AuthInfo.create(auth=auth) Aug 22 16:11:25.940618 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/auth/core.py", line 126, in create Aug 22 16:11:25.940618 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application auth_info._validate_and_normalize_auth_data(scope_only) Aug 22 16:11:25.940618 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/auth/core.py", line 340, in _validate_and_normalize_auth_data Aug 22 16:11:25.940618 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application self._validate_and_normalize_scope_data() Aug 22 16:11:25.940618 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/auth/core.py", line 290, in _validate_and_normalize_scope_data Aug 22 16:11:25.940618 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application project_ref = self._lookup_project(self.auth['scope']['project']) Aug 22 16:11:25.940618 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/auth/core.py", line 194, in _lookup_project Aug 22 16:11:25.940618 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application raise exception.ValidationError(attribute='domain', Aug 22 16:11:25.940618 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application keystone.exception.ValidationError: Expecting to find domain in project. The server could not comply with the request since it is either malformed or otherwise incorrect. The client is assumed to be in error. Aug 22 16:11:25.940618 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application Aug 22 16:11:25.942261 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 969/1960] 10.222.0.15 () {58 vars in 1009 bytes} [Mon Aug 22 16:11:25 2022] POST /identity/v3/auth/tokens => generated 230 bytes in 11 msecs (HTTP/1.1 400) 5 headers in 186 bytes (1 switches on core 0) Aug 22 16:11:26.463214 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73162960-fac9-4d0d-954a-f0aa9465c106 None None] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 16:11:26.543469 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73162960-fac9-4d0d-954a-f0aa9465c106 None None] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 16:11:26.547990 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53a1bbeb-77f1-4950-baf8-a6332c8cb48c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79706d480f864998b9f8594be407ceab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4ee0413f9ef4d4fb1f95a660babe090', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 16:11:26.548704 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53a1bbeb-77f1-4950-baf8-a6332c8cb48c service cinder] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:11:26.549294 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53a1bbeb-77f1-4950-baf8-a6332c8cb48c service cinder] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:11:26.549475 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53a1bbeb-77f1-4950-baf8-a6332c8cb48c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:11:26.561922 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53a1bbeb-77f1-4950-baf8-a6332c8cb48c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 16:11:26.563818 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53a1bbeb-77f1-4950-baf8-a6332c8cb48c service cinder] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 16:11:26.577310 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 992/1961] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Aug 22 16:11:26 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 115 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 16:11:26.905698 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=82,keystone:DELETE=16,keystone:INSERT=2 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 16:11:27.358567 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68211d68-e2df-41df-8f0d-2f0831ee8a37 None None] Authenticating user token {{(pid=81767) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 16:11:27.359022 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53a1bbeb-77f1-4950-baf8-a6332c8cb48c service cinder] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 16:11:27.360649 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.fernet_utils [None req-68211d68-e2df-41df-8f0d-2f0831ee8a37 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81767) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 16:11:27.367846 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53a1bbeb-77f1-4950-baf8-a6332c8cb48c service cinder] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 16:11:27.371828 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce70a196-f713-4db6-b052-b81ebb270663 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79706d480f864998b9f8594be407ceab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4ee0413f9ef4d4fb1f95a660babe090', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 16:11:27.372428 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce70a196-f713-4db6-b052-b81ebb270663 service cinder] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:11:27.372704 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce70a196-f713-4db6-b052-b81ebb270663 service cinder] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:11:27.373005 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce70a196-f713-4db6-b052-b81ebb270663 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:11:27.429954 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce70a196-f713-4db6-b052-b81ebb270663 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 16:11:27.430990 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68211d68-e2df-41df-8f0d-2f0831ee8a37 None None] Validating token access rules against request {{(pid=81767) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 16:11:27.431491 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce70a196-f713-4db6-b052-b81ebb270663 service cinder] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 16:11:27.438987 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62c38c6f-9e41-4a12-9ce7-cd1fab8f719b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a98b1e4431940ab8d1fc95df2858a6c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81767) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 16:11:27.438987 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62c38c6f-9e41-4a12-9ce7-cd1fab8f719b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:11:27.438987 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62c38c6f-9e41-4a12-9ce7-cd1fab8f719b admin admin] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:11:27.438987 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62c38c6f-9e41-4a12-9ce7-cd1fab8f719b admin admin] PATH_INFO: `/v3/users/50c24fc9331c41d6ae4aa1274c6c491d` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:11:27.439515 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62c38c6f-9e41-4a12-9ce7-cd1fab8f719b admin admin] RBAC: Authorizing `identity:delete_user(user_id=50c24fc9331c41d6ae4aa1274c6c491d)` {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 16:11:27.441171 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62c38c6f-9e41-4a12-9ce7-cd1fab8f719b admin admin] RBAC: Authorization granted {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 16:11:27.453786 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 993/1962] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Aug 22 16:11:27 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 22 16:11:27.576082 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG dbcounter [-] [81767] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81767) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 16:11:27.677096 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.notifications [None req-62c38c6f-9e41-4a12-9ce7-cd1fab8f719b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '50c24fc9331c41d6ae4aa1274c6c491d'} {{(pid=81767) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 16:11:27.677775 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.notifications [None req-62c38c6f-9e41-4a12-9ce7-cd1fab8f719b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '50c24fc9331c41d6ae4aa1274c6c491d'} {{(pid=81767) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 16:11:27.782135 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.notifications [None req-62c38c6f-9e41-4a12-9ce7-cd1fab8f719b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '50c24fc9331c41d6ae4aa1274c6c491d'} {{(pid=81767) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 16:11:27.796323 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.notifications [None req-62c38c6f-9e41-4a12-9ce7-cd1fab8f719b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '50c24fc9331c41d6ae4aa1274c6c491d'} {{(pid=81767) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 16:11:27.810601 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.notifications [None req-62c38c6f-9e41-4a12-9ce7-cd1fab8f719b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '50c24fc9331c41d6ae4aa1274c6c491d'} {{(pid=81767) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 16:11:27.817195 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 970/1963] 10.222.0.15 () {58 vars in 1224 bytes} [Mon Aug 22 16:11:27 2022] DELETE /identity/v3/users/50c24fc9331c41d6ae4aa1274c6c491d => generated 0 bytes in 459 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 16:11:27.821662 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f02011f-5137-4848-b304-bec561a26898 None None] REQUEST_METHOD: `POST` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:11:27.821945 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f02011f-5137-4848-b304-bec561a26898 None None] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:11:27.822216 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f02011f-5137-4848-b304-bec561a26898 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:11:27.846891 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.auth.core [None req-8f02011f-5137-4848-b304-bec561a26898 None None] MFA Rules not processed for user `6b2199ae0fb0480aa917ada391f5ec54`. Rule list: `[]` (Enabled: `True`). {{(pid=81768) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 16:11:27.849587 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-8f02011f-5137-4848-b304-bec561a26898 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 16:11:27.928034 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 994/1964] 10.222.0.15 () {58 vars in 958 bytes} [Mon Aug 22 16:11:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 22 16:11:27.941812 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62c38c6f-9e41-4a12-9ce7-cd1fab8f719b admin admin] Authenticating user token {{(pid=81767) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 16:11:27.943595 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.fernet_utils [None req-62c38c6f-9e41-4a12-9ce7-cd1fab8f719b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81767) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 16:11:28.051608 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62c38c6f-9e41-4a12-9ce7-cd1fab8f719b admin admin] Validating token access rules against request {{(pid=81767) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 16:11:28.055853 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-714189df-e784-47e3-b3a0-d2fd005a0675 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '234687adad594f1aa45775a565ffbdc7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4ee0413f9ef4d4fb1f95a660babe090', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81767) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 16:11:28.059485 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-714189df-e784-47e3-b3a0-d2fd005a0675 service neutron] REQUEST_METHOD: `GET` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:11:28.059485 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-714189df-e784-47e3-b3a0-d2fd005a0675 service neutron] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:11:28.059485 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-714189df-e784-47e3-b3a0-d2fd005a0675 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:11:28.067941 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-714189df-e784-47e3-b3a0-d2fd005a0675 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 16:11:28.069383 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-714189df-e784-47e3-b3a0-d2fd005a0675 service neutron] RBAC: Authorization granted {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 16:11:28.080163 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 971/1965] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Aug 22 16:11:27 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 139 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 16:11:28.633888 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f02011f-5137-4848-b304-bec561a26898 None None] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 16:11:28.635860 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-8f02011f-5137-4848-b304-bec561a26898 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 16:11:28.649250 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f02011f-5137-4848-b304-bec561a26898 None None] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 16:11:28.665241 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2009e0a-beba-47e4-9a97-34733d71fa3d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a98b1e4431940ab8d1fc95df2858a6c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 16:11:28.666143 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2009e0a-beba-47e4-9a97-34733d71fa3d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:11:28.666143 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2009e0a-beba-47e4-9a97-34733d71fa3d admin admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:11:28.666406 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2009e0a-beba-47e4-9a97-34733d71fa3d admin admin] PATH_INFO: `/v3/projects/715274740943419bb2989d6a2e4ccb43` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:11:28.667921 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2009e0a-beba-47e4-9a97-34733d71fa3d admin admin] RBAC: Authorizing `identity:delete_project(project_id=715274740943419bb2989d6a2e4ccb43)` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 16:11:28.669823 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2009e0a-beba-47e4-9a97-34733d71fa3d admin admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 16:11:28.846961 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-a2009e0a-beba-47e4-9a97-34733d71fa3d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '715274740943419bb2989d6a2e4ccb43', 'request_id': 'req-a2009e0a-beba-47e4-9a97-34733d71fa3d'} {{(pid=81768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 16:11:28.852536 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-a2009e0a-beba-47e4-9a97-34733d71fa3d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '715274740943419bb2989d6a2e4ccb43', 'request_id': 'req-a2009e0a-beba-47e4-9a97-34733d71fa3d'} {{(pid=81768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 16:11:29.006061 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-a2009e0a-beba-47e4-9a97-34733d71fa3d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '715274740943419bb2989d6a2e4ccb43', 'request_id': 'req-a2009e0a-beba-47e4-9a97-34733d71fa3d'} {{(pid=81768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 16:11:29.021967 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-a2009e0a-beba-47e4-9a97-34733d71fa3d admin admin] The token cache is being invalidate because project 715274740943419bb2989d6a2e4ccb43 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81768) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 22 16:11:29.022488 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-a2009e0a-beba-47e4-9a97-34733d71fa3d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 16:11:29.023346 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-a2009e0a-beba-47e4-9a97-34733d71fa3d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 16:11:29.028827 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 995/1966] 10.222.0.15 () {58 vars in 1233 bytes} [Mon Aug 22 16:11:28 2022] DELETE /identity/v3/projects/715274740943419bb2989d6a2e4ccb43 => generated 0 bytes in 392 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 22 16:11:29.029331 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-714189df-e784-47e3-b3a0-d2fd005a0675 service neutron] Authenticating user token {{(pid=81767) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 16:11:29.040247 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-714189df-e784-47e3-b3a0-d2fd005a0675 service neutron] Validating token access rules against request {{(pid=81767) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 16:11:29.102125 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70b8fd08-3435-4884-a144-4961e9eaecb7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a98b1e4431940ab8d1fc95df2858a6c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81767) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 16:11:29.102818 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b8fd08-3435-4884-a144-4961e9eaecb7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:11:29.103164 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b8fd08-3435-4884-a144-4961e9eaecb7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:11:29.103455 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b8fd08-3435-4884-a144-4961e9eaecb7 admin admin] PATH_INFO: `/v3/users/aacec4652582498185152be8d8ecb78e` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:11:29.104622 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70b8fd08-3435-4884-a144-4961e9eaecb7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=aacec4652582498185152be8d8ecb78e)` {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 16:11:29.106525 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70b8fd08-3435-4884-a144-4961e9eaecb7 admin admin] RBAC: Authorization granted {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 16:11:29.393169 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.notifications [None req-70b8fd08-3435-4884-a144-4961e9eaecb7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'aacec4652582498185152be8d8ecb78e'} {{(pid=81767) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 16:11:29.394255 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.notifications [None req-70b8fd08-3435-4884-a144-4961e9eaecb7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'aacec4652582498185152be8d8ecb78e'} {{(pid=81767) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 16:11:29.501967 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.notifications [None req-70b8fd08-3435-4884-a144-4961e9eaecb7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'aacec4652582498185152be8d8ecb78e'} {{(pid=81767) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 16:11:29.512416 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.notifications [None req-70b8fd08-3435-4884-a144-4961e9eaecb7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'aacec4652582498185152be8d8ecb78e'} {{(pid=81767) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 16:11:29.527750 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.notifications [None req-70b8fd08-3435-4884-a144-4961e9eaecb7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'aacec4652582498185152be8d8ecb78e'} {{(pid=81767) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 16:11:29.534022 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 972/1967] 10.222.0.15 () {58 vars in 1224 bytes} [Mon Aug 22 16:11:29 2022] DELETE /identity/v3/users/aacec4652582498185152be8d8ecb78e => generated 0 bytes in 505 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 16:11:30.070846 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:DELETE=11,keystone:SELECT=88,keystone:INSERT=1 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 16:11:30.194789 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2009e0a-beba-47e4-9a97-34733d71fa3d admin admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 16:11:30.200972 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-a2009e0a-beba-47e4-9a97-34733d71fa3d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 16:11:30.229984 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2009e0a-beba-47e4-9a97-34733d71fa3d admin admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 16:11:30.313968 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfb345bf-e4df-4ae2-94a7-6de302d41e2b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a98b1e4431940ab8d1fc95df2858a6c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 16:11:30.314939 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb345bf-e4df-4ae2-94a7-6de302d41e2b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:11:30.317726 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb345bf-e4df-4ae2-94a7-6de302d41e2b admin admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:11:30.317726 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb345bf-e4df-4ae2-94a7-6de302d41e2b admin admin] PATH_INFO: `/v3/projects/aa9e894c11ae4185b56665c499399ea6` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:11:30.317726 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfb345bf-e4df-4ae2-94a7-6de302d41e2b admin admin] RBAC: Authorizing `identity:delete_project(project_id=aa9e894c11ae4185b56665c499399ea6)` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 16:11:30.320037 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfb345bf-e4df-4ae2-94a7-6de302d41e2b admin admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 16:11:30.454648 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG dbcounter [-] [81767] Writing DB stats keystone:SELECT=82,keystone:DELETE=16,keystone:INSERT=2 {{(pid=81767) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 16:11:30.474664 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04fc48b9-d34f-4caf-9a39-dc11c7ce3732 None None] REQUEST_METHOD: `POST` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:11:30.475162 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04fc48b9-d34f-4caf-9a39-dc11c7ce3732 None None] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:11:30.475528 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04fc48b9-d34f-4caf-9a39-dc11c7ce3732 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:11:30.495259 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-bfb345bf-e4df-4ae2-94a7-6de302d41e2b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'aa9e894c11ae4185b56665c499399ea6', 'request_id': 'req-bfb345bf-e4df-4ae2-94a7-6de302d41e2b'} {{(pid=81768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 16:11:30.499236 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.auth.core [None req-04fc48b9-d34f-4caf-9a39-dc11c7ce3732 None None] MFA Rules not processed for user `be4684a264444a52ac5488b5b2ea5ff1`. Rule list: `[]` (Enabled: `True`). {{(pid=81767) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 16:11:30.500830 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.fernet_utils [None req-04fc48b9-d34f-4caf-9a39-dc11c7ce3732 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81767) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 16:11:30.503379 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-bfb345bf-e4df-4ae2-94a7-6de302d41e2b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'aa9e894c11ae4185b56665c499399ea6', 'request_id': 'req-bfb345bf-e4df-4ae2-94a7-6de302d41e2b'} {{(pid=81768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 16:11:30.560416 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 973/1968] 10.222.0.15 () {58 vars in 958 bytes} [Mon Aug 22 16:11:30 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 22 16:11:30.594962 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-bfb345bf-e4df-4ae2-94a7-6de302d41e2b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'aa9e894c11ae4185b56665c499399ea6', 'request_id': 'req-bfb345bf-e4df-4ae2-94a7-6de302d41e2b'} {{(pid=81768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 16:11:30.616649 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-bfb345bf-e4df-4ae2-94a7-6de302d41e2b admin admin] The token cache is being invalidate because project aa9e894c11ae4185b56665c499399ea6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81768) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 22 16:11:30.617027 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-bfb345bf-e4df-4ae2-94a7-6de302d41e2b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 16:11:30.617890 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-bfb345bf-e4df-4ae2-94a7-6de302d41e2b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 16:11:30.620421 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 996/1969] 10.222.0.15 () {58 vars in 1233 bytes} [Mon Aug 22 16:11:30 2022] DELETE /identity/v3/projects/aa9e894c11ae4185b56665c499399ea6 => generated 0 bytes in 426 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 22 16:11:30.624698 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d535f996-77b1-4380-8f7d-91ff1c45e4d4 None None] REQUEST_METHOD: `POST` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:11:30.624979 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d535f996-77b1-4380-8f7d-91ff1c45e4d4 None None] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:11:30.625616 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d535f996-77b1-4380-8f7d-91ff1c45e4d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:11:30.650699 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.auth.core [None req-d535f996-77b1-4380-8f7d-91ff1c45e4d4 None None] MFA Rules not processed for user `6b2199ae0fb0480aa917ada391f5ec54`. Rule list: `[]` (Enabled: `True`). {{(pid=81767) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 16:11:30.652512 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.fernet_utils [None req-d535f996-77b1-4380-8f7d-91ff1c45e4d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81767) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 16:11:30.717347 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 974/1970] 10.222.0.15 () {58 vars in 958 bytes} [Mon Aug 22 16:11:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 16:11:30.725444 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5945ae4e-5bcb-450b-b3f1-cdb7a478f738 None None] REQUEST_METHOD: `POST` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:11:30.725989 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5945ae4e-5bcb-450b-b3f1-cdb7a478f738 None None] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:11:30.726265 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5945ae4e-5bcb-450b-b3f1-cdb7a478f738 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:11:30.750236 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.auth.core [None req-5945ae4e-5bcb-450b-b3f1-cdb7a478f738 None None] MFA Rules not processed for user `6b2199ae0fb0480aa917ada391f5ec54`. Rule list: `[]` (Enabled: `True`). {{(pid=81768) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 16:11:30.751589 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-5945ae4e-5bcb-450b-b3f1-cdb7a478f738 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 16:11:30.760904 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 997/1971] 10.222.0.15 () {58 vars in 958 bytes} [Mon Aug 22 16:11:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 16:11:30.765229 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d535f996-77b1-4380-8f7d-91ff1c45e4d4 None None] Authenticating user token {{(pid=81767) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 16:11:30.775565 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d535f996-77b1-4380-8f7d-91ff1c45e4d4 None None] Validating token access rules against request {{(pid=81767) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 16:11:30.780397 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4628b9f1-2b5f-4f23-b0ef-27725cea92f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a98b1e4431940ab8d1fc95df2858a6c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81767) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 16:11:30.781012 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4628b9f1-2b5f-4f23-b0ef-27725cea92f6 admin admin] REQUEST_METHOD: `GET` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:11:30.781289 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4628b9f1-2b5f-4f23-b0ef-27725cea92f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:11:30.781564 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4628b9f1-2b5f-4f23-b0ef-27725cea92f6 admin admin] PATH_INFO: `/v3/domains` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:11:30.782247 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4628b9f1-2b5f-4f23-b0ef-27725cea92f6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81767) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 16:11:30.782549 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4628b9f1-2b5f-4f23-b0ef-27725cea92f6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 16:11:30.784390 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4628b9f1-2b5f-4f23-b0ef-27725cea92f6 admin admin] RBAC: Authorization granted {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 16:11:30.802892 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 975/1972] 10.222.0.15 () {58 vars in 1153 bytes} [Mon Aug 22 16:11:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 16:11:30.804045 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5945ae4e-5bcb-450b-b3f1-cdb7a478f738 None None] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 16:11:30.813585 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5945ae4e-5bcb-450b-b3f1-cdb7a478f738 None None] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 16:11:30.818881 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a9c156a-56bc-4417-90be-87876aa3f5dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a98b1e4431940ab8d1fc95df2858a6c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 16:11:30.819561 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a9c156a-56bc-4417-90be-87876aa3f5dd admin admin] REQUEST_METHOD: `POST` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:11:30.819843 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a9c156a-56bc-4417-90be-87876aa3f5dd admin admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:11:30.820111 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a9c156a-56bc-4417-90be-87876aa3f5dd admin admin] PATH_INFO: `/v3/projects` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:11:30.820753 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a9c156a-56bc-4417-90be-87876aa3f5dd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 16:11:30.822733 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a9c156a-56bc-4417-90be-87876aa3f5dd admin admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 16:11:30.897894 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b11d62ba-0c11-45f4-9fb6-ae6fc1480b01 None None] REQUEST_METHOD: `POST` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:11:30.898325 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b11d62ba-0c11-45f4-9fb6-ae6fc1480b01 None None] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:11:30.898722 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b11d62ba-0c11-45f4-9fb6-ae6fc1480b01 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:11:30.905762 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application [None req-b11d62ba-0c11-45f4-9fb6-ae6fc1480b01 None None] Expecting to find domain in project. The server could not comply with the request since it is either malformed or otherwise incorrect. The client is assumed to be in error.: keystone.exception.ValidationError: Expecting to find domain in project. The server could not comply with the request since it is either malformed or otherwise incorrect. The client is assumed to be in error. Aug 22 16:11:30.905762 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 16:11:30.905762 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 16:11:30.905762 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 16:11:30.905762 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 16:11:30.905762 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 16:11:30.905762 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 16:11:30.905762 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 16:11:30.905762 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 16:11:30.905762 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 16:11:30.905762 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 16:11:30.905762 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 16:11:30.905762 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/server/flask/common.py", line 1064, in wrapper Aug 22 16:11:30.906682 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return f(*args, **kwargs) Aug 22 16:11:30.906682 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/auth.py", line 315, in post Aug 22 16:11:30.906682 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application token = authentication.authenticate_for_token(auth_data) Aug 22 16:11:30.906682 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/_shared/authentication.py", line 182, in authenticate_for_token Aug 22 16:11:30.906682 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application auth_info = core.AuthInfo.create(auth=auth) Aug 22 16:11:30.906682 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/auth/core.py", line 126, in create Aug 22 16:11:30.906682 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application auth_info._validate_and_normalize_auth_data(scope_only) Aug 22 16:11:30.906682 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/auth/core.py", line 340, in _validate_and_normalize_auth_data Aug 22 16:11:30.906682 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application self._validate_and_normalize_scope_data() Aug 22 16:11:30.906682 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/auth/core.py", line 290, in _validate_and_normalize_scope_data Aug 22 16:11:30.906682 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application project_ref = self._lookup_project(self.auth['scope']['project']) Aug 22 16:11:30.906682 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/auth/core.py", line 194, in _lookup_project Aug 22 16:11:30.906682 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application raise exception.ValidationError(attribute='domain', Aug 22 16:11:30.906682 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application keystone.exception.ValidationError: Expecting to find domain in project. The server could not comply with the request since it is either malformed or otherwise incorrect. The client is assumed to be in error. Aug 22 16:11:30.906682 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application Aug 22 16:11:30.907698 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 976/1973] 10.222.0.15 () {58 vars in 1009 bytes} [Mon Aug 22 16:11:30 2022] POST /identity/v3/auth/tokens => generated 230 bytes in 10 msecs (HTTP/1.1 400) 5 headers in 186 bytes (2 switches on core 0) Aug 22 16:11:30.935007 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 998/1974] 10.222.0.15 () {60 vars in 1153 bytes} [Mon Aug 22 16:11:30 2022] POST /identity/v3/projects => generated 379 bytes in 131 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 22 16:11:30.935555 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b11d62ba-0c11-45f4-9fb6-ae6fc1480b01 None None] Authenticating user token {{(pid=81767) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 16:11:31.005157 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b11d62ba-0c11-45f4-9fb6-ae6fc1480b01 None None] Validating token access rules against request {{(pid=81767) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 16:11:31.009811 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8006e131-7af7-4921-8e49-66fd67b2c79e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a98b1e4431940ab8d1fc95df2858a6c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81767) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 16:11:31.010287 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8006e131-7af7-4921-8e49-66fd67b2c79e admin admin] REQUEST_METHOD: `POST` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:11:31.010569 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8006e131-7af7-4921-8e49-66fd67b2c79e admin admin] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:11:31.010840 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8006e131-7af7-4921-8e49-66fd67b2c79e admin admin] PATH_INFO: `/v3/users` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:11:31.011515 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8006e131-7af7-4921-8e49-66fd67b2c79e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 16:11:31.013510 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8006e131-7af7-4921-8e49-66fd67b2c79e admin admin] RBAC: Authorization granted {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 16:11:31.171697 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 977/1975] 10.222.0.15 () {60 vars in 1144 bytes} [Mon Aug 22 16:11:30 2022] POST /identity/v3/users => generated 290 bytes in 237 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 22 16:11:31.177112 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a9c156a-56bc-4417-90be-87876aa3f5dd admin admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 16:11:31.189610 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a9c156a-56bc-4417-90be-87876aa3f5dd admin admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 16:11:31.194459 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e38c586-e481-4d25-b5ab-72fc21344624 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a98b1e4431940ab8d1fc95df2858a6c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 16:11:31.195074 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e38c586-e481-4d25-b5ab-72fc21344624 admin admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:11:31.195410 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e38c586-e481-4d25-b5ab-72fc21344624 admin admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:11:31.195662 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e38c586-e481-4d25-b5ab-72fc21344624 admin admin] PATH_INFO: `/v3/roles` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:11:31.196339 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e38c586-e481-4d25-b5ab-72fc21344624 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 16:11:31.198044 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e38c586-e481-4d25-b5ab-72fc21344624 admin admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 16:11:31.209209 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 999/1976] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Aug 22 16:11:31 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 22 16:11:31.214257 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8006e131-7af7-4921-8e49-66fd67b2c79e admin admin] Authenticating user token {{(pid=81767) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 16:11:31.225533 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8006e131-7af7-4921-8e49-66fd67b2c79e admin admin] Validating token access rules against request {{(pid=81767) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 16:11:31.230350 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b69664f6-5c53-460a-a5cb-f1eaba7be1ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a98b1e4431940ab8d1fc95df2858a6c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81767) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 16:11:31.230957 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b69664f6-5c53-460a-a5cb-f1eaba7be1ca admin admin] REQUEST_METHOD: `GET` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:11:31.231235 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b69664f6-5c53-460a-a5cb-f1eaba7be1ca admin admin] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:11:31.231738 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b69664f6-5c53-460a-a5cb-f1eaba7be1ca admin admin] PATH_INFO: `/v3/roles` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:11:31.232180 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b69664f6-5c53-460a-a5cb-f1eaba7be1ca admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 16:11:31.233777 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b69664f6-5c53-460a-a5cb-f1eaba7be1ca admin admin] RBAC: Authorization granted {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 16:11:31.244504 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 978/1977] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Aug 22 16:11:31 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 22 16:11:31.248738 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e38c586-e481-4d25-b5ab-72fc21344624 admin admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 16:11:31.267190 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e38c586-e481-4d25-b5ab-72fc21344624 admin admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 16:11:31.268298 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-716efd2b-5bc0-44dc-a17b-545b88c82be2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a98b1e4431940ab8d1fc95df2858a6c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 16:11:31.272501 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-716efd2b-5bc0-44dc-a17b-545b88c82be2 admin admin] REQUEST_METHOD: `PUT` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:11:31.272805 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-716efd2b-5bc0-44dc-a17b-545b88c82be2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:11:31.273059 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-716efd2b-5bc0-44dc-a17b-545b88c82be2 admin admin] PATH_INFO: `/v3/projects/49ed116bd33d456ba7f6256f34462ffc/users/6d4136763ffe47249d865f3ff0e3adb1/roles/e6fc1ee3fb974125ba9b4fe55b12b45e` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:11:31.290185 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-716efd2b-5bc0-44dc-a17b-545b88c82be2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=49ed116bd33d456ba7f6256f34462ffc, user_id=6d4136763ffe47249d865f3ff0e3adb1, role_id=e6fc1ee3fb974125ba9b4fe55b12b45e)` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 16:11:31.293206 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-716efd2b-5bc0-44dc-a17b-545b88c82be2 admin admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 16:11:31.396943 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 1000/1978] 10.222.0.15 () {60 vars in 1483 bytes} [Mon Aug 22 16:11:31 2022] PUT /identity/v3/projects/49ed116bd33d456ba7f6256f34462ffc/users/6d4136763ffe47249d865f3ff0e3adb1/roles/e6fc1ee3fb974125ba9b4fe55b12b45e => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 16:11:31.407927 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af04bb95-d3a4-4052-afd4-9946c43d58ec None None] REQUEST_METHOD: `POST` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:11:31.408473 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af04bb95-d3a4-4052-afd4-9946c43d58ec None None] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:11:31.408681 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af04bb95-d3a4-4052-afd4-9946c43d58ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:11:31.437390 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.auth.core [None req-af04bb95-d3a4-4052-afd4-9946c43d58ec None None] MFA Rules not processed for user `6d4136763ffe47249d865f3ff0e3adb1`. Rule list: `[]` (Enabled: `True`). {{(pid=81767) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 16:11:31.438616 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.fernet_utils [None req-af04bb95-d3a4-4052-afd4-9946c43d58ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81767) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 16:11:31.521642 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 979/1979] 10.222.0.15 () {58 vars in 958 bytes} [Mon Aug 22 16:11:31 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 16:11:31.526965 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-716efd2b-5bc0-44dc-a17b-545b88c82be2 admin admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 16:11:31.614607 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-716efd2b-5bc0-44dc-a17b-545b88c82be2 admin admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 16:11:31.619883 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8047fa9d-60f1-4980-aac4-9ad660b8724b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a98b1e4431940ab8d1fc95df2858a6c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 16:11:31.620750 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8047fa9d-60f1-4980-aac4-9ad660b8724b admin admin] REQUEST_METHOD: `POST` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:11:31.621559 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8047fa9d-60f1-4980-aac4-9ad660b8724b admin admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:11:31.622049 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8047fa9d-60f1-4980-aac4-9ad660b8724b admin admin] PATH_INFO: `/v3/projects` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:11:31.622602 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8047fa9d-60f1-4980-aac4-9ad660b8724b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 16:11:31.624570 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8047fa9d-60f1-4980-aac4-9ad660b8724b admin admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 16:11:31.635057 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af04bb95-d3a4-4052-afd4-9946c43d58ec None None] Authenticating user token {{(pid=81767) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 16:11:31.637617 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.fernet_utils [None req-af04bb95-d3a4-4052-afd4-9946c43d58ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81767) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 16:11:31.657322 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af04bb95-d3a4-4052-afd4-9946c43d58ec None None] Validating token access rules against request {{(pid=81767) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 16:11:31.662834 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-260ee276-c449-460d-a2cc-14ba037e2411 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a98b1e4431940ab8d1fc95df2858a6c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81767) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 16:11:31.664703 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-260ee276-c449-460d-a2cc-14ba037e2411 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:11:31.665344 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-260ee276-c449-460d-a2cc-14ba037e2411 admin admin] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:11:31.668305 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-260ee276-c449-460d-a2cc-14ba037e2411 admin admin] PATH_INFO: `/v3/users/28a2242d4d894a87922d4e0a3fac6d11` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:11:31.668305 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-260ee276-c449-460d-a2cc-14ba037e2411 admin admin] RBAC: Authorizing `identity:delete_user(user_id=28a2242d4d894a87922d4e0a3fac6d11)` {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 16:11:31.669865 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-260ee276-c449-460d-a2cc-14ba037e2411 admin admin] RBAC: Authorization granted {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 16:11:31.758106 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 1001/1980] 10.222.0.15 () {60 vars in 1153 bytes} [Mon Aug 22 16:11:31 2022] POST /identity/v3/projects => generated 377 bytes in 229 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 22 16:11:31.759516 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8047fa9d-60f1-4980-aac4-9ad660b8724b admin admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 16:11:31.833439 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8047fa9d-60f1-4980-aac4-9ad660b8724b admin admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 16:11:31.838490 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebb7b3d3-b3e1-47fe-8501-51579e00da95 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a98b1e4431940ab8d1fc95df2858a6c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 16:11:31.839108 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebb7b3d3-b3e1-47fe-8501-51579e00da95 admin admin] REQUEST_METHOD: `POST` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:11:31.839427 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebb7b3d3-b3e1-47fe-8501-51579e00da95 admin admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:11:31.839700 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebb7b3d3-b3e1-47fe-8501-51579e00da95 admin admin] PATH_INFO: `/v3/users` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:11:31.840341 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebb7b3d3-b3e1-47fe-8501-51579e00da95 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 16:11:31.842372 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebb7b3d3-b3e1-47fe-8501-51579e00da95 admin admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 16:11:31.992639 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.notifications [None req-260ee276-c449-460d-a2cc-14ba037e2411 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '28a2242d4d894a87922d4e0a3fac6d11'} {{(pid=81767) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 16:11:31.992639 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.notifications [None req-260ee276-c449-460d-a2cc-14ba037e2411 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '28a2242d4d894a87922d4e0a3fac6d11'} {{(pid=81767) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 16:11:32.008696 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 1002/1981] 10.222.0.15 () {60 vars in 1144 bytes} [Mon Aug 22 16:11:31 2022] POST /identity/v3/users => generated 289 bytes in 247 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 22 16:11:32.008696 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebb7b3d3-b3e1-47fe-8501-51579e00da95 admin admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 16:11:32.010067 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-ebb7b3d3-b3e1-47fe-8501-51579e00da95 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 16:11:32.023104 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebb7b3d3-b3e1-47fe-8501-51579e00da95 admin admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 16:11:32.028828 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41898351-e402-453b-9e94-78892bc1a649 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a98b1e4431940ab8d1fc95df2858a6c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 16:11:32.028981 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41898351-e402-453b-9e94-78892bc1a649 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:11:32.029220 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41898351-e402-453b-9e94-78892bc1a649 admin admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:11:32.029501 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41898351-e402-453b-9e94-78892bc1a649 admin admin] PATH_INFO: `/v3/users/dae6669681b540e9be606226ff80dadd` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:11:32.030861 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41898351-e402-453b-9e94-78892bc1a649 admin admin] RBAC: Authorizing `identity:delete_user(user_id=dae6669681b540e9be606226ff80dadd)` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 16:11:32.033524 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41898351-e402-453b-9e94-78892bc1a649 admin admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 16:11:32.072699 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.notifications [None req-260ee276-c449-460d-a2cc-14ba037e2411 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '28a2242d4d894a87922d4e0a3fac6d11'} {{(pid=81767) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 16:11:32.080302 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.notifications [None req-260ee276-c449-460d-a2cc-14ba037e2411 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '28a2242d4d894a87922d4e0a3fac6d11'} {{(pid=81767) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 16:11:32.093884 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.notifications [None req-260ee276-c449-460d-a2cc-14ba037e2411 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '28a2242d4d894a87922d4e0a3fac6d11'} {{(pid=81767) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 16:11:32.101886 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 980/1982] 10.222.0.15 () {58 vars in 1224 bytes} [Mon Aug 22 16:11:31 2022] DELETE /identity/v3/users/28a2242d4d894a87922d4e0a3fac6d11 => generated 0 bytes in 468 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 16:11:32.107887 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-260ee276-c449-460d-a2cc-14ba037e2411 admin admin] Authenticating user token {{(pid=81767) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 16:11:32.107887 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.fernet_utils [None req-260ee276-c449-460d-a2cc-14ba037e2411 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81767) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 16:11:32.170181 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-260ee276-c449-460d-a2cc-14ba037e2411 admin admin] Validating token access rules against request {{(pid=81767) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 16:11:32.176462 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3ea5326-ba9b-4f21-bb9b-99f5d5259fb8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a98b1e4431940ab8d1fc95df2858a6c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81767) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 16:11:32.177092 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ea5326-ba9b-4f21-bb9b-99f5d5259fb8 admin admin] REQUEST_METHOD: `GET` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:11:32.177375 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ea5326-ba9b-4f21-bb9b-99f5d5259fb8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:11:32.177650 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ea5326-ba9b-4f21-bb9b-99f5d5259fb8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:11:32.178299 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3ea5326-ba9b-4f21-bb9b-99f5d5259fb8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 16:11:32.179954 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3ea5326-ba9b-4f21-bb9b-99f5d5259fb8 admin admin] RBAC: Authorization granted {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 16:11:32.190171 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 981/1983] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Aug 22 16:11:32 2022] GET /identity/v3/roles => generated 1420 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 22 16:11:32.191484 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-244e99f0-ee74-451b-8763-cf62becbfc60 None None] REQUEST_METHOD: `POST` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:11:32.191770 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-244e99f0-ee74-451b-8763-cf62becbfc60 None None] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:11:32.192041 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-244e99f0-ee74-451b-8763-cf62becbfc60 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:11:32.210877 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.auth.core [None req-244e99f0-ee74-451b-8763-cf62becbfc60 None None] MFA Rules not processed for user `6b2199ae0fb0480aa917ada391f5ec54`. Rule list: `[]` (Enabled: `True`). {{(pid=81767) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 16:11:32.214503 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.fernet_utils [None req-244e99f0-ee74-451b-8763-cf62becbfc60 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81767) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 16:11:32.219763 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 982/1984] 10.222.0.15 () {58 vars in 958 bytes} [Mon Aug 22 16:11:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 16:11:32.223573 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-244e99f0-ee74-451b-8763-cf62becbfc60 None None] Authenticating user token {{(pid=81767) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 16:11:32.230855 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-244e99f0-ee74-451b-8763-cf62becbfc60 None None] Validating token access rules against request {{(pid=81767) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 16:11:32.235914 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41157f2f-75ae-449d-b8ac-502f8d65c0fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a98b1e4431940ab8d1fc95df2858a6c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81767) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 16:11:32.236415 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41157f2f-75ae-449d-b8ac-502f8d65c0fb admin admin] REQUEST_METHOD: `GET` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:11:32.236697 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41157f2f-75ae-449d-b8ac-502f8d65c0fb admin admin] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:11:32.236968 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41157f2f-75ae-449d-b8ac-502f8d65c0fb admin admin] PATH_INFO: `/v3/roles` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:11:32.237800 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41157f2f-75ae-449d-b8ac-502f8d65c0fb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 16:11:32.240104 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41157f2f-75ae-449d-b8ac-502f8d65c0fb admin admin] RBAC: Authorization granted {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 16:11:32.248674 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-41898351-e402-453b-9e94-78892bc1a649 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'dae6669681b540e9be606226ff80dadd'} {{(pid=81768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 16:11:32.249495 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-41898351-e402-453b-9e94-78892bc1a649 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'dae6669681b540e9be606226ff80dadd'} {{(pid=81768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 16:11:32.250163 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-41898351-e402-453b-9e94-78892bc1a649 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'dae6669681b540e9be606226ff80dadd'} {{(pid=81768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 16:11:32.250872 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 983/1985] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Aug 22 16:11:32 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 22 16:11:32.251806 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41157f2f-75ae-449d-b8ac-502f8d65c0fb admin admin] Authenticating user token {{(pid=81767) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 16:11:32.253433 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.fernet_utils [None req-41157f2f-75ae-449d-b8ac-502f8d65c0fb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81767) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 16:11:32.258074 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-41898351-e402-453b-9e94-78892bc1a649 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'dae6669681b540e9be606226ff80dadd'} {{(pid=81768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 16:11:32.311072 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41157f2f-75ae-449d-b8ac-502f8d65c0fb admin admin] Validating token access rules against request {{(pid=81767) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 16:11:32.314663 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6f5ce7e-218d-43fa-8f75-ea75844e6d40 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '234687adad594f1aa45775a565ffbdc7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4ee0413f9ef4d4fb1f95a660babe090', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81767) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 16:11:32.315263 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6f5ce7e-218d-43fa-8f75-ea75844e6d40 service neutron] REQUEST_METHOD: `GET` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:11:32.315567 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6f5ce7e-218d-43fa-8f75-ea75844e6d40 service neutron] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:11:32.315840 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6f5ce7e-218d-43fa-8f75-ea75844e6d40 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:11:32.318107 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.fernet_utils [None req-b6f5ce7e-218d-43fa-8f75-ea75844e6d40 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81767) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 16:11:32.329890 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6f5ce7e-218d-43fa-8f75-ea75844e6d40 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 16:11:32.331341 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6f5ce7e-218d-43fa-8f75-ea75844e6d40 service neutron] RBAC: Authorization granted {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 16:11:32.342213 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 984/1986] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Aug 22 16:11:32 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 16:11:32.343027 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6f5ce7e-218d-43fa-8f75-ea75844e6d40 service neutron] Authenticating user token {{(pid=81767) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 16:11:32.344557 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.fernet_utils [None req-b6f5ce7e-218d-43fa-8f75-ea75844e6d40 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81767) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 16:11:32.353421 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-41898351-e402-453b-9e94-78892bc1a649 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'dae6669681b540e9be606226ff80dadd'} {{(pid=81768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 16:11:32.359977 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=90,keystone:INSERT=4,keystone:DELETE=6 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 16:11:32.369868 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6f5ce7e-218d-43fa-8f75-ea75844e6d40 service neutron] Validating token access rules against request {{(pid=81767) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 16:11:32.377746 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fff929a3-88d7-4399-aa61-e164cd670308 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a98b1e4431940ab8d1fc95df2858a6c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81767) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 16:11:32.377746 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff929a3-88d7-4399-aa61-e164cd670308 admin admin] REQUEST_METHOD: `PUT` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:11:32.377746 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff929a3-88d7-4399-aa61-e164cd670308 admin admin] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:11:32.377746 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff929a3-88d7-4399-aa61-e164cd670308 admin admin] PATH_INFO: `/v3/projects/cd9cd02faaa64a2d89782aba11c73b22/users/a88aa134951a4a139562609c27e14bc0/roles/e6fc1ee3fb974125ba9b4fe55b12b45e` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:11:32.383186 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 1003/1987] 10.222.0.15 () {58 vars in 1224 bytes} [Mon Aug 22 16:11:32 2022] DELETE /identity/v3/users/dae6669681b540e9be606226ff80dadd => generated 0 bytes in 377 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 16:11:32.399595 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fff929a3-88d7-4399-aa61-e164cd670308 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cd9cd02faaa64a2d89782aba11c73b22, user_id=a88aa134951a4a139562609c27e14bc0, role_id=e6fc1ee3fb974125ba9b4fe55b12b45e)` {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 16:11:32.402508 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fff929a3-88d7-4399-aa61-e164cd670308 admin admin] RBAC: Authorization granted {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 16:11:32.404629 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc9af832-5111-4c17-8066-0c76e4470eab None None] REQUEST_METHOD: `POST` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:11:32.406192 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc9af832-5111-4c17-8066-0c76e4470eab None None] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:11:32.406480 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc9af832-5111-4c17-8066-0c76e4470eab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:11:32.442763 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.auth.core [None req-cc9af832-5111-4c17-8066-0c76e4470eab None None] MFA Rules not processed for user `6b2199ae0fb0480aa917ada391f5ec54`. Rule list: `[]` (Enabled: `True`). {{(pid=81768) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 16:11:32.444193 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-cc9af832-5111-4c17-8066-0c76e4470eab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 16:11:32.496407 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 985/1988] 10.222.0.15 () {60 vars in 1483 bytes} [Mon Aug 22 16:11:32 2022] PUT /identity/v3/projects/cd9cd02faaa64a2d89782aba11c73b22/users/a88aa134951a4a139562609c27e14bc0/roles/e6fc1ee3fb974125ba9b4fe55b12b45e => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 16:11:32.509193 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 1004/1989] 10.222.0.15 () {58 vars in 958 bytes} [Mon Aug 22 16:11:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 22 16:11:32.509418 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f55a66c-af5d-4602-9070-bbcba0f2360f None None] REQUEST_METHOD: `POST` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:11:32.509418 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f55a66c-af5d-4602-9070-bbcba0f2360f None None] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:11:32.509591 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f55a66c-af5d-4602-9070-bbcba0f2360f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:11:32.524016 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc9af832-5111-4c17-8066-0c76e4470eab None None] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 16:11:32.546572 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.auth.core [None req-4f55a66c-af5d-4602-9070-bbcba0f2360f None None] MFA Rules not processed for user `a88aa134951a4a139562609c27e14bc0`. Rule list: `[]` (Enabled: `True`). {{(pid=81767) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 16:11:32.547903 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.fernet_utils [None req-4f55a66c-af5d-4602-9070-bbcba0f2360f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81767) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 16:11:32.628475 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc9af832-5111-4c17-8066-0c76e4470eab None None] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 16:11:32.638713 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG dbcounter [-] [81767] Writing DB stats keystone:SELECT=95,keystone:DELETE=1,keystone:INSERT=4 {{(pid=81767) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 16:11:32.665110 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 986/1990] 10.222.0.15 () {58 vars in 958 bytes} [Mon Aug 22 16:11:32 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 22 16:11:32.669862 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f55a66c-af5d-4602-9070-bbcba0f2360f None None] Authenticating user token {{(pid=81767) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 16:11:32.694671 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97a4f5b9-80eb-4e32-89eb-4e621f1fc1bd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '234687adad594f1aa45775a565ffbdc7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4ee0413f9ef4d4fb1f95a660babe090', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 16:11:32.695425 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97a4f5b9-80eb-4e32-89eb-4e621f1fc1bd service neutron] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:11:32.695779 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97a4f5b9-80eb-4e32-89eb-4e621f1fc1bd service neutron] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:11:32.696119 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97a4f5b9-80eb-4e32-89eb-4e621f1fc1bd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:11:32.749568 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f55a66c-af5d-4602-9070-bbcba0f2360f None None] Validating token access rules against request {{(pid=81767) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 16:11:32.757078 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2983c1f-6880-48ff-80fe-a6f79cde5e2b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a98b1e4431940ab8d1fc95df2858a6c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81767) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 16:11:32.757694 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2983c1f-6880-48ff-80fe-a6f79cde5e2b admin admin] REQUEST_METHOD: `POST` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:11:32.757981 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2983c1f-6880-48ff-80fe-a6f79cde5e2b admin admin] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:11:32.758244 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2983c1f-6880-48ff-80fe-a6f79cde5e2b admin admin] PATH_INFO: `/v3/projects` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:11:32.758925 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2983c1f-6880-48ff-80fe-a6f79cde5e2b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 16:11:32.761211 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2983c1f-6880-48ff-80fe-a6f79cde5e2b admin admin] RBAC: Authorization granted {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 16:11:32.771297 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97a4f5b9-80eb-4e32-89eb-4e621f1fc1bd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 16:11:32.773044 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97a4f5b9-80eb-4e32-89eb-4e621f1fc1bd service neutron] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 16:11:32.784508 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 1005/1991] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Aug 22 16:11:32 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 262 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 22 16:11:32.851378 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97a4f5b9-80eb-4e32-89eb-4e621f1fc1bd service neutron] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 16:11:32.853990 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-97a4f5b9-80eb-4e32-89eb-4e621f1fc1bd service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 16:11:32.878664 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97a4f5b9-80eb-4e32-89eb-4e621f1fc1bd service neutron] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 16:11:32.879883 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 987/1992] 10.222.0.15 () {60 vars in 1153 bytes} [Mon Aug 22 16:11:32 2022] POST /identity/v3/projects => generated 377 bytes in 210 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 22 16:11:32.883911 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2983c1f-6880-48ff-80fe-a6f79cde5e2b admin admin] Authenticating user token {{(pid=81767) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 16:11:32.947330 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-828c4fd6-2ad7-45f4-b40a-22c1d5162f75 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a98b1e4431940ab8d1fc95df2858a6c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 16:11:32.947977 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-828c4fd6-2ad7-45f4-b40a-22c1d5162f75 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:11:32.948255 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-828c4fd6-2ad7-45f4-b40a-22c1d5162f75 admin admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:11:32.948528 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-828c4fd6-2ad7-45f4-b40a-22c1d5162f75 admin admin] PATH_INFO: `/v3/projects/253d69d423624a3aae7a58f08aabb941` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:11:32.949695 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-828c4fd6-2ad7-45f4-b40a-22c1d5162f75 admin admin] RBAC: Authorizing `identity:delete_project(project_id=253d69d423624a3aae7a58f08aabb941)` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 16:11:32.951644 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-828c4fd6-2ad7-45f4-b40a-22c1d5162f75 admin admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 16:11:32.961050 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2983c1f-6880-48ff-80fe-a6f79cde5e2b admin admin] Validating token access rules against request {{(pid=81767) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 16:11:32.966462 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-241c0b37-6fb4-4c55-b48d-30f1c144f7b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a98b1e4431940ab8d1fc95df2858a6c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81767) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 16:11:32.967173 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-241c0b37-6fb4-4c55-b48d-30f1c144f7b7 admin admin] REQUEST_METHOD: `POST` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:11:32.967541 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-241c0b37-6fb4-4c55-b48d-30f1c144f7b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:11:32.967875 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-241c0b37-6fb4-4c55-b48d-30f1c144f7b7 admin admin] PATH_INFO: `/v3/users` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:11:32.968561 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-241c0b37-6fb4-4c55-b48d-30f1c144f7b7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 16:11:32.970551 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-241c0b37-6fb4-4c55-b48d-30f1c144f7b7 admin admin] RBAC: Authorization granted {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 16:11:33.052068 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 988/1993] 10.222.0.15 () {60 vars in 1144 bytes} [Mon Aug 22 16:11:32 2022] POST /identity/v3/users => generated 295 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 22 16:11:33.053743 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-241c0b37-6fb4-4c55-b48d-30f1c144f7b7 admin admin] Authenticating user token {{(pid=81767) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 16:11:33.061619 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-828c4fd6-2ad7-45f4-b40a-22c1d5162f75 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '253d69d423624a3aae7a58f08aabb941', 'request_id': 'req-828c4fd6-2ad7-45f4-b40a-22c1d5162f75'} {{(pid=81768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 16:11:33.065203 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-241c0b37-6fb4-4c55-b48d-30f1c144f7b7 admin admin] Validating token access rules against request {{(pid=81767) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 16:11:33.069451 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-828c4fd6-2ad7-45f4-b40a-22c1d5162f75 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '253d69d423624a3aae7a58f08aabb941', 'request_id': 'req-828c4fd6-2ad7-45f4-b40a-22c1d5162f75'} {{(pid=81768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 16:11:33.137617 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dae02a4-e866-4990-be25-aea18de4b75b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a98b1e4431940ab8d1fc95df2858a6c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81767) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 16:11:33.138257 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dae02a4-e866-4990-be25-aea18de4b75b admin admin] REQUEST_METHOD: `GET` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:11:33.138543 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dae02a4-e866-4990-be25-aea18de4b75b admin admin] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:11:33.138817 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dae02a4-e866-4990-be25-aea18de4b75b admin admin] PATH_INFO: `/v3/roles` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:11:33.139502 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dae02a4-e866-4990-be25-aea18de4b75b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 16:11:33.141090 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dae02a4-e866-4990-be25-aea18de4b75b admin admin] RBAC: Authorization granted {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 16:11:33.151848 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 989/1994] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Aug 22 16:11:33 2022] GET /identity/v3/roles => generated 1420 bytes in 99 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 22 16:11:33.156242 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dae02a4-e866-4990-be25-aea18de4b75b admin admin] Authenticating user token {{(pid=81767) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 16:11:33.167355 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dae02a4-e866-4990-be25-aea18de4b75b admin admin] Validating token access rules against request {{(pid=81767) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 16:11:33.172265 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75796b28-8fed-4310-b72d-43ebc6390bf1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a98b1e4431940ab8d1fc95df2858a6c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81767) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 16:11:33.172862 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75796b28-8fed-4310-b72d-43ebc6390bf1 admin admin] REQUEST_METHOD: `PUT` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:11:33.173107 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75796b28-8fed-4310-b72d-43ebc6390bf1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:11:33.173379 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75796b28-8fed-4310-b72d-43ebc6390bf1 admin admin] PATH_INFO: `/v3/projects/a23886ab1d2a48f3a0a21077d585f1a0/users/b527a92a1be840a19745f067f2729477/roles/9d9848cfd22e4b089db443c350e56670` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:11:33.174270 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-828c4fd6-2ad7-45f4-b40a-22c1d5162f75 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '253d69d423624a3aae7a58f08aabb941', 'request_id': 'req-828c4fd6-2ad7-45f4-b40a-22c1d5162f75'} {{(pid=81768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 16:11:33.188955 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-828c4fd6-2ad7-45f4-b40a-22c1d5162f75 admin admin] The token cache is being invalidate because project 253d69d423624a3aae7a58f08aabb941 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81768) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 22 16:11:33.189425 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-828c4fd6-2ad7-45f4-b40a-22c1d5162f75 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 16:11:33.189586 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75796b28-8fed-4310-b72d-43ebc6390bf1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a23886ab1d2a48f3a0a21077d585f1a0, user_id=b527a92a1be840a19745f067f2729477, role_id=9d9848cfd22e4b089db443c350e56670)` {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 16:11:33.190337 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-828c4fd6-2ad7-45f4-b40a-22c1d5162f75 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 16:11:33.192298 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 1006/1995] 10.222.0.15 () {58 vars in 1233 bytes} [Mon Aug 22 16:11:32 2022] DELETE /identity/v3/projects/253d69d423624a3aae7a58f08aabb941 => generated 0 bytes in 341 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 22 16:11:33.192409 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75796b28-8fed-4310-b72d-43ebc6390bf1 admin admin] RBAC: Authorization granted {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 16:11:33.196039 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-828c4fd6-2ad7-45f4-b40a-22c1d5162f75 admin admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 16:11:33.197397 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-828c4fd6-2ad7-45f4-b40a-22c1d5162f75 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 16:11:33.212784 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-828c4fd6-2ad7-45f4-b40a-22c1d5162f75 admin admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 16:11:33.217404 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63fe9357-ccf0-442d-b1c5-5f9c077bc57a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a98b1e4431940ab8d1fc95df2858a6c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 16:11:33.218083 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63fe9357-ccf0-442d-b1c5-5f9c077bc57a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:11:33.218358 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63fe9357-ccf0-442d-b1c5-5f9c077bc57a admin admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:11:33.218628 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63fe9357-ccf0-442d-b1c5-5f9c077bc57a admin admin] PATH_INFO: `/v3/users/e4caeae7c3f840858460d8243e706210` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:11:33.219816 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63fe9357-ccf0-442d-b1c5-5f9c077bc57a admin admin] RBAC: Authorizing `identity:delete_user(user_id=e4caeae7c3f840858460d8243e706210)` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 16:11:33.221750 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63fe9357-ccf0-442d-b1c5-5f9c077bc57a admin admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 16:11:33.319894 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 990/1996] 10.222.0.15 () {60 vars in 1483 bytes} [Mon Aug 22 16:11:33 2022] PUT /identity/v3/projects/a23886ab1d2a48f3a0a21077d585f1a0/users/b527a92a1be840a19745f067f2729477/roles/9d9848cfd22e4b089db443c350e56670 => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 16:11:33.325818 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75796b28-8fed-4310-b72d-43ebc6390bf1 admin admin] Authenticating user token {{(pid=81767) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 16:11:33.325818 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.fernet_utils [None req-75796b28-8fed-4310-b72d-43ebc6390bf1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81767) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 16:11:33.390916 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75796b28-8fed-4310-b72d-43ebc6390bf1 admin admin] Validating token access rules against request {{(pid=81767) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 16:11:33.393238 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bf9fba1-b0b9-4d7d-92d6-eef47ef75fd8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a98b1e4431940ab8d1fc95df2858a6c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81767) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 16:11:33.394052 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bf9fba1-b0b9-4d7d-92d6-eef47ef75fd8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:11:33.394137 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bf9fba1-b0b9-4d7d-92d6-eef47ef75fd8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:11:33.394431 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bf9fba1-b0b9-4d7d-92d6-eef47ef75fd8 admin admin] PATH_INFO: `/v3/projects/9404a43483c44564a0dcf7d5895c3b08` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:11:33.395595 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bf9fba1-b0b9-4d7d-92d6-eef47ef75fd8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9404a43483c44564a0dcf7d5895c3b08)` {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 16:11:33.397561 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bf9fba1-b0b9-4d7d-92d6-eef47ef75fd8 admin admin] RBAC: Authorization granted {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 16:11:33.498524 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-63fe9357-ccf0-442d-b1c5-5f9c077bc57a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e4caeae7c3f840858460d8243e706210'} {{(pid=81768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 16:11:33.499164 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-63fe9357-ccf0-442d-b1c5-5f9c077bc57a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e4caeae7c3f840858460d8243e706210'} {{(pid=81768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 16:11:33.500584 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-63fe9357-ccf0-442d-b1c5-5f9c077bc57a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e4caeae7c3f840858460d8243e706210'} {{(pid=81768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 16:11:33.508770 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-63fe9357-ccf0-442d-b1c5-5f9c077bc57a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e4caeae7c3f840858460d8243e706210'} {{(pid=81768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 16:11:33.532466 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.notifications [None req-5bf9fba1-b0b9-4d7d-92d6-eef47ef75fd8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9404a43483c44564a0dcf7d5895c3b08', 'request_id': 'req-5bf9fba1-b0b9-4d7d-92d6-eef47ef75fd8'} {{(pid=81767) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 16:11:33.538324 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.notifications [None req-5bf9fba1-b0b9-4d7d-92d6-eef47ef75fd8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9404a43483c44564a0dcf7d5895c3b08', 'request_id': 'req-5bf9fba1-b0b9-4d7d-92d6-eef47ef75fd8'} {{(pid=81767) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 16:11:33.633100 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-63fe9357-ccf0-442d-b1c5-5f9c077bc57a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e4caeae7c3f840858460d8243e706210'} {{(pid=81768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 16:11:33.658758 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 1007/1997] 10.222.0.15 () {58 vars in 1224 bytes} [Mon Aug 22 16:11:33 2022] DELETE /identity/v3/users/e4caeae7c3f840858460d8243e706210 => generated 0 bytes in 460 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 22 16:11:33.658758 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63fe9357-ccf0-442d-b1c5-5f9c077bc57a admin admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 16:11:33.658758 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-63fe9357-ccf0-442d-b1c5-5f9c077bc57a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 16:11:33.697800 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.notifications [None req-5bf9fba1-b0b9-4d7d-92d6-eef47ef75fd8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9404a43483c44564a0dcf7d5895c3b08', 'request_id': 'req-5bf9fba1-b0b9-4d7d-92d6-eef47ef75fd8'} {{(pid=81767) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 16:11:33.717599 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.notifications [None req-5bf9fba1-b0b9-4d7d-92d6-eef47ef75fd8 admin admin] The token cache is being invalidate because project 9404a43483c44564a0dcf7d5895c3b08 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81767) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 22 16:11:33.718264 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.notifications [None req-5bf9fba1-b0b9-4d7d-92d6-eef47ef75fd8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81767) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 16:11:33.719216 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.notifications [None req-5bf9fba1-b0b9-4d7d-92d6-eef47ef75fd8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81767) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 16:11:33.724873 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 991/1998] 10.222.0.15 () {58 vars in 1233 bytes} [Mon Aug 22 16:11:33 2022] DELETE /identity/v3/projects/9404a43483c44564a0dcf7d5895c3b08 => generated 0 bytes in 404 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 16:11:33.728980 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bf9fba1-b0b9-4d7d-92d6-eef47ef75fd8 admin admin] Authenticating user token {{(pid=81767) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 16:11:33.733769 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.fernet_utils [None req-5bf9fba1-b0b9-4d7d-92d6-eef47ef75fd8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81767) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 16:11:33.746412 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63fe9357-ccf0-442d-b1c5-5f9c077bc57a admin admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 16:11:33.751545 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b082fa6-2714-4f52-bc63-c42335ff40f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a98b1e4431940ab8d1fc95df2858a6c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 16:11:33.752181 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b082fa6-2714-4f52-bc63-c42335ff40f3 admin admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:11:33.753085 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b082fa6-2714-4f52-bc63-c42335ff40f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:11:33.753371 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b082fa6-2714-4f52-bc63-c42335ff40f3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:11:33.754561 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b082fa6-2714-4f52-bc63-c42335ff40f3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 16:11:33.756259 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b082fa6-2714-4f52-bc63-c42335ff40f3 admin admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 16:11:33.767102 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 1008/1999] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Aug 22 16:11:33 2022] GET /identity/v3/roles => generated 1420 bytes in 111 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 22 16:11:33.771885 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b082fa6-2714-4f52-bc63-c42335ff40f3 admin admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 16:11:33.773665 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-2b082fa6-2714-4f52-bc63-c42335ff40f3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 16:11:33.786980 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b082fa6-2714-4f52-bc63-c42335ff40f3 admin admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 16:11:33.792717 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-299f9e2c-c877-4488-8e3a-609c4b1b4939 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a98b1e4431940ab8d1fc95df2858a6c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 16:11:33.793480 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-299f9e2c-c877-4488-8e3a-609c4b1b4939 admin admin] REQUEST_METHOD: `PUT` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:11:33.793836 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-299f9e2c-c877-4488-8e3a-609c4b1b4939 admin admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:11:33.794177 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-299f9e2c-c877-4488-8e3a-609c4b1b4939 admin admin] PATH_INFO: `/v3/projects/a23886ab1d2a48f3a0a21077d585f1a0/users/b527a92a1be840a19745f067f2729477/roles/9d9848cfd22e4b089db443c350e56670` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:11:33.796399 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-299f9e2c-c877-4488-8e3a-609c4b1b4939 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a23886ab1d2a48f3a0a21077d585f1a0, user_id=b527a92a1be840a19745f067f2729477, role_id=9d9848cfd22e4b089db443c350e56670)` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 16:11:33.799407 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-299f9e2c-c877-4488-8e3a-609c4b1b4939 admin admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 16:11:33.820167 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 1009/2000] 10.222.0.15 () {60 vars in 1483 bytes} [Mon Aug 22 16:11:33 2022] PUT /identity/v3/projects/a23886ab1d2a48f3a0a21077d585f1a0/users/b527a92a1be840a19745f067f2729477/roles/9d9848cfd22e4b089db443c350e56670 => generated 0 bytes in 40 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 16:11:33.822631 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9b71de1-7675-4d35-a848-edf003c684b6 None None] REQUEST_METHOD: `POST` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:11:33.823019 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9b71de1-7675-4d35-a848-edf003c684b6 None None] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:11:33.823436 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9b71de1-7675-4d35-a848-edf003c684b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:11:33.829247 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bf9fba1-b0b9-4d7d-92d6-eef47ef75fd8 admin admin] Validating token access rules against request {{(pid=81767) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 16:11:33.854105 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.auth.core [None req-b9b71de1-7675-4d35-a848-edf003c684b6 None None] MFA Rules not processed for user `b527a92a1be840a19745f067f2729477`. Rule list: `[]` (Enabled: `True`). {{(pid=81768) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 16:11:33.855643 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-b9b71de1-7675-4d35-a848-edf003c684b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 16:11:33.894213 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1f068db-b411-42f8-82fc-baa4b0809e11 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a98b1e4431940ab8d1fc95df2858a6c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81767) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 16:11:33.895776 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f068db-b411-42f8-82fc-baa4b0809e11 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:11:33.896067 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f068db-b411-42f8-82fc-baa4b0809e11 admin admin] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:11:33.896339 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f068db-b411-42f8-82fc-baa4b0809e11 admin admin] PATH_INFO: `/v3/users/8834d2789562476e902ba236b381a580` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:11:33.897553 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1f068db-b411-42f8-82fc-baa4b0809e11 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8834d2789562476e902ba236b381a580)` {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 16:11:33.899613 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1f068db-b411-42f8-82fc-baa4b0809e11 admin admin] RBAC: Authorization granted {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 16:11:33.938546 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 1010/2001] 10.222.0.15 () {58 vars in 958 bytes} [Mon Aug 22 16:11:33 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 16:11:33.950771 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9b71de1-7675-4d35-a848-edf003c684b6 None None] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 16:11:33.953119 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-b9b71de1-7675-4d35-a848-edf003c684b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 16:11:34.033259 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9b71de1-7675-4d35-a848-edf003c684b6 None None] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 16:11:34.037840 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0cac4eb-c8d6-4a3a-baa4-c282b5fb294f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79706d480f864998b9f8594be407ceab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4ee0413f9ef4d4fb1f95a660babe090', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 16:11:34.038812 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0cac4eb-c8d6-4a3a-baa4-c282b5fb294f service cinder] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:11:34.038812 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0cac4eb-c8d6-4a3a-baa4-c282b5fb294f service cinder] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:11:34.039008 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0cac4eb-c8d6-4a3a-baa4-c282b5fb294f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:11:34.050079 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0cac4eb-c8d6-4a3a-baa4-c282b5fb294f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 16:11:34.052323 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0cac4eb-c8d6-4a3a-baa4-c282b5fb294f service cinder] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 16:11:34.075546 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 1011/2002] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Aug 22 16:11:33 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 125 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 22 16:11:34.102260 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.notifications [None req-b1f068db-b411-42f8-82fc-baa4b0809e11 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8834d2789562476e902ba236b381a580'} {{(pid=81767) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 16:11:34.102942 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.notifications [None req-b1f068db-b411-42f8-82fc-baa4b0809e11 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8834d2789562476e902ba236b381a580'} {{(pid=81767) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 16:11:34.179388 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0cac4eb-c8d6-4a3a-baa4-c282b5fb294f service cinder] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 16:11:34.180899 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-c0cac4eb-c8d6-4a3a-baa4-c282b5fb294f service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 16:11:34.190973 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.notifications [None req-b1f068db-b411-42f8-82fc-baa4b0809e11 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8834d2789562476e902ba236b381a580'} {{(pid=81767) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 16:11:34.193519 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0cac4eb-c8d6-4a3a-baa4-c282b5fb294f service cinder] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 16:11:34.200853 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-546f7090-0718-4648-8114-e49f86f6e2b0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79706d480f864998b9f8594be407ceab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4ee0413f9ef4d4fb1f95a660babe090', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 16:11:34.200853 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-546f7090-0718-4648-8114-e49f86f6e2b0 service cinder] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:11:34.200853 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-546f7090-0718-4648-8114-e49f86f6e2b0 service cinder] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:11:34.200853 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-546f7090-0718-4648-8114-e49f86f6e2b0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:11:34.201224 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-546f7090-0718-4648-8114-e49f86f6e2b0 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 16:11:34.202310 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.notifications [None req-b1f068db-b411-42f8-82fc-baa4b0809e11 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8834d2789562476e902ba236b381a580'} {{(pid=81767) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 16:11:34.234689 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.notifications [None req-b1f068db-b411-42f8-82fc-baa4b0809e11 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8834d2789562476e902ba236b381a580'} {{(pid=81767) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 16:11:34.241540 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 992/2003] 10.222.0.15 () {58 vars in 1224 bytes} [Mon Aug 22 16:11:33 2022] DELETE /identity/v3/users/8834d2789562476e902ba236b381a580 => generated 0 bytes in 513 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 16:11:34.242682 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1f068db-b411-42f8-82fc-baa4b0809e11 admin admin] Authenticating user token {{(pid=81767) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 16:11:34.252786 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.fernet_utils [None req-b1f068db-b411-42f8-82fc-baa4b0809e11 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81767) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 16:11:34.272769 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-546f7090-0718-4648-8114-e49f86f6e2b0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 16:11:34.274638 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-546f7090-0718-4648-8114-e49f86f6e2b0 service cinder] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 16:11:34.285910 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 1012/2004] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Aug 22 16:11:34 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 22 16:11:34.338700 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1f068db-b411-42f8-82fc-baa4b0809e11 admin admin] Validating token access rules against request {{(pid=81767) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 16:11:34.345743 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13562915-20ff-4baa-b993-f95cc8e24cb6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a98b1e4431940ab8d1fc95df2858a6c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81767) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 16:11:34.346414 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13562915-20ff-4baa-b993-f95cc8e24cb6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:11:34.346699 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13562915-20ff-4baa-b993-f95cc8e24cb6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:11:34.346976 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13562915-20ff-4baa-b993-f95cc8e24cb6 admin admin] PATH_INFO: `/v3/projects/4c1815c448c740ca9657c6dd6920c887` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:11:34.348720 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13562915-20ff-4baa-b993-f95cc8e24cb6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4c1815c448c740ca9657c6dd6920c887)` {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 16:11:34.350702 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13562915-20ff-4baa-b993-f95cc8e24cb6 admin admin] RBAC: Authorization granted {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 16:11:34.464617 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=94,keystone:DELETE=4,keystone:INSERT=2 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 16:11:34.496068 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.notifications [None req-13562915-20ff-4baa-b993-f95cc8e24cb6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4c1815c448c740ca9657c6dd6920c887', 'request_id': 'req-13562915-20ff-4baa-b993-f95cc8e24cb6'} {{(pid=81767) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 16:11:34.501824 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.notifications [None req-13562915-20ff-4baa-b993-f95cc8e24cb6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4c1815c448c740ca9657c6dd6920c887', 'request_id': 'req-13562915-20ff-4baa-b993-f95cc8e24cb6'} {{(pid=81767) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 16:11:34.592952 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.notifications [None req-13562915-20ff-4baa-b993-f95cc8e24cb6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4c1815c448c740ca9657c6dd6920c887', 'request_id': 'req-13562915-20ff-4baa-b993-f95cc8e24cb6'} {{(pid=81767) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 16:11:34.605886 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.notifications [None req-13562915-20ff-4baa-b993-f95cc8e24cb6 admin admin] The token cache is being invalidate because project 4c1815c448c740ca9657c6dd6920c887 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81767) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 22 16:11:34.606120 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.notifications [None req-13562915-20ff-4baa-b993-f95cc8e24cb6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81767) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 16:11:34.609453 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.notifications [None req-13562915-20ff-4baa-b993-f95cc8e24cb6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81767) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 16:11:34.611107 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 993/2005] 10.222.0.15 () {58 vars in 1233 bytes} [Mon Aug 22 16:11:34 2022] DELETE /identity/v3/projects/4c1815c448c740ca9657c6dd6920c887 => generated 0 bytes in 369 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 16:11:34.616023 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1b20c2d-e86f-468e-8c47-ff7bd288906f None None] REQUEST_METHOD: `POST` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:11:34.616506 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1b20c2d-e86f-468e-8c47-ff7bd288906f None None] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:11:34.616689 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1b20c2d-e86f-468e-8c47-ff7bd288906f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:11:34.638138 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.auth.core [None req-c1b20c2d-e86f-468e-8c47-ff7bd288906f None None] MFA Rules not processed for user `6b2199ae0fb0480aa917ada391f5ec54`. Rule list: `[]` (Enabled: `True`). {{(pid=81768) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 16:11:34.639386 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-c1b20c2d-e86f-468e-8c47-ff7bd288906f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 16:11:34.684813 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG dbcounter [-] [81767] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81767) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 16:11:34.717189 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 1013/2006] 10.222.0.15 () {58 vars in 958 bytes} [Mon Aug 22 16:11:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 16:11:34.730788 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eac4fe7-ed7d-41df-ae73-87ff57a50fe7 None None] REQUEST_METHOD: `POST` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:11:34.731404 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eac4fe7-ed7d-41df-ae73-87ff57a50fe7 None None] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:11:34.731601 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eac4fe7-ed7d-41df-ae73-87ff57a50fe7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:11:34.752226 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.auth.core [None req-9eac4fe7-ed7d-41df-ae73-87ff57a50fe7 None None] MFA Rules not processed for user `6b2199ae0fb0480aa917ada391f5ec54`. Rule list: `[]` (Enabled: `True`). {{(pid=81767) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 16:11:34.753564 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.fernet_utils [None req-9eac4fe7-ed7d-41df-ae73-87ff57a50fe7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81767) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 16:11:34.762024 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 994/2007] 10.222.0.15 () {58 vars in 958 bytes} [Mon Aug 22 16:11:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 16:11:34.766205 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1b20c2d-e86f-468e-8c47-ff7bd288906f None None] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 16:11:34.776513 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1b20c2d-e86f-468e-8c47-ff7bd288906f None None] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 16:11:34.781767 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0996f9b-3344-410e-b8d6-00b883fd95de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a98b1e4431940ab8d1fc95df2858a6c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 16:11:34.782372 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0996f9b-3344-410e-b8d6-00b883fd95de admin admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:11:34.782649 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0996f9b-3344-410e-b8d6-00b883fd95de admin admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:11:34.782915 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0996f9b-3344-410e-b8d6-00b883fd95de admin admin] PATH_INFO: `/v3/domains` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:11:34.783850 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0996f9b-3344-410e-b8d6-00b883fd95de admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81768) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 16:11:34.784058 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0996f9b-3344-410e-b8d6-00b883fd95de admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 16:11:34.789872 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0996f9b-3344-410e-b8d6-00b883fd95de admin admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 16:11:34.800228 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 1014/2008] 10.222.0.15 () {58 vars in 1153 bytes} [Mon Aug 22 16:11:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 16:11:34.804785 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eac4fe7-ed7d-41df-ae73-87ff57a50fe7 None None] Authenticating user token {{(pid=81767) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 16:11:34.814167 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eac4fe7-ed7d-41df-ae73-87ff57a50fe7 None None] Validating token access rules against request {{(pid=81767) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 16:11:34.819625 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09202545-3eed-41e8-8dd7-544080b85eba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a98b1e4431940ab8d1fc95df2858a6c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81767) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 16:11:34.820235 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09202545-3eed-41e8-8dd7-544080b85eba admin admin] REQUEST_METHOD: `POST` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:11:34.820572 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09202545-3eed-41e8-8dd7-544080b85eba admin admin] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:11:34.820850 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09202545-3eed-41e8-8dd7-544080b85eba admin admin] PATH_INFO: `/v3/projects` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:11:34.821488 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09202545-3eed-41e8-8dd7-544080b85eba admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 16:11:34.823409 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09202545-3eed-41e8-8dd7-544080b85eba admin admin] RBAC: Authorization granted {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 16:11:34.844497 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0996f9b-3344-410e-b8d6-00b883fd95de admin admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 16:11:34.846432 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-b0996f9b-3344-410e-b8d6-00b883fd95de admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 16:11:34.859918 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0996f9b-3344-410e-b8d6-00b883fd95de admin admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 16:11:34.864794 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db09c2e8-8e5a-4ee2-8a49-e8ac84745e3c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a98b1e4431940ab8d1fc95df2858a6c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 16:11:34.865434 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db09c2e8-8e5a-4ee2-8a49-e8ac84745e3c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:11:34.865707 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db09c2e8-8e5a-4ee2-8a49-e8ac84745e3c admin admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:11:34.866000 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db09c2e8-8e5a-4ee2-8a49-e8ac84745e3c admin admin] PATH_INFO: `/v3/projects/7c86b0d40f2847ad83c93fbc8de8cf44` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:11:34.867135 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db09c2e8-8e5a-4ee2-8a49-e8ac84745e3c admin admin] RBAC: Authorizing `identity:delete_project(project_id=7c86b0d40f2847ad83c93fbc8de8cf44)` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 16:11:34.869095 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db09c2e8-8e5a-4ee2-8a49-e8ac84745e3c admin admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 16:11:34.994123 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 995/2009] 10.222.0.15 () {60 vars in 1153 bytes} [Mon Aug 22 16:11:34 2022] POST /identity/v3/projects => generated 381 bytes in 190 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 22 16:11:34.998997 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09202545-3eed-41e8-8dd7-544080b85eba admin admin] Authenticating user token {{(pid=81767) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 16:11:35.006685 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-db09c2e8-8e5a-4ee2-8a49-e8ac84745e3c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7c86b0d40f2847ad83c93fbc8de8cf44', 'request_id': 'req-db09c2e8-8e5a-4ee2-8a49-e8ac84745e3c'} {{(pid=81768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 16:11:35.012594 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-db09c2e8-8e5a-4ee2-8a49-e8ac84745e3c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7c86b0d40f2847ad83c93fbc8de8cf44', 'request_id': 'req-db09c2e8-8e5a-4ee2-8a49-e8ac84745e3c'} {{(pid=81768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 16:11:35.065709 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09202545-3eed-41e8-8dd7-544080b85eba admin admin] Validating token access rules against request {{(pid=81767) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 16:11:35.070859 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afe8856d-776c-4d94-9913-aba873762664 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a98b1e4431940ab8d1fc95df2858a6c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81767) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 16:11:35.071568 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afe8856d-776c-4d94-9913-aba873762664 admin admin] REQUEST_METHOD: `POST` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:11:35.071840 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afe8856d-776c-4d94-9913-aba873762664 admin admin] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:11:35.072110 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afe8856d-776c-4d94-9913-aba873762664 admin admin] PATH_INFO: `/v3/users` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:11:35.072741 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afe8856d-776c-4d94-9913-aba873762664 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 16:11:35.074696 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afe8856d-776c-4d94-9913-aba873762664 admin admin] RBAC: Authorization granted {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 16:11:35.128455 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-db09c2e8-8e5a-4ee2-8a49-e8ac84745e3c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7c86b0d40f2847ad83c93fbc8de8cf44', 'request_id': 'req-db09c2e8-8e5a-4ee2-8a49-e8ac84745e3c'} {{(pid=81768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 16:11:35.141764 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-db09c2e8-8e5a-4ee2-8a49-e8ac84745e3c admin admin] The token cache is being invalidate because project 7c86b0d40f2847ad83c93fbc8de8cf44 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81768) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 22 16:11:35.142057 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-db09c2e8-8e5a-4ee2-8a49-e8ac84745e3c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 16:11:35.142759 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-db09c2e8-8e5a-4ee2-8a49-e8ac84745e3c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 16:11:35.144372 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 1015/2010] 10.222.0.15 () {58 vars in 1233 bytes} [Mon Aug 22 16:11:34 2022] DELETE /identity/v3/projects/7c86b0d40f2847ad83c93fbc8de8cf44 => generated 0 bytes in 300 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 16:11:35.148684 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66cbfc3e-b9c1-4417-afb5-22f6104cd89e None None] REQUEST_METHOD: `POST` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:11:35.148684 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66cbfc3e-b9c1-4417-afb5-22f6104cd89e None None] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:11:35.148684 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66cbfc3e-b9c1-4417-afb5-22f6104cd89e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:11:35.163638 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 996/2011] 10.222.0.15 () {60 vars in 1144 bytes} [Mon Aug 22 16:11:34 2022] POST /identity/v3/users => generated 291 bytes in 165 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 22 16:11:35.165070 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a036a782-5d9a-4c6d-a115-94eb57c539b8 None None] REQUEST_METHOD: `POST` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:11:35.165361 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a036a782-5d9a-4c6d-a115-94eb57c539b8 None None] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:11:35.165791 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a036a782-5d9a-4c6d-a115-94eb57c539b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:11:35.166439 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.auth.core [None req-66cbfc3e-b9c1-4417-afb5-22f6104cd89e None None] MFA Rules not processed for user `be4684a264444a52ac5488b5b2ea5ff1`. Rule list: `[]` (Enabled: `True`). {{(pid=81768) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 16:11:35.167688 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-66cbfc3e-b9c1-4417-afb5-22f6104cd89e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 16:11:35.199254 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.auth.core [None req-a036a782-5d9a-4c6d-a115-94eb57c539b8 None None] MFA Rules not processed for user `6b2199ae0fb0480aa917ada391f5ec54`. Rule list: `[]` (Enabled: `True`). {{(pid=81767) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 16:11:35.199254 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.fernet_utils [None req-a036a782-5d9a-4c6d-a115-94eb57c539b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81767) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 16:11:35.200238 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 997/2012] 10.222.0.15 () {58 vars in 958 bytes} [Mon Aug 22 16:11:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 16:11:35.210730 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a036a782-5d9a-4c6d-a115-94eb57c539b8 None None] Authenticating user token {{(pid=81767) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 16:11:35.214161 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.fernet_utils [None req-a036a782-5d9a-4c6d-a115-94eb57c539b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81767) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 16:11:35.226887 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a036a782-5d9a-4c6d-a115-94eb57c539b8 None None] Validating token access rules against request {{(pid=81767) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 16:11:35.233050 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 1016/2013] 10.222.0.15 () {58 vars in 958 bytes} [Mon Aug 22 16:11:35 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 16:11:35.233050 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ad47cfa-d482-412e-ba0d-ac8a57bddfad None None] REQUEST_METHOD: `POST` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:11:35.233050 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ad47cfa-d482-412e-ba0d-ac8a57bddfad None None] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:11:35.233050 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ad47cfa-d482-412e-ba0d-ac8a57bddfad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:11:35.241993 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3d78f9e-97ed-421a-a66c-e1c092b526f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a98b1e4431940ab8d1fc95df2858a6c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81767) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 16:11:35.242893 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d78f9e-97ed-421a-a66c-e1c092b526f5 admin admin] REQUEST_METHOD: `GET` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:11:35.243350 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d78f9e-97ed-421a-a66c-e1c092b526f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:11:35.243771 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d78f9e-97ed-421a-a66c-e1c092b526f5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:11:35.244681 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3d78f9e-97ed-421a-a66c-e1c092b526f5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 16:11:35.247557 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3d78f9e-97ed-421a-a66c-e1c092b526f5 admin admin] RBAC: Authorization granted {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 16:11:35.257819 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.auth.core [None req-5ad47cfa-d482-412e-ba0d-ac8a57bddfad None None] MFA Rules not processed for user `6b2199ae0fb0480aa917ada391f5ec54`. Rule list: `[]` (Enabled: `True`). {{(pid=81768) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 16:11:35.259219 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 998/2014] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Aug 22 16:11:35 2022] GET /identity/v3/roles => generated 1420 bytes in 58 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 22 16:11:35.261900 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-5ad47cfa-d482-412e-ba0d-ac8a57bddfad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 16:11:35.262981 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3d78f9e-97ed-421a-a66c-e1c092b526f5 admin admin] Authenticating user token {{(pid=81767) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 16:11:35.271737 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 1017/2015] 10.222.0.15 () {58 vars in 958 bytes} [Mon Aug 22 16:11:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 16:11:35.275443 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3d78f9e-97ed-421a-a66c-e1c092b526f5 admin admin] Validating token access rules against request {{(pid=81767) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 16:11:35.276291 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ad47cfa-d482-412e-ba0d-ac8a57bddfad None None] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 16:11:35.280462 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b93150e-0e84-4aca-a299-e8f9fc49938c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a98b1e4431940ab8d1fc95df2858a6c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81767) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 16:11:35.280928 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b93150e-0e84-4aca-a299-e8f9fc49938c admin admin] REQUEST_METHOD: `GET` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:11:35.281211 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b93150e-0e84-4aca-a299-e8f9fc49938c admin admin] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:11:35.281482 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b93150e-0e84-4aca-a299-e8f9fc49938c admin admin] PATH_INFO: `/v3/roles` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:11:35.282126 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b93150e-0e84-4aca-a299-e8f9fc49938c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 16:11:35.284483 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b93150e-0e84-4aca-a299-e8f9fc49938c admin admin] RBAC: Authorization granted {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 16:11:35.290286 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ad47cfa-d482-412e-ba0d-ac8a57bddfad None None] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 16:11:35.297996 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3be77e0b-2d21-46c2-83af-d1cfb42b1bf2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a98b1e4431940ab8d1fc95df2858a6c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 16:11:35.297996 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3be77e0b-2d21-46c2-83af-d1cfb42b1bf2 admin admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:11:35.297996 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3be77e0b-2d21-46c2-83af-d1cfb42b1bf2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:11:35.297996 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3be77e0b-2d21-46c2-83af-d1cfb42b1bf2 admin admin] PATH_INFO: `/v3/domains` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:11:35.298427 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3be77e0b-2d21-46c2-83af-d1cfb42b1bf2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81768) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 16:11:35.298670 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3be77e0b-2d21-46c2-83af-d1cfb42b1bf2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 16:11:35.299160 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 999/2016] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Aug 22 16:11:35 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 22 16:11:35.300196 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b93150e-0e84-4aca-a299-e8f9fc49938c admin admin] Authenticating user token {{(pid=81767) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 16:11:35.300416 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3be77e0b-2d21-46c2-83af-d1cfb42b1bf2 admin admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 16:11:35.310961 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b93150e-0e84-4aca-a299-e8f9fc49938c admin admin] Validating token access rules against request {{(pid=81767) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 16:11:35.315349 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 1018/2017] 10.222.0.15 () {58 vars in 1153 bytes} [Mon Aug 22 16:11:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 22 16:11:35.316101 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98d90390-80fe-4049-942d-ddd6e4535908 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a98b1e4431940ab8d1fc95df2858a6c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81767) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 16:11:35.317366 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98d90390-80fe-4049-942d-ddd6e4535908 admin admin] REQUEST_METHOD: `PUT` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:11:35.317645 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98d90390-80fe-4049-942d-ddd6e4535908 admin admin] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:11:35.317914 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98d90390-80fe-4049-942d-ddd6e4535908 admin admin] PATH_INFO: `/v3/projects/5ec3508af5984913aa5254de1e5fd2a4/users/fd4fd72776794f84a3dca6d8fb59b4d1/roles/e6fc1ee3fb974125ba9b4fe55b12b45e` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:11:35.320212 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3be77e0b-2d21-46c2-83af-d1cfb42b1bf2 admin admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 16:11:35.336793 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98d90390-80fe-4049-942d-ddd6e4535908 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5ec3508af5984913aa5254de1e5fd2a4, user_id=fd4fd72776794f84a3dca6d8fb59b4d1, role_id=e6fc1ee3fb974125ba9b4fe55b12b45e)` {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 16:11:35.339998 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98d90390-80fe-4049-942d-ddd6e4535908 admin admin] RBAC: Authorization granted {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 16:11:35.350563 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3be77e0b-2d21-46c2-83af-d1cfb42b1bf2 admin admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 16:11:35.355742 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eaeb79b-4165-469d-8723-ed9fd2929a02 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a98b1e4431940ab8d1fc95df2858a6c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 16:11:35.356360 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eaeb79b-4165-469d-8723-ed9fd2929a02 admin admin] REQUEST_METHOD: `POST` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:11:35.356643 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eaeb79b-4165-469d-8723-ed9fd2929a02 admin admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:11:35.356975 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eaeb79b-4165-469d-8723-ed9fd2929a02 admin admin] PATH_INFO: `/v3/projects` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:11:35.357566 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eaeb79b-4165-469d-8723-ed9fd2929a02 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 16:11:35.359509 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eaeb79b-4165-469d-8723-ed9fd2929a02 admin admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 16:11:35.416503 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 1000/2018] 10.222.0.15 () {60 vars in 1483 bytes} [Mon Aug 22 16:11:35 2022] PUT /identity/v3/projects/5ec3508af5984913aa5254de1e5fd2a4/users/fd4fd72776794f84a3dca6d8fb59b4d1/roles/e6fc1ee3fb974125ba9b4fe55b12b45e => generated 0 bytes in 117 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 16:11:35.426691 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec2c4288-088c-4264-ad4f-be62b8f04e45 None None] REQUEST_METHOD: `POST` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:11:35.426981 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec2c4288-088c-4264-ad4f-be62b8f04e45 None None] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:11:35.427247 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec2c4288-088c-4264-ad4f-be62b8f04e45 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:11:35.459527 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.auth.core [None req-ec2c4288-088c-4264-ad4f-be62b8f04e45 None None] MFA Rules not processed for user `fd4fd72776794f84a3dca6d8fb59b4d1`. Rule list: `[]` (Enabled: `True`). {{(pid=81767) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 16:11:35.461163 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.fernet_utils [None req-ec2c4288-088c-4264-ad4f-be62b8f04e45 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81767) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 16:11:35.474475 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 1019/2019] 10.222.0.15 () {60 vars in 1153 bytes} [Mon Aug 22 16:11:35 2022] POST /identity/v3/projects => generated 391 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 22 16:11:35.479021 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eaeb79b-4165-469d-8723-ed9fd2929a02 admin admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 16:11:35.543357 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 1001/2020] 10.222.0.15 () {58 vars in 958 bytes} [Mon Aug 22 16:11:35 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 22 16:11:35.546726 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eaeb79b-4165-469d-8723-ed9fd2929a02 admin admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 16:11:35.547664 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec2c4288-088c-4264-ad4f-be62b8f04e45 None None] Authenticating user token {{(pid=81767) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 16:11:35.552129 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3d76d29-6ab9-4ea6-8c5d-198de0f08c01 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a98b1e4431940ab8d1fc95df2858a6c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 16:11:35.552750 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3d76d29-6ab9-4ea6-8c5d-198de0f08c01 admin admin] REQUEST_METHOD: `POST` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:11:35.553033 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3d76d29-6ab9-4ea6-8c5d-198de0f08c01 admin admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:11:35.553306 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3d76d29-6ab9-4ea6-8c5d-198de0f08c01 admin admin] PATH_INFO: `/v3/users` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:11:35.553932 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3d76d29-6ab9-4ea6-8c5d-198de0f08c01 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 16:11:35.555924 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3d76d29-6ab9-4ea6-8c5d-198de0f08c01 admin admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 16:11:35.618341 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec2c4288-088c-4264-ad4f-be62b8f04e45 None None] Validating token access rules against request {{(pid=81767) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 16:11:35.623308 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7829f2ac-bbec-45c5-8e6b-c3d7758f80f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a98b1e4431940ab8d1fc95df2858a6c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81767) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 16:11:35.623924 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7829f2ac-bbec-45c5-8e6b-c3d7758f80f5 admin admin] REQUEST_METHOD: `POST` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:11:35.624203 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7829f2ac-bbec-45c5-8e6b-c3d7758f80f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:11:35.624478 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7829f2ac-bbec-45c5-8e6b-c3d7758f80f5 admin admin] PATH_INFO: `/v3/projects` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:11:35.625114 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7829f2ac-bbec-45c5-8e6b-c3d7758f80f5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 16:11:35.627053 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7829f2ac-bbec-45c5-8e6b-c3d7758f80f5 admin admin] RBAC: Authorization granted {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 16:11:35.661790 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 1020/2021] 10.222.0.15 () {60 vars in 1144 bytes} [Mon Aug 22 16:11:35 2022] POST /identity/v3/users => generated 296 bytes in 183 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 22 16:11:35.661790 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6b58b7b-c9d2-422a-9e82-e70cdec69161 None None] REQUEST_METHOD: `POST` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:11:35.661790 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6b58b7b-c9d2-422a-9e82-e70cdec69161 None None] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:11:35.661790 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6b58b7b-c9d2-422a-9e82-e70cdec69161 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:11:35.670248 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application [None req-d6b58b7b-c9d2-422a-9e82-e70cdec69161 None None] Expecting to find domain in project. The server could not comply with the request since it is either malformed or otherwise incorrect. The client is assumed to be in error.: keystone.exception.ValidationError: Expecting to find domain in project. The server could not comply with the request since it is either malformed or otherwise incorrect. The client is assumed to be in error. Aug 22 16:11:35.670248 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 16:11:35.670248 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 16:11:35.670248 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 16:11:35.670248 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 16:11:35.670248 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 16:11:35.670248 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 16:11:35.670248 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 16:11:35.670248 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 16:11:35.670248 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 16:11:35.670248 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 16:11:35.670248 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 16:11:35.670248 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/server/flask/common.py", line 1064, in wrapper Aug 22 16:11:35.671034 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return f(*args, **kwargs) Aug 22 16:11:35.671034 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/auth.py", line 315, in post Aug 22 16:11:35.671034 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application token = authentication.authenticate_for_token(auth_data) Aug 22 16:11:35.671034 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/_shared/authentication.py", line 182, in authenticate_for_token Aug 22 16:11:35.671034 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application auth_info = core.AuthInfo.create(auth=auth) Aug 22 16:11:35.671034 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/auth/core.py", line 126, in create Aug 22 16:11:35.671034 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application auth_info._validate_and_normalize_auth_data(scope_only) Aug 22 16:11:35.671034 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/auth/core.py", line 340, in _validate_and_normalize_auth_data Aug 22 16:11:35.671034 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application self._validate_and_normalize_scope_data() Aug 22 16:11:35.671034 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/auth/core.py", line 290, in _validate_and_normalize_scope_data Aug 22 16:11:35.671034 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application project_ref = self._lookup_project(self.auth['scope']['project']) Aug 22 16:11:35.671034 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/auth/core.py", line 194, in _lookup_project Aug 22 16:11:35.671034 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application raise exception.ValidationError(attribute='domain', Aug 22 16:11:35.671034 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application keystone.exception.ValidationError: Expecting to find domain in project. The server could not comply with the request since it is either malformed or otherwise incorrect. The client is assumed to be in error. Aug 22 16:11:35.671034 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application Aug 22 16:11:35.671862 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 1021/2022] 10.222.0.15 () {58 vars in 1009 bytes} [Mon Aug 22 16:11:35 2022] POST /identity/v3/auth/tokens => generated 230 bytes in 9 msecs (HTTP/1.1 400) 5 headers in 186 bytes (1 switches on core 0) Aug 22 16:11:35.672176 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6b58b7b-c9d2-422a-9e82-e70cdec69161 None None] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 16:11:35.690224 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6b58b7b-c9d2-422a-9e82-e70cdec69161 None None] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 16:11:35.702408 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0a8fa40-40f9-4a0f-bcb1-7a45502a7ed0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a98b1e4431940ab8d1fc95df2858a6c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 16:11:35.706303 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0a8fa40-40f9-4a0f-bcb1-7a45502a7ed0 admin admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:11:35.706303 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0a8fa40-40f9-4a0f-bcb1-7a45502a7ed0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:11:35.706303 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0a8fa40-40f9-4a0f-bcb1-7a45502a7ed0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:11:35.706303 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0a8fa40-40f9-4a0f-bcb1-7a45502a7ed0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 16:11:35.711635 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0a8fa40-40f9-4a0f-bcb1-7a45502a7ed0 admin admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 16:11:35.721920 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 1022/2023] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Aug 22 16:11:35 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 22 16:11:35.724669 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0a8fa40-40f9-4a0f-bcb1-7a45502a7ed0 admin admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 16:11:35.735754 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0a8fa40-40f9-4a0f-bcb1-7a45502a7ed0 admin admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 16:11:35.741617 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 1002/2024] 10.222.0.15 () {60 vars in 1153 bytes} [Mon Aug 22 16:11:35 2022] POST /identity/v3/projects => generated 381 bytes in 194 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 22 16:11:35.742080 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcf13b60-3902-47f0-99dc-d9eedeb92aa7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a98b1e4431940ab8d1fc95df2858a6c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 16:11:35.742621 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcf13b60-3902-47f0-99dc-d9eedeb92aa7 admin admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:11:35.742905 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcf13b60-3902-47f0-99dc-d9eedeb92aa7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:11:35.743180 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcf13b60-3902-47f0-99dc-d9eedeb92aa7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:11:35.743860 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcf13b60-3902-47f0-99dc-d9eedeb92aa7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 16:11:35.745581 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7829f2ac-bbec-45c5-8e6b-c3d7758f80f5 admin admin] Authenticating user token {{(pid=81767) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 16:11:35.745748 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcf13b60-3902-47f0-99dc-d9eedeb92aa7 admin admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 16:11:35.761203 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 1023/2025] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Aug 22 16:11:35 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 22 16:11:35.762398 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcf13b60-3902-47f0-99dc-d9eedeb92aa7 admin admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 16:11:35.773657 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcf13b60-3902-47f0-99dc-d9eedeb92aa7 admin admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 16:11:35.816518 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7829f2ac-bbec-45c5-8e6b-c3d7758f80f5 admin admin] Validating token access rules against request {{(pid=81767) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 16:11:35.822257 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3d67586-43b2-4830-b469-6a399e17acbb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a98b1e4431940ab8d1fc95df2858a6c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81767) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 16:11:35.823020 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d67586-43b2-4830-b469-6a399e17acbb admin admin] REQUEST_METHOD: `POST` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:11:35.823450 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d67586-43b2-4830-b469-6a399e17acbb admin admin] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:11:35.823894 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d67586-43b2-4830-b469-6a399e17acbb admin admin] PATH_INFO: `/v3/users` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:11:35.824520 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3d67586-43b2-4830-b469-6a399e17acbb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 16:11:35.826621 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3d67586-43b2-4830-b469-6a399e17acbb admin admin] RBAC: Authorization granted {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 16:11:35.832999 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b73991f-ff33-4517-af61-7c739a9e3ca6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a98b1e4431940ab8d1fc95df2858a6c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 16:11:35.834043 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b73991f-ff33-4517-af61-7c739a9e3ca6 admin admin] REQUEST_METHOD: `PUT` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:11:35.834439 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b73991f-ff33-4517-af61-7c739a9e3ca6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:11:35.834811 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b73991f-ff33-4517-af61-7c739a9e3ca6 admin admin] PATH_INFO: `/v3/projects/ce9c2f7bd3e94da7a3b5c3de403fb67c/users/394b669ff68949e8909ba5e7248c8984/roles/e6fc1ee3fb974125ba9b4fe55b12b45e` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:11:35.851839 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b73991f-ff33-4517-af61-7c739a9e3ca6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ce9c2f7bd3e94da7a3b5c3de403fb67c, user_id=394b669ff68949e8909ba5e7248c8984, role_id=e6fc1ee3fb974125ba9b4fe55b12b45e)` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 16:11:35.854685 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b73991f-ff33-4517-af61-7c739a9e3ca6 admin admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 16:11:35.982333 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 1024/2026] 10.222.0.15 () {60 vars in 1483 bytes} [Mon Aug 22 16:11:35 2022] PUT /identity/v3/projects/ce9c2f7bd3e94da7a3b5c3de403fb67c/users/394b669ff68949e8909ba5e7248c8984/roles/e6fc1ee3fb974125ba9b4fe55b12b45e => generated 0 bytes in 220 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 16:11:35.985426 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 1003/2027] 10.222.0.15 () {60 vars in 1144 bytes} [Mon Aug 22 16:11:35 2022] POST /identity/v3/users => generated 297 bytes in 237 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 22 16:11:35.986593 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b73991f-ff33-4517-af61-7c739a9e3ca6 admin admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 16:11:35.994200 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82cb4613-5e47-47a6-a75f-19e65e6747d9 None None] REQUEST_METHOD: `POST` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:11:35.994593 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82cb4613-5e47-47a6-a75f-19e65e6747d9 None None] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:11:35.994929 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82cb4613-5e47-47a6-a75f-19e65e6747d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:11:36.026766 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.auth.core [None req-82cb4613-5e47-47a6-a75f-19e65e6747d9 None None] MFA Rules not processed for user `394b669ff68949e8909ba5e7248c8984`. Rule list: `[]` (Enabled: `True`). {{(pid=81767) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 16:11:36.028013 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.fernet_utils [None req-82cb4613-5e47-47a6-a75f-19e65e6747d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81767) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 16:11:36.051488 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b73991f-ff33-4517-af61-7c739a9e3ca6 admin admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 16:11:36.057190 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-130eb998-70a4-44f1-adcc-56479243bb07 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a98b1e4431940ab8d1fc95df2858a6c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 16:11:36.057786 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-130eb998-70a4-44f1-adcc-56479243bb07 admin admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:11:36.058070 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-130eb998-70a4-44f1-adcc-56479243bb07 admin admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:11:36.058443 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-130eb998-70a4-44f1-adcc-56479243bb07 admin admin] PATH_INFO: `/v3/roles` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:11:36.059124 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-130eb998-70a4-44f1-adcc-56479243bb07 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 16:11:36.061394 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-130eb998-70a4-44f1-adcc-56479243bb07 admin admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 16:11:36.081209 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 1025/2028] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Aug 22 16:11:35 2022] GET /identity/v3/roles => generated 1420 bytes in 95 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 22 16:11:36.085693 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG dbcounter [-] [81767] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81767) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 16:11:36.085981 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-130eb998-70a4-44f1-adcc-56479243bb07 admin admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 16:11:36.097500 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-130eb998-70a4-44f1-adcc-56479243bb07 admin admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 16:11:36.103115 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80a17f4d-f0b2-4d56-a269-6f10e7510b68 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a98b1e4431940ab8d1fc95df2858a6c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 16:11:36.104978 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a17f4d-f0b2-4d56-a269-6f10e7510b68 admin admin] REQUEST_METHOD: `PUT` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:11:36.104978 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a17f4d-f0b2-4d56-a269-6f10e7510b68 admin admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:11:36.104978 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a17f4d-f0b2-4d56-a269-6f10e7510b68 admin admin] PATH_INFO: `/v3/projects/50bef07e316c4c5e996aab33974e32bc/users/f5329d1f1baa4439a5a0b8f220115825/roles/9d9848cfd22e4b089db443c350e56670` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:11:36.122442 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80a17f4d-f0b2-4d56-a269-6f10e7510b68 admin admin] RBAC: Authorizing `identity:create_grant(project_id=50bef07e316c4c5e996aab33974e32bc, user_id=f5329d1f1baa4439a5a0b8f220115825, role_id=9d9848cfd22e4b089db443c350e56670)` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 16:11:36.125116 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80a17f4d-f0b2-4d56-a269-6f10e7510b68 admin admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 16:11:36.132844 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 1004/2029] 10.222.0.15 () {58 vars in 958 bytes} [Mon Aug 22 16:11:35 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 22 16:11:36.141696 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82cb4613-5e47-47a6-a75f-19e65e6747d9 None None] Authenticating user token {{(pid=81767) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 16:11:36.153081 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82cb4613-5e47-47a6-a75f-19e65e6747d9 None None] Validating token access rules against request {{(pid=81767) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 16:11:36.157782 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a24b5bc0-4af0-4b80-a6c1-af82edf7f1c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a98b1e4431940ab8d1fc95df2858a6c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81767) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 16:11:36.158399 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a24b5bc0-4af0-4b80-a6c1-af82edf7f1c0 admin admin] REQUEST_METHOD: `POST` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:11:36.158678 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a24b5bc0-4af0-4b80-a6c1-af82edf7f1c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:11:36.158951 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a24b5bc0-4af0-4b80-a6c1-af82edf7f1c0 admin admin] PATH_INFO: `/v3/projects` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:11:36.159625 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a24b5bc0-4af0-4b80-a6c1-af82edf7f1c0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 16:11:36.162348 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a24b5bc0-4af0-4b80-a6c1-af82edf7f1c0 admin admin] RBAC: Authorization granted {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 16:11:36.227128 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 1026/2030] 10.222.0.15 () {60 vars in 1483 bytes} [Mon Aug 22 16:11:36 2022] PUT /identity/v3/projects/50bef07e316c4c5e996aab33974e32bc/users/f5329d1f1baa4439a5a0b8f220115825/roles/9d9848cfd22e4b089db443c350e56670 => generated 0 bytes in 142 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 22 16:11:36.231077 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80a17f4d-f0b2-4d56-a269-6f10e7510b68 admin admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 16:11:36.295083 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 1005/2031] 10.222.0.15 () {60 vars in 1153 bytes} [Mon Aug 22 16:11:36 2022] POST /identity/v3/projects => generated 393 bytes in 154 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 22 16:11:36.299852 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80a17f4d-f0b2-4d56-a269-6f10e7510b68 admin admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 16:11:36.300040 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a24b5bc0-4af0-4b80-a6c1-af82edf7f1c0 admin admin] Authenticating user token {{(pid=81767) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 16:11:36.368181 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-004ed740-5b7f-40fb-b7d5-1d21db7bf038 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a98b1e4431940ab8d1fc95df2858a6c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 16:11:36.368850 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-004ed740-5b7f-40fb-b7d5-1d21db7bf038 admin admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:11:36.369125 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-004ed740-5b7f-40fb-b7d5-1d21db7bf038 admin admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:11:36.369702 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-004ed740-5b7f-40fb-b7d5-1d21db7bf038 admin admin] PATH_INFO: `/v3/roles` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:11:36.370263 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-004ed740-5b7f-40fb-b7d5-1d21db7bf038 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 16:11:36.372258 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-004ed740-5b7f-40fb-b7d5-1d21db7bf038 admin admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 16:11:36.380298 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a24b5bc0-4af0-4b80-a6c1-af82edf7f1c0 admin admin] Validating token access rules against request {{(pid=81767) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 16:11:36.382951 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 1027/2032] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Aug 22 16:11:36 2022] GET /identity/v3/roles => generated 1420 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 22 16:11:36.386317 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e520987a-6cb1-4701-86c0-de331697d8ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a98b1e4431940ab8d1fc95df2858a6c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81767) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 16:11:36.387023 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-004ed740-5b7f-40fb-b7d5-1d21db7bf038 admin admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 16:11:36.387161 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e520987a-6cb1-4701-86c0-de331697d8ab admin admin] REQUEST_METHOD: `POST` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:11:36.387512 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e520987a-6cb1-4701-86c0-de331697d8ab admin admin] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:11:36.387876 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e520987a-6cb1-4701-86c0-de331697d8ab admin admin] PATH_INFO: `/v3/users` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:11:36.388585 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e520987a-6cb1-4701-86c0-de331697d8ab admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 16:11:36.390580 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e520987a-6cb1-4701-86c0-de331697d8ab admin admin] RBAC: Authorization granted {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 16:11:36.398221 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-004ed740-5b7f-40fb-b7d5-1d21db7bf038 admin admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 16:11:36.403687 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6e5b4d9-b181-4443-864c-582df3f7398f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a98b1e4431940ab8d1fc95df2858a6c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 16:11:36.408435 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6e5b4d9-b181-4443-864c-582df3f7398f admin admin] REQUEST_METHOD: `PUT` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:11:36.408435 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6e5b4d9-b181-4443-864c-582df3f7398f admin admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:11:36.408435 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6e5b4d9-b181-4443-864c-582df3f7398f admin admin] PATH_INFO: `/v3/projects/50bef07e316c4c5e996aab33974e32bc/users/f5329d1f1baa4439a5a0b8f220115825/roles/9d9848cfd22e4b089db443c350e56670` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:11:36.408435 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6e5b4d9-b181-4443-864c-582df3f7398f admin admin] RBAC: Authorizing `identity:create_grant(project_id=50bef07e316c4c5e996aab33974e32bc, user_id=f5329d1f1baa4439a5a0b8f220115825, role_id=9d9848cfd22e4b089db443c350e56670)` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 16:11:36.409467 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6e5b4d9-b181-4443-864c-582df3f7398f admin admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 16:11:36.420687 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 1028/2033] 10.222.0.15 () {60 vars in 1483 bytes} [Mon Aug 22 16:11:36 2022] PUT /identity/v3/projects/50bef07e316c4c5e996aab33974e32bc/users/f5329d1f1baa4439a5a0b8f220115825/roles/9d9848cfd22e4b089db443c350e56670 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 22 16:11:36.431421 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-893655b3-0e99-41a3-8da9-344f42205ace None None] REQUEST_METHOD: `POST` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:11:36.431705 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-893655b3-0e99-41a3-8da9-344f42205ace None None] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:11:36.431981 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-893655b3-0e99-41a3-8da9-344f42205ace None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:11:36.461484 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.auth.core [None req-893655b3-0e99-41a3-8da9-344f42205ace None None] MFA Rules not processed for user `f5329d1f1baa4439a5a0b8f220115825`. Rule list: `[]` (Enabled: `True`). {{(pid=81768) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 16:11:36.462755 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-893655b3-0e99-41a3-8da9-344f42205ace None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 16:11:36.515404 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 1006/2034] 10.222.0.15 () {60 vars in 1144 bytes} [Mon Aug 22 16:11:36 2022] POST /identity/v3/users => generated 303 bytes in 215 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 22 16:11:36.519976 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e520987a-6cb1-4701-86c0-de331697d8ab admin admin] Authenticating user token {{(pid=81767) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 16:11:36.530565 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e520987a-6cb1-4701-86c0-de331697d8ab admin admin] Validating token access rules against request {{(pid=81767) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 16:11:36.538822 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 16:11:36.563540 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 1029/2035] 10.222.0.15 () {58 vars in 958 bytes} [Mon Aug 22 16:11:36 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 22 16:11:36.578924 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-893655b3-0e99-41a3-8da9-344f42205ace None None] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 16:11:36.580650 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-893655b3-0e99-41a3-8da9-344f42205ace None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 16:11:36.626617 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c21298a-cb62-4541-a250-fce329587cab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a98b1e4431940ab8d1fc95df2858a6c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81767) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 16:11:36.627403 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c21298a-cb62-4541-a250-fce329587cab admin admin] REQUEST_METHOD: `GET` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:11:36.627663 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c21298a-cb62-4541-a250-fce329587cab admin admin] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:11:36.627937 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c21298a-cb62-4541-a250-fce329587cab admin admin] PATH_INFO: `/v3/roles` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:11:36.628587 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c21298a-cb62-4541-a250-fce329587cab admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 16:11:36.630869 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c21298a-cb62-4541-a250-fce329587cab admin admin] RBAC: Authorization granted {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 16:11:36.642448 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 1007/2036] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Aug 22 16:11:36 2022] GET /identity/v3/roles => generated 1420 bytes in 123 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 22 16:11:36.647472 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c21298a-cb62-4541-a250-fce329587cab admin admin] Authenticating user token {{(pid=81767) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 16:11:36.660763 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c21298a-cb62-4541-a250-fce329587cab admin admin] Validating token access rules against request {{(pid=81767) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 16:11:36.665878 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81a82286-511b-49fd-9a7d-10bf97fe5cc0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a98b1e4431940ab8d1fc95df2858a6c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81767) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 16:11:36.666600 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81a82286-511b-49fd-9a7d-10bf97fe5cc0 admin admin] REQUEST_METHOD: `PUT` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:11:36.666900 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81a82286-511b-49fd-9a7d-10bf97fe5cc0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:11:36.667171 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81a82286-511b-49fd-9a7d-10bf97fe5cc0 admin admin] PATH_INFO: `/v3/projects/1d9445484db94d76bf676b2f24693c32/users/ffb8485c36c147239e279087f49735dd/roles/9d9848cfd22e4b089db443c350e56670` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:11:36.675207 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-893655b3-0e99-41a3-8da9-344f42205ace None None] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 16:11:36.679430 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60fc97ae-49a2-44f2-8355-84e17faec7bd service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79706d480f864998b9f8594be407ceab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4ee0413f9ef4d4fb1f95a660babe090', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 16:11:36.680049 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60fc97ae-49a2-44f2-8355-84e17faec7bd service cinder] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:11:36.680320 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60fc97ae-49a2-44f2-8355-84e17faec7bd service cinder] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:11:36.680587 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60fc97ae-49a2-44f2-8355-84e17faec7bd service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:11:36.685469 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81a82286-511b-49fd-9a7d-10bf97fe5cc0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1d9445484db94d76bf676b2f24693c32, user_id=ffb8485c36c147239e279087f49735dd, role_id=9d9848cfd22e4b089db443c350e56670)` {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 16:11:36.688753 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81a82286-511b-49fd-9a7d-10bf97fe5cc0 admin admin] RBAC: Authorization granted {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 16:11:36.691032 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60fc97ae-49a2-44f2-8355-84e17faec7bd service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 16:11:36.693504 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60fc97ae-49a2-44f2-8355-84e17faec7bd service cinder] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 16:11:36.704125 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 1030/2037] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Aug 22 16:11:36 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 126 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 22 16:11:36.851079 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 1008/2038] 10.222.0.15 () {60 vars in 1483 bytes} [Mon Aug 22 16:11:36 2022] PUT /identity/v3/projects/1d9445484db94d76bf676b2f24693c32/users/ffb8485c36c147239e279087f49735dd/roles/9d9848cfd22e4b089db443c350e56670 => generated 0 bytes in 205 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 22 16:11:36.855183 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60fc97ae-49a2-44f2-8355-84e17faec7bd service cinder] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 16:11:36.867795 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60fc97ae-49a2-44f2-8355-84e17faec7bd service cinder] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 16:11:36.924924 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eed48d22-feb5-474c-b432-42cffd0659ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a98b1e4431940ab8d1fc95df2858a6c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 16:11:36.926091 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eed48d22-feb5-474c-b432-42cffd0659ae admin admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:11:36.926378 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eed48d22-feb5-474c-b432-42cffd0659ae admin admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:11:36.926648 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eed48d22-feb5-474c-b432-42cffd0659ae admin admin] PATH_INFO: `/v3/roles` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:11:36.927296 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eed48d22-feb5-474c-b432-42cffd0659ae admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 16:11:36.929076 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eed48d22-feb5-474c-b432-42cffd0659ae admin admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 16:11:36.939669 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 1031/2039] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Aug 22 16:11:36 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 22 16:11:36.943874 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81a82286-511b-49fd-9a7d-10bf97fe5cc0 admin admin] Authenticating user token {{(pid=81767) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 16:11:36.959796 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81a82286-511b-49fd-9a7d-10bf97fe5cc0 admin admin] Validating token access rules against request {{(pid=81767) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 16:11:36.967735 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-004c6962-3734-44da-8837-baa904026c7d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a98b1e4431940ab8d1fc95df2858a6c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81767) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 16:11:36.968813 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-004c6962-3734-44da-8837-baa904026c7d admin admin] REQUEST_METHOD: `PUT` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:11:36.969176 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-004c6962-3734-44da-8837-baa904026c7d admin admin] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:11:36.969555 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-004c6962-3734-44da-8837-baa904026c7d admin admin] PATH_INFO: `/v3/projects/1d9445484db94d76bf676b2f24693c32/users/ffb8485c36c147239e279087f49735dd/roles/9d9848cfd22e4b089db443c350e56670` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:11:36.971896 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-004c6962-3734-44da-8837-baa904026c7d admin admin] RBAC: Authorizing `identity:create_grant(project_id=1d9445484db94d76bf676b2f24693c32, user_id=ffb8485c36c147239e279087f49735dd, role_id=9d9848cfd22e4b089db443c350e56670)` {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 16:11:36.975947 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-004c6962-3734-44da-8837-baa904026c7d admin admin] RBAC: Authorization granted {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 16:11:36.990653 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 1009/2040] 10.222.0.15 () {60 vars in 1483 bytes} [Mon Aug 22 16:11:36 2022] PUT /identity/v3/projects/1d9445484db94d76bf676b2f24693c32/users/ffb8485c36c147239e279087f49735dd/roles/9d9848cfd22e4b089db443c350e56670 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 16:11:37.002838 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73e6337f-bc7c-4a15-a49c-b73b05874137 None None] REQUEST_METHOD: `POST` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:11:37.003358 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73e6337f-bc7c-4a15-a49c-b73b05874137 None None] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:11:37.003859 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73e6337f-bc7c-4a15-a49c-b73b05874137 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:11:37.037464 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.auth.core [None req-73e6337f-bc7c-4a15-a49c-b73b05874137 None None] MFA Rules not processed for user `ffb8485c36c147239e279087f49735dd`. Rule list: `[]` (Enabled: `True`). {{(pid=81768) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 16:11:37.039008 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-73e6337f-bc7c-4a15-a49c-b73b05874137 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 16:11:37.127073 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 1032/2041] 10.222.0.15 () {58 vars in 958 bytes} [Mon Aug 22 16:11:37 2022] POST /identity/v3/auth/tokens => generated 3337 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 16:11:37.265534 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-004c6962-3734-44da-8837-baa904026c7d admin admin] Authenticating user token {{(pid=81767) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 16:11:37.270615 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.fernet_utils [None req-004c6962-3734-44da-8837-baa904026c7d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81767) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 16:11:37.324819 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-004c6962-3734-44da-8837-baa904026c7d admin admin] Validating token access rules against request {{(pid=81767) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 16:11:37.328442 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c0ecb8c-3768-4ea8-9e45-55f8e9269faf service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79706d480f864998b9f8594be407ceab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4ee0413f9ef4d4fb1f95a660babe090', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81767) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 16:11:37.329047 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c0ecb8c-3768-4ea8-9e45-55f8e9269faf service cinder] REQUEST_METHOD: `GET` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:11:37.329318 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c0ecb8c-3768-4ea8-9e45-55f8e9269faf service cinder] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:11:37.329584 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c0ecb8c-3768-4ea8-9e45-55f8e9269faf service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:11:37.343149 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c0ecb8c-3768-4ea8-9e45-55f8e9269faf service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 16:11:37.344831 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c0ecb8c-3768-4ea8-9e45-55f8e9269faf service cinder] RBAC: Authorization granted {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 16:11:37.365474 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 1010/2042] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Aug 22 16:11:37 2022] GET /identity/v3/auth/tokens => generated 3337 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 16:11:37.812314 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73e6337f-bc7c-4a15-a49c-b73b05874137 None None] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 16:11:37.825166 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73e6337f-bc7c-4a15-a49c-b73b05874137 None None] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 16:11:37.850083 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9642412b-5680-4629-bc37-61d95c531e1f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79706d480f864998b9f8594be407ceab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4ee0413f9ef4d4fb1f95a660babe090', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 16:11:37.850083 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9642412b-5680-4629-bc37-61d95c531e1f service cinder] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:11:37.850083 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9642412b-5680-4629-bc37-61d95c531e1f service cinder] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:11:37.850083 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9642412b-5680-4629-bc37-61d95c531e1f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:11:37.905981 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9642412b-5680-4629-bc37-61d95c531e1f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 16:11:37.907599 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9642412b-5680-4629-bc37-61d95c531e1f service cinder] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 16:11:37.917564 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 1033/2043] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Aug 22 16:11:37 2022] GET /identity/v3/auth/tokens => generated 3268 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 16:11:38.104053 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c0ecb8c-3768-4ea8-9e45-55f8e9269faf service cinder] Authenticating user token {{(pid=81767) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 16:11:38.108605 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG dbcounter [-] [81767] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81767) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 16:11:38.175159 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c0ecb8c-3768-4ea8-9e45-55f8e9269faf service cinder] Validating token access rules against request {{(pid=81767) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 16:11:38.180053 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b38e93f-cdcf-4d54-80e7-5f6e7678ffd2 tempest-VolumeQuotaClassesTest-1744796405 tempest-VolumeQuotaClassesTest-1744796405-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f5329d1f1baa4439a5a0b8f220115825', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50bef07e316c4c5e996aab33974e32bc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81767) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 16:11:38.180647 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b38e93f-cdcf-4d54-80e7-5f6e7678ffd2 tempest-VolumeQuotaClassesTest-1744796405 tempest-VolumeQuotaClassesTest-1744796405-project-admin] REQUEST_METHOD: `GET` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:11:38.180963 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b38e93f-cdcf-4d54-80e7-5f6e7678ffd2 tempest-VolumeQuotaClassesTest-1744796405 tempest-VolumeQuotaClassesTest-1744796405-project-admin] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:11:38.181245 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b38e93f-cdcf-4d54-80e7-5f6e7678ffd2 tempest-VolumeQuotaClassesTest-1744796405 tempest-VolumeQuotaClassesTest-1744796405-project-admin] PATH_INFO: `/v3/domains` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:11:38.181912 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b38e93f-cdcf-4d54-80e7-5f6e7678ffd2 tempest-VolumeQuotaClassesTest-1744796405 tempest-VolumeQuotaClassesTest-1744796405-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=81767) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 16:11:38.182217 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b38e93f-cdcf-4d54-80e7-5f6e7678ffd2 tempest-VolumeQuotaClassesTest-1744796405 tempest-VolumeQuotaClassesTest-1744796405-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 16:11:38.183891 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b38e93f-cdcf-4d54-80e7-5f6e7678ffd2 tempest-VolumeQuotaClassesTest-1744796405 tempest-VolumeQuotaClassesTest-1744796405-project-admin] RBAC: Authorization granted {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 16:11:38.196326 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 1011/2044] 10.222.0.15 () {58 vars in 1153 bytes} [Mon Aug 22 16:11:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 93 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 16:11:38.201104 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9642412b-5680-4629-bc37-61d95c531e1f service cinder] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 16:11:38.213195 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9642412b-5680-4629-bc37-61d95c531e1f service cinder] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 16:11:38.217989 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6b17682-5a38-4828-98a1-710994eb18ff tempest-VolumeQuotaClassesTest-1744796405 tempest-VolumeQuotaClassesTest-1744796405-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f5329d1f1baa4439a5a0b8f220115825', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50bef07e316c4c5e996aab33974e32bc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 16:11:38.219048 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6b17682-5a38-4828-98a1-710994eb18ff tempest-VolumeQuotaClassesTest-1744796405 tempest-VolumeQuotaClassesTest-1744796405-project-admin] REQUEST_METHOD: `POST` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:11:38.220398 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6b17682-5a38-4828-98a1-710994eb18ff tempest-VolumeQuotaClassesTest-1744796405 tempest-VolumeQuotaClassesTest-1744796405-project-admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:11:38.221696 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6b17682-5a38-4828-98a1-710994eb18ff tempest-VolumeQuotaClassesTest-1744796405 tempest-VolumeQuotaClassesTest-1744796405-project-admin] PATH_INFO: `/v3/projects` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:11:38.221903 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6b17682-5a38-4828-98a1-710994eb18ff tempest-VolumeQuotaClassesTest-1744796405 tempest-VolumeQuotaClassesTest-1744796405-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 16:11:38.224983 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6b17682-5a38-4828-98a1-710994eb18ff tempest-VolumeQuotaClassesTest-1744796405 tempest-VolumeQuotaClassesTest-1744796405-project-admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 16:11:38.361468 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b38e93f-cdcf-4d54-80e7-5f6e7678ffd2 tempest-VolumeQuotaClassesTest-1744796405 tempest-VolumeQuotaClassesTest-1744796405-project-admin] Authenticating user token {{(pid=81767) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 16:11:38.375364 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 1034/2045] 10.222.0.15 () {60 vars in 1153 bytes} [Mon Aug 22 16:11:38 2022] POST /identity/v3/projects => generated 354 bytes in 174 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 22 16:11:38.392158 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b38e93f-cdcf-4d54-80e7-5f6e7678ffd2 tempest-VolumeQuotaClassesTest-1744796405 tempest-VolumeQuotaClassesTest-1744796405-project-admin] Validating token access rules against request {{(pid=81767) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 16:11:38.444759 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=83,keystone:DELETE=15,keystone:INSERT=2 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 16:11:38.501007 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eba967df-b314-401e-b92c-e8d98c4de3aa tempest-VolumeQuotaClassesTest-1744796405 tempest-VolumeQuotaClassesTest-1744796405-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f5329d1f1baa4439a5a0b8f220115825', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50bef07e316c4c5e996aab33974e32bc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81767) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 16:11:38.501889 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eba967df-b314-401e-b92c-e8d98c4de3aa tempest-VolumeQuotaClassesTest-1744796405 tempest-VolumeQuotaClassesTest-1744796405-project-admin] REQUEST_METHOD: `GET` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:11:38.502017 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eba967df-b314-401e-b92c-e8d98c4de3aa tempest-VolumeQuotaClassesTest-1744796405 tempest-VolumeQuotaClassesTest-1744796405-project-admin] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:11:38.502217 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eba967df-b314-401e-b92c-e8d98c4de3aa tempest-VolumeQuotaClassesTest-1744796405 tempest-VolumeQuotaClassesTest-1744796405-project-admin] PATH_INFO: `/v3/domains` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:11:38.502869 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eba967df-b314-401e-b92c-e8d98c4de3aa tempest-VolumeQuotaClassesTest-1744796405 tempest-VolumeQuotaClassesTest-1744796405-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=81767) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 16:11:38.503177 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eba967df-b314-401e-b92c-e8d98c4de3aa tempest-VolumeQuotaClassesTest-1744796405 tempest-VolumeQuotaClassesTest-1744796405-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 16:11:38.504903 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eba967df-b314-401e-b92c-e8d98c4de3aa tempest-VolumeQuotaClassesTest-1744796405 tempest-VolumeQuotaClassesTest-1744796405-project-admin] RBAC: Authorization granted {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 16:11:38.521515 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 1012/2046] 10.222.0.15 () {58 vars in 1153 bytes} [Mon Aug 22 16:11:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 16:11:38.577321 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6b17682-5a38-4828-98a1-710994eb18ff tempest-VolumeQuotaClassesTest-1744796405 tempest-VolumeQuotaClassesTest-1744796405-project-admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 16:11:38.593654 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6b17682-5a38-4828-98a1-710994eb18ff tempest-VolumeQuotaClassesTest-1744796405 tempest-VolumeQuotaClassesTest-1744796405-project-admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 16:11:38.612427 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95eea76c-5dac-4d07-8170-dc7b0fbd6157 tempest-VolumeQuotaClassesTest-1744796405 tempest-VolumeQuotaClassesTest-1744796405-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f5329d1f1baa4439a5a0b8f220115825', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50bef07e316c4c5e996aab33974e32bc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 16:11:38.614446 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95eea76c-5dac-4d07-8170-dc7b0fbd6157 tempest-VolumeQuotaClassesTest-1744796405 tempest-VolumeQuotaClassesTest-1744796405-project-admin] REQUEST_METHOD: `DELETE` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:11:38.615043 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95eea76c-5dac-4d07-8170-dc7b0fbd6157 tempest-VolumeQuotaClassesTest-1744796405 tempest-VolumeQuotaClassesTest-1744796405-project-admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:11:38.615592 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95eea76c-5dac-4d07-8170-dc7b0fbd6157 tempest-VolumeQuotaClassesTest-1744796405 tempest-VolumeQuotaClassesTest-1744796405-project-admin] PATH_INFO: `/v3/projects/e4f3fb9d3ea645b2b0a7b589f1e31692` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:11:38.620601 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95eea76c-5dac-4d07-8170-dc7b0fbd6157 tempest-VolumeQuotaClassesTest-1744796405 tempest-VolumeQuotaClassesTest-1744796405-project-admin] RBAC: Authorizing `identity:delete_project(project_id=e4f3fb9d3ea645b2b0a7b589f1e31692)` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 16:11:38.624760 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95eea76c-5dac-4d07-8170-dc7b0fbd6157 tempest-VolumeQuotaClassesTest-1744796405 tempest-VolumeQuotaClassesTest-1744796405-project-admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 16:11:38.770038 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-95eea76c-5dac-4d07-8170-dc7b0fbd6157 tempest-VolumeQuotaClassesTest-1744796405 tempest-VolumeQuotaClassesTest-1744796405-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e4f3fb9d3ea645b2b0a7b589f1e31692', 'request_id': 'req-95eea76c-5dac-4d07-8170-dc7b0fbd6157'} {{(pid=81768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 16:11:38.777035 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-95eea76c-5dac-4d07-8170-dc7b0fbd6157 tempest-VolumeQuotaClassesTest-1744796405 tempest-VolumeQuotaClassesTest-1744796405-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e4f3fb9d3ea645b2b0a7b589f1e31692', 'request_id': 'req-95eea76c-5dac-4d07-8170-dc7b0fbd6157'} {{(pid=81768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 16:11:38.894483 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-95eea76c-5dac-4d07-8170-dc7b0fbd6157 tempest-VolumeQuotaClassesTest-1744796405 tempest-VolumeQuotaClassesTest-1744796405-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e4f3fb9d3ea645b2b0a7b589f1e31692', 'request_id': 'req-95eea76c-5dac-4d07-8170-dc7b0fbd6157'} {{(pid=81768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 16:11:38.913875 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-95eea76c-5dac-4d07-8170-dc7b0fbd6157 tempest-VolumeQuotaClassesTest-1744796405 tempest-VolumeQuotaClassesTest-1744796405-project-admin] The token cache is being invalidate because project e4f3fb9d3ea645b2b0a7b589f1e31692 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81768) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 22 16:11:38.914414 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-95eea76c-5dac-4d07-8170-dc7b0fbd6157 tempest-VolumeQuotaClassesTest-1744796405 tempest-VolumeQuotaClassesTest-1744796405-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 16:11:38.915345 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-95eea76c-5dac-4d07-8170-dc7b0fbd6157 tempest-VolumeQuotaClassesTest-1744796405 tempest-VolumeQuotaClassesTest-1744796405-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 16:11:38.917025 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 1035/2047] 10.222.0.15 () {58 vars in 1233 bytes} [Mon Aug 22 16:11:38 2022] DELETE /identity/v3/projects/e4f3fb9d3ea645b2b0a7b589f1e31692 => generated 0 bytes in 340 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 16:11:39.357919 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG dbcounter [-] [81767] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81767) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 16:11:39.573895 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eba967df-b314-401e-b92c-e8d98c4de3aa tempest-VolumeQuotaClassesTest-1744796405 tempest-VolumeQuotaClassesTest-1744796405-project-admin] Authenticating user token {{(pid=81767) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 16:11:39.575642 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.fernet_utils [None req-eba967df-b314-401e-b92c-e8d98c4de3aa tempest-VolumeQuotaClassesTest-1744796405 tempest-VolumeQuotaClassesTest-1744796405-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81767) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 16:11:39.661673 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eba967df-b314-401e-b92c-e8d98c4de3aa tempest-VolumeQuotaClassesTest-1744796405 tempest-VolumeQuotaClassesTest-1744796405-project-admin] Validating token access rules against request {{(pid=81767) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 16:11:39.723896 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afad5896-ff51-40a6-a8cb-ef01148fc4ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a98b1e4431940ab8d1fc95df2858a6c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81767) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 16:11:39.724521 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afad5896-ff51-40a6-a8cb-ef01148fc4ba admin admin] REQUEST_METHOD: `DELETE` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:11:39.724797 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afad5896-ff51-40a6-a8cb-ef01148fc4ba admin admin] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:11:39.725094 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afad5896-ff51-40a6-a8cb-ef01148fc4ba admin admin] PATH_INFO: `/v3/users/fd4fd72776794f84a3dca6d8fb59b4d1` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:11:39.726493 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afad5896-ff51-40a6-a8cb-ef01148fc4ba admin admin] RBAC: Authorizing `identity:delete_user(user_id=fd4fd72776794f84a3dca6d8fb59b4d1)` {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 16:11:39.728567 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afad5896-ff51-40a6-a8cb-ef01148fc4ba admin admin] RBAC: Authorization granted {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 16:11:40.002343 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.notifications [None req-afad5896-ff51-40a6-a8cb-ef01148fc4ba admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fd4fd72776794f84a3dca6d8fb59b4d1'} {{(pid=81767) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 16:11:40.003496 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.notifications [None req-afad5896-ff51-40a6-a8cb-ef01148fc4ba admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fd4fd72776794f84a3dca6d8fb59b4d1'} {{(pid=81767) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 16:11:40.090140 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.notifications [None req-afad5896-ff51-40a6-a8cb-ef01148fc4ba admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fd4fd72776794f84a3dca6d8fb59b4d1'} {{(pid=81767) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 16:11:40.098765 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.notifications [None req-afad5896-ff51-40a6-a8cb-ef01148fc4ba admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fd4fd72776794f84a3dca6d8fb59b4d1'} {{(pid=81767) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 16:11:40.114730 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.notifications [None req-afad5896-ff51-40a6-a8cb-ef01148fc4ba admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fd4fd72776794f84a3dca6d8fb59b4d1'} {{(pid=81767) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 16:11:40.121607 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 1013/2048] 10.222.0.15 () {58 vars in 1224 bytes} [Mon Aug 22 16:11:39 2022] DELETE /identity/v3/users/fd4fd72776794f84a3dca6d8fb59b4d1 => generated 0 bytes in 548 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 16:11:40.126768 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a65b95e4-9402-43aa-b2e6-99c1d08157f3 None None] REQUEST_METHOD: `POST` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:11:40.127244 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a65b95e4-9402-43aa-b2e6-99c1d08157f3 None None] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:11:40.127440 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a65b95e4-9402-43aa-b2e6-99c1d08157f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:11:40.152703 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.auth.core [None req-a65b95e4-9402-43aa-b2e6-99c1d08157f3 None None] MFA Rules not processed for user `6b2199ae0fb0480aa917ada391f5ec54`. Rule list: `[]` (Enabled: `True`). {{(pid=81768) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 16:11:40.154170 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-a65b95e4-9402-43aa-b2e6-99c1d08157f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 16:11:40.227241 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 1036/2049] 10.222.0.15 () {58 vars in 958 bytes} [Mon Aug 22 16:11:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 16:11:40.239017 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afad5896-ff51-40a6-a8cb-ef01148fc4ba admin admin] Authenticating user token {{(pid=81767) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 16:11:40.241890 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.fernet_utils [None req-afad5896-ff51-40a6-a8cb-ef01148fc4ba admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81767) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 16:11:40.272372 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:DELETE=2,keystone:SELECT=98 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 16:11:40.303673 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afad5896-ff51-40a6-a8cb-ef01148fc4ba admin admin] Validating token access rules against request {{(pid=81767) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 16:11:40.307446 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaf1ebd8-862f-415e-a500-74c843251607 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '234687adad594f1aa45775a565ffbdc7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4ee0413f9ef4d4fb1f95a660babe090', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81767) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 16:11:40.308172 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaf1ebd8-862f-415e-a500-74c843251607 service neutron] REQUEST_METHOD: `GET` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:11:40.308450 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaf1ebd8-862f-415e-a500-74c843251607 service neutron] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:11:40.308746 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaf1ebd8-862f-415e-a500-74c843251607 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:11:40.318528 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaf1ebd8-862f-415e-a500-74c843251607 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 16:11:40.319997 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaf1ebd8-862f-415e-a500-74c843251607 service neutron] RBAC: Authorization granted {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 16:11:40.330089 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 1014/2050] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Aug 22 16:11:40 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 22 16:11:40.536350 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG dbcounter [-] [81767] Writing DB stats keystone:DELETE=7,keystone:SELECT=92,keystone:INSERT=1 {{(pid=81767) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 16:11:40.741926 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a65b95e4-9402-43aa-b2e6-99c1d08157f3 None None] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 16:11:40.743631 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-a65b95e4-9402-43aa-b2e6-99c1d08157f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 16:11:40.757705 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a65b95e4-9402-43aa-b2e6-99c1d08157f3 None None] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 16:11:40.763604 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fbd041c-0da7-453d-9e7d-00533ad04c4b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a98b1e4431940ab8d1fc95df2858a6c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 16:11:40.764246 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fbd041c-0da7-453d-9e7d-00533ad04c4b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:11:40.764523 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fbd041c-0da7-453d-9e7d-00533ad04c4b admin admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:11:40.764784 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fbd041c-0da7-453d-9e7d-00533ad04c4b admin admin] PATH_INFO: `/v3/users/be4684a264444a52ac5488b5b2ea5ff1` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:11:40.765921 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fbd041c-0da7-453d-9e7d-00533ad04c4b admin admin] RBAC: Authorizing `identity:delete_user(user_id=be4684a264444a52ac5488b5b2ea5ff1)` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 16:11:40.768556 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fbd041c-0da7-453d-9e7d-00533ad04c4b admin admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 16:11:41.004747 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaf1ebd8-862f-415e-a500-74c843251607 service neutron] Authenticating user token {{(pid=81767) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 16:11:41.006251 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.fernet_utils [None req-aaf1ebd8-862f-415e-a500-74c843251607 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81767) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 16:11:41.026913 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaf1ebd8-862f-415e-a500-74c843251607 service neutron] Validating token access rules against request {{(pid=81767) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 16:11:41.037718 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12be9df6-4f13-4365-ab08-69fdffa6d766 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a98b1e4431940ab8d1fc95df2858a6c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81767) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 16:11:41.038399 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12be9df6-4f13-4365-ab08-69fdffa6d766 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:11:41.038680 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12be9df6-4f13-4365-ab08-69fdffa6d766 admin admin] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:11:41.038955 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12be9df6-4f13-4365-ab08-69fdffa6d766 admin admin] PATH_INFO: `/v3/projects/5ec3508af5984913aa5254de1e5fd2a4` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:11:41.039581 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-9fbd041c-0da7-453d-9e7d-00533ad04c4b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'be4684a264444a52ac5488b5b2ea5ff1'} {{(pid=81768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 16:11:41.040141 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12be9df6-4f13-4365-ab08-69fdffa6d766 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5ec3508af5984913aa5254de1e5fd2a4)` {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 16:11:41.040301 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-9fbd041c-0da7-453d-9e7d-00533ad04c4b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'be4684a264444a52ac5488b5b2ea5ff1'} {{(pid=81768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 16:11:41.041058 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-9fbd041c-0da7-453d-9e7d-00533ad04c4b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'be4684a264444a52ac5488b5b2ea5ff1'} {{(pid=81768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 16:11:41.042299 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12be9df6-4f13-4365-ab08-69fdffa6d766 admin admin] RBAC: Authorization granted {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 16:11:41.047978 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-9fbd041c-0da7-453d-9e7d-00533ad04c4b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'be4684a264444a52ac5488b5b2ea5ff1'} {{(pid=81768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 16:11:41.110585 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-9fbd041c-0da7-453d-9e7d-00533ad04c4b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'be4684a264444a52ac5488b5b2ea5ff1'} {{(pid=81768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 16:11:41.130398 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 1037/2051] 10.222.0.15 () {58 vars in 1224 bytes} [Mon Aug 22 16:11:40 2022] DELETE /identity/v3/users/be4684a264444a52ac5488b5b2ea5ff1 => generated 0 bytes in 389 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 16:11:41.151910 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.notifications [None req-12be9df6-4f13-4365-ab08-69fdffa6d766 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5ec3508af5984913aa5254de1e5fd2a4', 'request_id': 'req-12be9df6-4f13-4365-ab08-69fdffa6d766'} {{(pid=81767) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 16:11:41.157959 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.notifications [None req-12be9df6-4f13-4365-ab08-69fdffa6d766 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5ec3508af5984913aa5254de1e5fd2a4', 'request_id': 'req-12be9df6-4f13-4365-ab08-69fdffa6d766'} {{(pid=81767) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 16:11:41.255648 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.notifications [None req-12be9df6-4f13-4365-ab08-69fdffa6d766 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5ec3508af5984913aa5254de1e5fd2a4', 'request_id': 'req-12be9df6-4f13-4365-ab08-69fdffa6d766'} {{(pid=81767) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 16:11:41.267796 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.notifications [None req-12be9df6-4f13-4365-ab08-69fdffa6d766 admin admin] The token cache is being invalidate because project 5ec3508af5984913aa5254de1e5fd2a4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81767) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 22 16:11:41.268088 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.notifications [None req-12be9df6-4f13-4365-ab08-69fdffa6d766 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81767) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 16:11:41.268734 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.notifications [None req-12be9df6-4f13-4365-ab08-69fdffa6d766 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81767) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 16:11:41.270415 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 1015/2052] 10.222.0.15 () {58 vars in 1233 bytes} [Mon Aug 22 16:11:41 2022] DELETE /identity/v3/projects/5ec3508af5984913aa5254de1e5fd2a4 => generated 0 bytes in 267 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 22 16:11:41.274564 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fbd041c-0da7-453d-9e7d-00533ad04c4b admin admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 16:11:41.276124 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-9fbd041c-0da7-453d-9e7d-00533ad04c4b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 16:11:41.343395 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fbd041c-0da7-453d-9e7d-00533ad04c4b admin admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 16:11:41.399519 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0a64823-61ef-496e-aa6e-5d5980d400eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a98b1e4431940ab8d1fc95df2858a6c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 16:11:41.400137 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a64823-61ef-496e-aa6e-5d5980d400eb admin admin] REQUEST_METHOD: `DELETE` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:11:41.400410 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a64823-61ef-496e-aa6e-5d5980d400eb admin admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:11:41.400675 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a64823-61ef-496e-aa6e-5d5980d400eb admin admin] PATH_INFO: `/v3/users/f5329d1f1baa4439a5a0b8f220115825` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:11:41.401796 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0a64823-61ef-496e-aa6e-5d5980d400eb admin admin] RBAC: Authorizing `identity:delete_user(user_id=f5329d1f1baa4439a5a0b8f220115825)` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 16:11:41.403807 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0a64823-61ef-496e-aa6e-5d5980d400eb admin admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 16:11:41.412609 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12be9df6-4f13-4365-ab08-69fdffa6d766 admin admin] Authenticating user token {{(pid=81767) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 16:11:41.414339 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.fernet_utils [None req-12be9df6-4f13-4365-ab08-69fdffa6d766 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81767) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 16:11:41.429492 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12be9df6-4f13-4365-ab08-69fdffa6d766 admin admin] Validating token access rules against request {{(pid=81767) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 16:11:41.434460 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8206647-9add-4e26-81fe-268024fa8efa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a98b1e4431940ab8d1fc95df2858a6c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81767) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 16:11:41.435182 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8206647-9add-4e26-81fe-268024fa8efa admin admin] REQUEST_METHOD: `DELETE` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:11:41.435556 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8206647-9add-4e26-81fe-268024fa8efa admin admin] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:11:41.435903 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8206647-9add-4e26-81fe-268024fa8efa admin admin] PATH_INFO: `/v3/projects/a9a5e391a11747809428d29256acf9bb` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:11:41.437209 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8206647-9add-4e26-81fe-268024fa8efa admin admin] RBAC: Authorizing `identity:delete_project(project_id=a9a5e391a11747809428d29256acf9bb)` {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 16:11:41.439366 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8206647-9add-4e26-81fe-268024fa8efa admin admin] RBAC: Authorization granted {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 16:11:41.503632 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=94,keystone:INSERT=4,keystone:DELETE=2 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 16:11:41.618481 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-c0a64823-61ef-496e-aa6e-5d5980d400eb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f5329d1f1baa4439a5a0b8f220115825'} {{(pid=81768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 16:11:41.619208 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-c0a64823-61ef-496e-aa6e-5d5980d400eb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f5329d1f1baa4439a5a0b8f220115825'} {{(pid=81768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 16:11:41.620125 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-c0a64823-61ef-496e-aa6e-5d5980d400eb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f5329d1f1baa4439a5a0b8f220115825'} {{(pid=81768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 16:11:41.620241 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.notifications [None req-f8206647-9add-4e26-81fe-268024fa8efa admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a9a5e391a11747809428d29256acf9bb', 'request_id': 'req-f8206647-9add-4e26-81fe-268024fa8efa'} {{(pid=81767) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 16:11:41.625896 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.notifications [None req-f8206647-9add-4e26-81fe-268024fa8efa admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a9a5e391a11747809428d29256acf9bb', 'request_id': 'req-f8206647-9add-4e26-81fe-268024fa8efa'} {{(pid=81767) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 16:11:41.628777 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-c0a64823-61ef-496e-aa6e-5d5980d400eb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f5329d1f1baa4439a5a0b8f220115825'} {{(pid=81768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 16:11:41.724822 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.notifications [None req-f8206647-9add-4e26-81fe-268024fa8efa admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a9a5e391a11747809428d29256acf9bb', 'request_id': 'req-f8206647-9add-4e26-81fe-268024fa8efa'} {{(pid=81767) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 16:11:41.738598 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.notifications [None req-f8206647-9add-4e26-81fe-268024fa8efa admin admin] The token cache is being invalidate because project a9a5e391a11747809428d29256acf9bb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81767) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 22 16:11:41.738889 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.notifications [None req-f8206647-9add-4e26-81fe-268024fa8efa admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81767) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 16:11:41.739614 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.notifications [None req-f8206647-9add-4e26-81fe-268024fa8efa admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81767) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 16:11:41.741335 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 1016/2053] 10.222.0.15 () {58 vars in 1233 bytes} [Mon Aug 22 16:11:41 2022] DELETE /identity/v3/projects/a9a5e391a11747809428d29256acf9bb => generated 0 bytes in 329 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 16:11:41.746202 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7db9023-d09e-4094-900b-621c01036b2d None None] REQUEST_METHOD: `POST` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:11:41.746490 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7db9023-d09e-4094-900b-621c01036b2d None None] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:11:41.746764 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7db9023-d09e-4094-900b-621c01036b2d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:11:41.766491 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.auth.core [None req-c7db9023-d09e-4094-900b-621c01036b2d None None] MFA Rules not processed for user `6b2199ae0fb0480aa917ada391f5ec54`. Rule list: `[]` (Enabled: `True`). {{(pid=81767) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 16:11:41.767742 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.fernet_utils [None req-c7db9023-d09e-4094-900b-621c01036b2d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81767) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 16:11:41.770540 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-c0a64823-61ef-496e-aa6e-5d5980d400eb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f5329d1f1baa4439a5a0b8f220115825'} {{(pid=81768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 16:11:41.796729 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 1038/2054] 10.222.0.15 () {58 vars in 1224 bytes} [Mon Aug 22 16:11:41 2022] DELETE /identity/v3/users/f5329d1f1baa4439a5a0b8f220115825 => generated 0 bytes in 520 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 16:11:41.837436 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 1017/2055] 10.222.0.15 () {58 vars in 958 bytes} [Mon Aug 22 16:11:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 22 16:11:41.840720 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-279c39fa-5ced-4713-a51b-1ed3f2055e60 None None] REQUEST_METHOD: `POST` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:11:41.841042 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-279c39fa-5ced-4713-a51b-1ed3f2055e60 None None] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:11:41.841944 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-279c39fa-5ced-4713-a51b-1ed3f2055e60 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:11:41.862005 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.auth.core [None req-279c39fa-5ced-4713-a51b-1ed3f2055e60 None None] MFA Rules not processed for user `6b2199ae0fb0480aa917ada391f5ec54`. Rule list: `[]` (Enabled: `True`). {{(pid=81768) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 16:11:41.863310 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-279c39fa-5ced-4713-a51b-1ed3f2055e60 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 16:11:41.929043 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 1039/2056] 10.222.0.15 () {58 vars in 958 bytes} [Mon Aug 22 16:11:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 16:11:41.934042 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7db9023-d09e-4094-900b-621c01036b2d None None] Authenticating user token {{(pid=81767) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 16:11:41.944611 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7db9023-d09e-4094-900b-621c01036b2d None None] Validating token access rules against request {{(pid=81767) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 16:11:41.949287 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54afa557-aff2-4c9d-86f0-b0145665d2b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a98b1e4431940ab8d1fc95df2858a6c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81767) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 16:11:41.949897 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54afa557-aff2-4c9d-86f0-b0145665d2b7 admin admin] REQUEST_METHOD: `GET` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:11:41.950176 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54afa557-aff2-4c9d-86f0-b0145665d2b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:11:41.950447 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54afa557-aff2-4c9d-86f0-b0145665d2b7 admin admin] PATH_INFO: `/v3/domains` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:11:41.951099 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54afa557-aff2-4c9d-86f0-b0145665d2b7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81767) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 16:11:41.951434 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54afa557-aff2-4c9d-86f0-b0145665d2b7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 16:11:41.953028 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54afa557-aff2-4c9d-86f0-b0145665d2b7 admin admin] RBAC: Authorization granted {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 16:11:41.966271 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 1018/2057] 10.222.0.15 () {58 vars in 1153 bytes} [Mon Aug 22 16:11:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 16:11:41.970674 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-279c39fa-5ced-4713-a51b-1ed3f2055e60 None None] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 16:11:41.980278 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-279c39fa-5ced-4713-a51b-1ed3f2055e60 None None] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 16:11:41.985069 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1e1faf1-5293-4cc5-8792-dc289635205f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a98b1e4431940ab8d1fc95df2858a6c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 16:11:41.985686 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e1faf1-5293-4cc5-8792-dc289635205f admin admin] REQUEST_METHOD: `POST` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:11:41.985998 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e1faf1-5293-4cc5-8792-dc289635205f admin admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:11:41.986271 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e1faf1-5293-4cc5-8792-dc289635205f admin admin] PATH_INFO: `/v3/projects` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:11:41.986908 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1e1faf1-5293-4cc5-8792-dc289635205f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 16:11:41.997498 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1e1faf1-5293-4cc5-8792-dc289635205f admin admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 16:11:42.082838 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 1040/2058] 10.222.0.15 () {60 vars in 1153 bytes} [Mon Aug 22 16:11:41 2022] POST /identity/v3/projects => generated 355 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 22 16:11:42.084788 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54afa557-aff2-4c9d-86f0-b0145665d2b7 admin admin] Authenticating user token {{(pid=81767) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 16:11:42.095321 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54afa557-aff2-4c9d-86f0-b0145665d2b7 admin admin] Validating token access rules against request {{(pid=81767) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 16:11:42.167433 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da343373-b326-482f-8a5f-3cad38ea7610 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a98b1e4431940ab8d1fc95df2858a6c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81767) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 16:11:42.168058 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da343373-b326-482f-8a5f-3cad38ea7610 admin admin] REQUEST_METHOD: `POST` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:11:42.168445 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da343373-b326-482f-8a5f-3cad38ea7610 admin admin] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:11:42.168602 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da343373-b326-482f-8a5f-3cad38ea7610 admin admin] PATH_INFO: `/v3/users` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:11:42.169435 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da343373-b326-482f-8a5f-3cad38ea7610 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 16:11:42.171407 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da343373-b326-482f-8a5f-3cad38ea7610 admin admin] RBAC: Authorization granted {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 16:11:42.257509 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1e1faf1-5293-4cc5-8792-dc289635205f admin admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 16:11:42.260217 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-c1e1faf1-5293-4cc5-8792-dc289635205f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 16:11:42.279312 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG dbcounter [-] [81767] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81767) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 16:11:42.282239 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1e1faf1-5293-4cc5-8792-dc289635205f admin admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 16:11:42.287329 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 1019/2059] 10.222.0.15 () {60 vars in 1144 bytes} [Mon Aug 22 16:11:42 2022] POST /identity/v3/users => generated 278 bytes in 203 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 22 16:11:42.291225 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da343373-b326-482f-8a5f-3cad38ea7610 admin admin] Authenticating user token {{(pid=81767) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 16:11:42.296133 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-976ad5c3-8118-47ad-ad86-32405d635808 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a98b1e4431940ab8d1fc95df2858a6c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 16:11:42.296783 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-976ad5c3-8118-47ad-ad86-32405d635808 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:11:42.297059 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-976ad5c3-8118-47ad-ad86-32405d635808 admin admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:11:42.297330 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-976ad5c3-8118-47ad-ad86-32405d635808 admin admin] PATH_INFO: `/v3/projects/50bef07e316c4c5e996aab33974e32bc` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:11:42.300948 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-976ad5c3-8118-47ad-ad86-32405d635808 admin admin] RBAC: Authorizing `identity:delete_project(project_id=50bef07e316c4c5e996aab33974e32bc)` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 16:11:42.303397 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da343373-b326-482f-8a5f-3cad38ea7610 admin admin] Validating token access rules against request {{(pid=81767) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 16:11:42.309138 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-976ad5c3-8118-47ad-ad86-32405d635808 admin admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 16:11:42.309631 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78f5c5eb-7872-4eea-9807-6bac203cbb3c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a98b1e4431940ab8d1fc95df2858a6c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81767) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 16:11:42.310240 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f5c5eb-7872-4eea-9807-6bac203cbb3c admin admin] REQUEST_METHOD: `GET` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:11:42.311037 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f5c5eb-7872-4eea-9807-6bac203cbb3c admin admin] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:11:42.311352 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f5c5eb-7872-4eea-9807-6bac203cbb3c admin admin] PATH_INFO: `/v3/roles` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:11:42.312016 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78f5c5eb-7872-4eea-9807-6bac203cbb3c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 16:11:42.313874 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78f5c5eb-7872-4eea-9807-6bac203cbb3c admin admin] RBAC: Authorization granted {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 16:11:42.326247 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 1020/2060] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Aug 22 16:11:42 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 22 16:11:42.329908 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78f5c5eb-7872-4eea-9807-6bac203cbb3c admin admin] Authenticating user token {{(pid=81767) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 16:11:42.340170 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78f5c5eb-7872-4eea-9807-6bac203cbb3c admin admin] Validating token access rules against request {{(pid=81767) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 16:11:42.344893 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-611c694d-9c11-4efe-896b-56bfc34cb586 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a98b1e4431940ab8d1fc95df2858a6c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81767) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 16:11:42.345488 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-611c694d-9c11-4efe-896b-56bfc34cb586 admin admin] REQUEST_METHOD: `GET` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:11:42.345759 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-611c694d-9c11-4efe-896b-56bfc34cb586 admin admin] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:11:42.347921 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-611c694d-9c11-4efe-896b-56bfc34cb586 admin admin] PATH_INFO: `/v3/roles` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:11:42.347921 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-611c694d-9c11-4efe-896b-56bfc34cb586 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 16:11:42.348237 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-611c694d-9c11-4efe-896b-56bfc34cb586 admin admin] RBAC: Authorization granted {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 16:11:42.358191 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 1021/2061] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Aug 22 16:11:42 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 22 16:11:42.362577 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-611c694d-9c11-4efe-896b-56bfc34cb586 admin admin] Authenticating user token {{(pid=81767) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 16:11:42.373375 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-611c694d-9c11-4efe-896b-56bfc34cb586 admin admin] Validating token access rules against request {{(pid=81767) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 16:11:42.381877 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf0752b3-94e6-4bfb-8986-05ebc6b5e231 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a98b1e4431940ab8d1fc95df2858a6c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81767) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 16:11:42.381877 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf0752b3-94e6-4bfb-8986-05ebc6b5e231 admin admin] REQUEST_METHOD: `PUT` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:11:42.381877 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf0752b3-94e6-4bfb-8986-05ebc6b5e231 admin admin] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:11:42.381877 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf0752b3-94e6-4bfb-8986-05ebc6b5e231 admin admin] PATH_INFO: `/v3/projects/5ac04c67c44b48a19e86f367a1f3acc0/users/6199a95ab49f400ca53c846e292a4712/roles/e6fc1ee3fb974125ba9b4fe55b12b45e` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:11:42.394814 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf0752b3-94e6-4bfb-8986-05ebc6b5e231 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5ac04c67c44b48a19e86f367a1f3acc0, user_id=6199a95ab49f400ca53c846e292a4712, role_id=e6fc1ee3fb974125ba9b4fe55b12b45e)` {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 16:11:42.397940 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf0752b3-94e6-4bfb-8986-05ebc6b5e231 admin admin] RBAC: Authorization granted {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 16:11:42.415381 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-976ad5c3-8118-47ad-ad86-32405d635808 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '50bef07e316c4c5e996aab33974e32bc', 'request_id': 'req-976ad5c3-8118-47ad-ad86-32405d635808'} {{(pid=81768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 16:11:42.420947 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-976ad5c3-8118-47ad-ad86-32405d635808 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '50bef07e316c4c5e996aab33974e32bc', 'request_id': 'req-976ad5c3-8118-47ad-ad86-32405d635808'} {{(pid=81768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 16:11:42.490532 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 1022/2062] 10.222.0.15 () {60 vars in 1483 bytes} [Mon Aug 22 16:11:42 2022] PUT /identity/v3/projects/5ac04c67c44b48a19e86f367a1f3acc0/users/6199a95ab49f400ca53c846e292a4712/roles/e6fc1ee3fb974125ba9b4fe55b12b45e => generated 0 bytes in 128 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 22 16:11:42.501473 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a564d71-b263-4a16-9c2d-25b521415ae0 None None] REQUEST_METHOD: `POST` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:11:42.501751 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a564d71-b263-4a16-9c2d-25b521415ae0 None None] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:11:42.502019 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a564d71-b263-4a16-9c2d-25b521415ae0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:11:42.507566 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-976ad5c3-8118-47ad-ad86-32405d635808 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '50bef07e316c4c5e996aab33974e32bc', 'request_id': 'req-976ad5c3-8118-47ad-ad86-32405d635808'} {{(pid=81768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 16:11:42.519381 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-976ad5c3-8118-47ad-ad86-32405d635808 admin admin] The token cache is being invalidate because project 50bef07e316c4c5e996aab33974e32bc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81768) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 22 16:11:42.519666 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-976ad5c3-8118-47ad-ad86-32405d635808 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 16:11:42.520399 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-976ad5c3-8118-47ad-ad86-32405d635808 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 16:11:42.521942 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 1041/2063] 10.222.0.15 () {58 vars in 1233 bytes} [Mon Aug 22 16:11:42 2022] DELETE /identity/v3/projects/50bef07e316c4c5e996aab33974e32bc => generated 0 bytes in 267 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 22 16:11:42.526667 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30164a76-9807-4a8c-afed-1cadc65d33ee None None] REQUEST_METHOD: `POST` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:11:42.527000 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30164a76-9807-4a8c-afed-1cadc65d33ee None None] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:11:42.527332 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30164a76-9807-4a8c-afed-1cadc65d33ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:11:42.532044 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.auth.core [None req-4a564d71-b263-4a16-9c2d-25b521415ae0 None None] MFA Rules not processed for user `6199a95ab49f400ca53c846e292a4712`. Rule list: `[]` (Enabled: `True`). {{(pid=81767) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 16:11:42.533566 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.fernet_utils [None req-4a564d71-b263-4a16-9c2d-25b521415ae0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81767) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 16:11:42.549784 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.auth.core [None req-30164a76-9807-4a8c-afed-1cadc65d33ee None None] MFA Rules not processed for user `6b2199ae0fb0480aa917ada391f5ec54`. Rule list: `[]` (Enabled: `True`). {{(pid=81768) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 16:11:42.551559 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-30164a76-9807-4a8c-afed-1cadc65d33ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 16:11:42.601746 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 1023/2064] 10.222.0.15 () {58 vars in 958 bytes} [Mon Aug 22 16:11:42 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 22 16:11:42.606214 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a564d71-b263-4a16-9c2d-25b521415ae0 None None] Authenticating user token {{(pid=81767) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 16:11:42.607716 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.fernet_utils [None req-4a564d71-b263-4a16-9c2d-25b521415ae0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81767) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 16:11:42.611433 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 1042/2065] 10.222.0.15 () {58 vars in 958 bytes} [Mon Aug 22 16:11:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 16:11:42.620642 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcd83e0f-adbc-47ab-9b9e-c21d63d78073 None None] REQUEST_METHOD: `POST` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:11:42.620983 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcd83e0f-adbc-47ab-9b9e-c21d63d78073 None None] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:11:42.621198 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcd83e0f-adbc-47ab-9b9e-c21d63d78073 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:11:42.626555 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a564d71-b263-4a16-9c2d-25b521415ae0 None None] Validating token access rules against request {{(pid=81767) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 16:11:42.634471 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-659d5990-dfc3-47d0-9f76-a2d7dde3bfa7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a98b1e4431940ab8d1fc95df2858a6c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81767) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 16:11:42.635471 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-659d5990-dfc3-47d0-9f76-a2d7dde3bfa7 admin admin] REQUEST_METHOD: `POST` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:11:42.635891 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-659d5990-dfc3-47d0-9f76-a2d7dde3bfa7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:11:42.636233 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-659d5990-dfc3-47d0-9f76-a2d7dde3bfa7 admin admin] PATH_INFO: `/v3/projects` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:11:42.636901 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-659d5990-dfc3-47d0-9f76-a2d7dde3bfa7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 16:11:42.641369 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-659d5990-dfc3-47d0-9f76-a2d7dde3bfa7 admin admin] RBAC: Authorization granted {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 16:11:42.641564 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.auth.core [None req-fcd83e0f-adbc-47ab-9b9e-c21d63d78073 None None] MFA Rules not processed for user `6b2199ae0fb0480aa917ada391f5ec54`. Rule list: `[]` (Enabled: `True`). {{(pid=81768) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 16:11:42.642744 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-fcd83e0f-adbc-47ab-9b9e-c21d63d78073 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 16:11:42.652463 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 1043/2066] 10.222.0.15 () {58 vars in 958 bytes} [Mon Aug 22 16:11:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 22 16:11:42.658572 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcd83e0f-adbc-47ab-9b9e-c21d63d78073 None None] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 16:11:42.669441 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcd83e0f-adbc-47ab-9b9e-c21d63d78073 None None] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 16:11:42.674939 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40216d31-1b59-4c9d-9c02-7c1324e18b7f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a98b1e4431940ab8d1fc95df2858a6c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 16:11:42.680375 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40216d31-1b59-4c9d-9c02-7c1324e18b7f admin admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:11:42.680375 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40216d31-1b59-4c9d-9c02-7c1324e18b7f admin admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:11:42.680375 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40216d31-1b59-4c9d-9c02-7c1324e18b7f admin admin] PATH_INFO: `/v3/domains` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:11:42.680375 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40216d31-1b59-4c9d-9c02-7c1324e18b7f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81768) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 16:11:42.680375 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40216d31-1b59-4c9d-9c02-7c1324e18b7f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 16:11:42.680375 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40216d31-1b59-4c9d-9c02-7c1324e18b7f admin admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 16:11:42.692021 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 1044/2067] 10.222.0.15 () {58 vars in 1153 bytes} [Mon Aug 22 16:11:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 16:11:42.697322 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40216d31-1b59-4c9d-9c02-7c1324e18b7f admin admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 16:11:42.710302 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40216d31-1b59-4c9d-9c02-7c1324e18b7f admin admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 16:11:42.715045 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4299f6b9-2db7-423e-844f-49998d6ce1af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a98b1e4431940ab8d1fc95df2858a6c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 16:11:42.715677 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4299f6b9-2db7-423e-844f-49998d6ce1af admin admin] REQUEST_METHOD: `POST` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:11:42.716005 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4299f6b9-2db7-423e-844f-49998d6ce1af admin admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:11:42.716285 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4299f6b9-2db7-423e-844f-49998d6ce1af admin admin] PATH_INFO: `/v3/projects` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:11:42.716901 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4299f6b9-2db7-423e-844f-49998d6ce1af admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 16:11:42.718791 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4299f6b9-2db7-423e-844f-49998d6ce1af admin admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 16:11:42.798084 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 1024/2068] 10.222.0.15 () {60 vars in 1153 bytes} [Mon Aug 22 16:11:42 2022] POST /identity/v3/projects => generated 357 bytes in 192 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 22 16:11:42.802775 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-659d5990-dfc3-47d0-9f76-a2d7dde3bfa7 admin admin] Authenticating user token {{(pid=81767) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 16:11:42.820796 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 1045/2069] 10.222.0.15 () {60 vars in 1153 bytes} [Mon Aug 22 16:11:42 2022] POST /identity/v3/projects => generated 377 bytes in 124 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 22 16:11:42.825319 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4299f6b9-2db7-423e-844f-49998d6ce1af admin admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 16:11:42.869794 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-659d5990-dfc3-47d0-9f76-a2d7dde3bfa7 admin admin] Validating token access rules against request {{(pid=81767) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 16:11:42.889454 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4299f6b9-2db7-423e-844f-49998d6ce1af admin admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 16:11:42.894720 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c186a2c-3fb4-41a1-bdf5-b3daa99a2eed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a98b1e4431940ab8d1fc95df2858a6c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 16:11:42.895490 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c186a2c-3fb4-41a1-bdf5-b3daa99a2eed admin admin] REQUEST_METHOD: `POST` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:11:42.895763 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c186a2c-3fb4-41a1-bdf5-b3daa99a2eed admin admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:11:42.896040 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c186a2c-3fb4-41a1-bdf5-b3daa99a2eed admin admin] PATH_INFO: `/v3/users` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:11:42.896660 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c186a2c-3fb4-41a1-bdf5-b3daa99a2eed admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 16:11:42.899180 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c186a2c-3fb4-41a1-bdf5-b3daa99a2eed admin admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 16:11:42.924957 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8669af6-0308-439b-b0e5-c48f4b4249dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a98b1e4431940ab8d1fc95df2858a6c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81767) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 16:11:42.926132 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8669af6-0308-439b-b0e5-c48f4b4249dd admin admin] REQUEST_METHOD: `POST` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:11:42.926411 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8669af6-0308-439b-b0e5-c48f4b4249dd admin admin] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:11:42.926724 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8669af6-0308-439b-b0e5-c48f4b4249dd admin admin] PATH_INFO: `/v3/users` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:11:42.927373 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8669af6-0308-439b-b0e5-c48f4b4249dd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 16:11:42.929341 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8669af6-0308-439b-b0e5-c48f4b4249dd admin admin] RBAC: Authorization granted {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 16:11:42.982047 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 1046/2070] 10.222.0.15 () {60 vars in 1144 bytes} [Mon Aug 22 16:11:42 2022] POST /identity/v3/users => generated 289 bytes in 157 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 22 16:11:42.986955 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c186a2c-3fb4-41a1-bdf5-b3daa99a2eed admin admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 16:11:42.997095 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c186a2c-3fb4-41a1-bdf5-b3daa99a2eed admin admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 16:11:43.001717 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c726f48-4f53-4c64-a5bf-10274ae957ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a98b1e4431940ab8d1fc95df2858a6c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 16:11:43.002339 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c726f48-4f53-4c64-a5bf-10274ae957ab admin admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:11:43.002615 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c726f48-4f53-4c64-a5bf-10274ae957ab admin admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:11:43.002883 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c726f48-4f53-4c64-a5bf-10274ae957ab admin admin] PATH_INFO: `/v3/roles` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:11:43.003601 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c726f48-4f53-4c64-a5bf-10274ae957ab admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 16:11:43.005210 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c726f48-4f53-4c64-a5bf-10274ae957ab admin admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 16:11:43.010193 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 1025/2071] 10.222.0.15 () {60 vars in 1144 bytes} [Mon Aug 22 16:11:42 2022] POST /identity/v3/users => generated 285 bytes in 208 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 22 16:11:43.014160 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8669af6-0308-439b-b0e5-c48f4b4249dd admin admin] Authenticating user token {{(pid=81767) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 16:11:43.017440 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 1047/2072] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Aug 22 16:11:42 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 22 16:11:43.019703 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c726f48-4f53-4c64-a5bf-10274ae957ab admin admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 16:11:43.024603 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8669af6-0308-439b-b0e5-c48f4b4249dd admin admin] Validating token access rules against request {{(pid=81767) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 16:11:43.029667 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c726f48-4f53-4c64-a5bf-10274ae957ab admin admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 16:11:43.030834 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e635b5c6-3c67-46af-803f-3c9ed160f7b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a98b1e4431940ab8d1fc95df2858a6c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81767) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 16:11:43.031560 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e635b5c6-3c67-46af-803f-3c9ed160f7b2 admin admin] REQUEST_METHOD: `GET` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:11:43.031678 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e635b5c6-3c67-46af-803f-3c9ed160f7b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:11:43.031899 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e635b5c6-3c67-46af-803f-3c9ed160f7b2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:11:43.032576 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e635b5c6-3c67-46af-803f-3c9ed160f7b2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 16:11:43.034149 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e635b5c6-3c67-46af-803f-3c9ed160f7b2 admin admin] RBAC: Authorization granted {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 16:11:43.034511 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-273df97f-c8ac-4287-a2f0-932d8df564f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a98b1e4431940ab8d1fc95df2858a6c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 16:11:43.035700 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-273df97f-c8ac-4287-a2f0-932d8df564f0 admin admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:11:43.035980 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-273df97f-c8ac-4287-a2f0-932d8df564f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:11:43.036255 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-273df97f-c8ac-4287-a2f0-932d8df564f0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:11:43.036894 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-273df97f-c8ac-4287-a2f0-932d8df564f0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 16:11:43.038487 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-273df97f-c8ac-4287-a2f0-932d8df564f0 admin admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 16:11:43.045532 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 1026/2073] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Aug 22 16:11:43 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 22 16:11:43.054259 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 1048/2074] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Aug 22 16:11:43 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 22 16:11:43.055156 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-273df97f-c8ac-4287-a2f0-932d8df564f0 admin admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 16:11:43.057221 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e635b5c6-3c67-46af-803f-3c9ed160f7b2 admin admin] Authenticating user token {{(pid=81767) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 16:11:43.065660 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-273df97f-c8ac-4287-a2f0-932d8df564f0 admin admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 16:11:43.066948 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e635b5c6-3c67-46af-803f-3c9ed160f7b2 admin admin] Validating token access rules against request {{(pid=81767) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 16:11:43.071504 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c647ec56-e189-49c9-ae1b-68c98d6ba145 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a98b1e4431940ab8d1fc95df2858a6c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 16:11:43.072445 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c647ec56-e189-49c9-ae1b-68c98d6ba145 admin admin] REQUEST_METHOD: `PUT` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:11:43.072690 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c858c17-6554-4b62-8230-9c2760b79fec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a98b1e4431940ab8d1fc95df2858a6c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81767) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 16:11:43.072901 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c647ec56-e189-49c9-ae1b-68c98d6ba145 admin admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:11:43.073439 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c647ec56-e189-49c9-ae1b-68c98d6ba145 admin admin] PATH_INFO: `/v3/projects/505e7545cef44a3483118ab73b66c5ba/users/12883cc302cc404493734439ba2c3e28/roles/e6fc1ee3fb974125ba9b4fe55b12b45e` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:11:43.073569 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c858c17-6554-4b62-8230-9c2760b79fec admin admin] REQUEST_METHOD: `PUT` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:11:43.073649 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c858c17-6554-4b62-8230-9c2760b79fec admin admin] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:11:43.073874 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c858c17-6554-4b62-8230-9c2760b79fec admin admin] PATH_INFO: `/v3/projects/0be493010bc64873bd83fc32b2d0ca5f/users/e5f4c1d845314a7dadd032b792cbdf7d/roles/9d9848cfd22e4b089db443c350e56670` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:11:43.091442 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c858c17-6554-4b62-8230-9c2760b79fec admin admin] RBAC: Authorizing `identity:create_grant(project_id=0be493010bc64873bd83fc32b2d0ca5f, user_id=e5f4c1d845314a7dadd032b792cbdf7d, role_id=9d9848cfd22e4b089db443c350e56670)` {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 16:11:43.092516 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c647ec56-e189-49c9-ae1b-68c98d6ba145 admin admin] RBAC: Authorizing `identity:create_grant(project_id=505e7545cef44a3483118ab73b66c5ba, user_id=12883cc302cc404493734439ba2c3e28, role_id=e6fc1ee3fb974125ba9b4fe55b12b45e)` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 16:11:43.094105 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c858c17-6554-4b62-8230-9c2760b79fec admin admin] RBAC: Authorization granted {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 16:11:43.095407 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c647ec56-e189-49c9-ae1b-68c98d6ba145 admin admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 16:11:43.194274 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=93,keystone:DELETE=5,keystone:INSERT=2 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 16:11:43.201676 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 1049/2075] 10.222.0.15 () {60 vars in 1483 bytes} [Mon Aug 22 16:11:43 2022] PUT /identity/v3/projects/505e7545cef44a3483118ab73b66c5ba/users/12883cc302cc404493734439ba2c3e28/roles/e6fc1ee3fb974125ba9b4fe55b12b45e => generated 0 bytes in 147 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 16:11:43.205371 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 1027/2076] 10.222.0.15 () {60 vars in 1483 bytes} [Mon Aug 22 16:11:43 2022] PUT /identity/v3/projects/0be493010bc64873bd83fc32b2d0ca5f/users/e5f4c1d845314a7dadd032b792cbdf7d/roles/9d9848cfd22e4b089db443c350e56670 => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 22 16:11:43.212899 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c647ec56-e189-49c9-ae1b-68c98d6ba145 admin admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 16:11:43.213829 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28da07da-4714-42ad-beed-d1a03207f600 None None] REQUEST_METHOD: `POST` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:11:43.214710 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28da07da-4714-42ad-beed-d1a03207f600 None None] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:11:43.215524 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28da07da-4714-42ad-beed-d1a03207f600 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:11:43.264866 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.auth.core [None req-28da07da-4714-42ad-beed-d1a03207f600 None None] MFA Rules not processed for user `12883cc302cc404493734439ba2c3e28`. Rule list: `[]` (Enabled: `True`). {{(pid=81767) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 16:11:43.267381 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.fernet_utils [None req-28da07da-4714-42ad-beed-d1a03207f600 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81767) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 16:11:43.314702 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c647ec56-e189-49c9-ae1b-68c98d6ba145 admin admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 16:11:43.322847 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e40105a3-0383-4755-a641-83ea00136c5f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a98b1e4431940ab8d1fc95df2858a6c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 16:11:43.323770 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e40105a3-0383-4755-a641-83ea00136c5f admin admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:11:43.324220 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e40105a3-0383-4755-a641-83ea00136c5f admin admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:11:43.326101 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e40105a3-0383-4755-a641-83ea00136c5f admin admin] PATH_INFO: `/v3/roles` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:11:43.326101 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e40105a3-0383-4755-a641-83ea00136c5f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 16:11:43.328098 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e40105a3-0383-4755-a641-83ea00136c5f admin admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 16:11:43.352413 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 1050/2077] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Aug 22 16:11:43 2022] GET /identity/v3/roles => generated 1420 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 22 16:11:43.353839 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e40105a3-0383-4755-a641-83ea00136c5f admin admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 16:11:43.379054 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e40105a3-0383-4755-a641-83ea00136c5f admin admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 16:11:43.384696 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8cdefa4-32fa-4c1a-9838-0393e340f155 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a98b1e4431940ab8d1fc95df2858a6c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 16:11:43.385352 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8cdefa4-32fa-4c1a-9838-0393e340f155 admin admin] REQUEST_METHOD: `PUT` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:11:43.385679 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8cdefa4-32fa-4c1a-9838-0393e340f155 admin admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:11:43.385949 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8cdefa4-32fa-4c1a-9838-0393e340f155 admin admin] PATH_INFO: `/v3/projects/0be493010bc64873bd83fc32b2d0ca5f/users/e5f4c1d845314a7dadd032b792cbdf7d/roles/9d9848cfd22e4b089db443c350e56670` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:11:43.388201 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8cdefa4-32fa-4c1a-9838-0393e340f155 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0be493010bc64873bd83fc32b2d0ca5f, user_id=e5f4c1d845314a7dadd032b792cbdf7d, role_id=9d9848cfd22e4b089db443c350e56670)` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 16:11:43.390773 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8cdefa4-32fa-4c1a-9838-0393e340f155 admin admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 16:11:43.405628 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 1028/2078] 10.222.0.15 () {58 vars in 958 bytes} [Mon Aug 22 16:11:43 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 22 16:11:43.406680 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 1051/2079] 10.222.0.15 () {60 vars in 1483 bytes} [Mon Aug 22 16:11:43 2022] PUT /identity/v3/projects/0be493010bc64873bd83fc32b2d0ca5f/users/e5f4c1d845314a7dadd032b792cbdf7d/roles/9d9848cfd22e4b089db443c350e56670 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 22 16:11:43.411680 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG dbcounter [-] [81767] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81767) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 16:11:43.417575 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f009391-2cd4-488d-b1bf-dd59e8e8d284 None None] REQUEST_METHOD: `POST` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:11:43.417929 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f009391-2cd4-488d-b1bf-dd59e8e8d284 None None] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:11:43.418026 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8cdefa4-32fa-4c1a-9838-0393e340f155 admin admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 16:11:43.418210 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f009391-2cd4-488d-b1bf-dd59e8e8d284 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:11:43.420767 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-d8cdefa4-32fa-4c1a-9838-0393e340f155 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 16:11:43.449629 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.auth.core [None req-1f009391-2cd4-488d-b1bf-dd59e8e8d284 None None] MFA Rules not processed for user `e5f4c1d845314a7dadd032b792cbdf7d`. Rule list: `[]` (Enabled: `True`). {{(pid=81767) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 16:11:43.451017 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.fernet_utils [None req-1f009391-2cd4-488d-b1bf-dd59e8e8d284 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81767) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 16:11:43.506539 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8cdefa4-32fa-4c1a-9838-0393e340f155 admin admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 16:11:43.510418 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36f26dfc-cab0-4928-ad2f-d77d74f20ed8 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91f565473c824a15930e2b12863c1ed3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4ee0413f9ef4d4fb1f95a660babe090', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 16:11:43.511017 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36f26dfc-cab0-4928-ad2f-d77d74f20ed8 service glance] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:11:43.511373 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36f26dfc-cab0-4928-ad2f-d77d74f20ed8 service glance] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:11:43.511584 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36f26dfc-cab0-4928-ad2f-d77d74f20ed8 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:11:43.525363 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 1029/2080] 10.222.0.15 () {58 vars in 958 bytes} [Mon Aug 22 16:11:43 2022] POST /identity/v3/auth/tokens => generated 3301 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 16:11:43.537097 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f009391-2cd4-488d-b1bf-dd59e8e8d284 None None] Authenticating user token {{(pid=81767) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 16:11:43.538498 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.fernet_utils [None req-1f009391-2cd4-488d-b1bf-dd59e8e8d284 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81767) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 16:11:43.568976 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36f26dfc-cab0-4928-ad2f-d77d74f20ed8 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 16:11:43.570412 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36f26dfc-cab0-4928-ad2f-d77d74f20ed8 service glance] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 16:11:43.579899 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 1052/2081] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Aug 22 16:11:43 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 162 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 22 16:11:43.598452 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f009391-2cd4-488d-b1bf-dd59e8e8d284 None None] Validating token access rules against request {{(pid=81767) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 16:11:43.602352 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-105a70f6-0306-4ed8-94cc-9700169168f3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79706d480f864998b9f8594be407ceab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4ee0413f9ef4d4fb1f95a660babe090', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81767) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 16:11:43.603052 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-105a70f6-0306-4ed8-94cc-9700169168f3 service cinder] REQUEST_METHOD: `GET` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:11:43.603226 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-105a70f6-0306-4ed8-94cc-9700169168f3 service cinder] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:11:43.603595 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-105a70f6-0306-4ed8-94cc-9700169168f3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:11:43.614863 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-105a70f6-0306-4ed8-94cc-9700169168f3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 16:11:43.616429 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-105a70f6-0306-4ed8-94cc-9700169168f3 service cinder] RBAC: Authorization granted {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 16:11:43.626466 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 1030/2082] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Aug 22 16:11:43 2022] GET /identity/v3/auth/tokens => generated 3301 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 22 16:11:43.888606 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36f26dfc-cab0-4928-ad2f-d77d74f20ed8 service glance] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 16:11:43.898385 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36f26dfc-cab0-4928-ad2f-d77d74f20ed8 service glance] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 16:11:43.901931 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba32704e-6445-4719-b40f-802267c23371 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79706d480f864998b9f8594be407ceab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4ee0413f9ef4d4fb1f95a660babe090', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 16:11:43.902527 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba32704e-6445-4719-b40f-802267c23371 service cinder] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:11:43.902809 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba32704e-6445-4719-b40f-802267c23371 service cinder] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:11:43.903174 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba32704e-6445-4719-b40f-802267c23371 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:11:43.961913 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba32704e-6445-4719-b40f-802267c23371 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 16:11:43.963508 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba32704e-6445-4719-b40f-802267c23371 service cinder] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 16:11:43.973153 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 1053/2083] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Aug 22 16:11:43 2022] GET /identity/v3/auth/tokens => generated 3232 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 16:11:44.387085 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG dbcounter [-] [81767] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81767) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 16:11:44.653761 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 16:11:45.105575 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ceaff99-698c-4b39-b276-211100ec5b5d None None] REQUEST_METHOD: `POST` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:11:45.105843 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ceaff99-698c-4b39-b276-211100ec5b5d None None] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:11:45.106139 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ceaff99-698c-4b39-b276-211100ec5b5d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:11:45.154905 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.auth.core [None req-6ceaff99-698c-4b39-b276-211100ec5b5d None None] MFA Rules not processed for user `058e76354dce412cb78b2875c88f2791`. Rule list: `[]` (Enabled: `True`). {{(pid=81767) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 16:11:45.156132 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.fernet_utils [None req-6ceaff99-698c-4b39-b276-211100ec5b5d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81767) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 16:11:45.225277 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 1031/2084] 10.222.0.15 () {58 vars in 1009 bytes} [Mon Aug 22 16:11:45 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 16:11:45.460021 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba32704e-6445-4719-b40f-802267c23371 service cinder] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 16:11:45.461463 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-ba32704e-6445-4719-b40f-802267c23371 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 16:11:45.572993 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba32704e-6445-4719-b40f-802267c23371 service cinder] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 16:11:45.579483 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2e920da-eb0d-497e-b694-87baae3b5535 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a09d128dee9438283633e51f4834643', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4ee0413f9ef4d4fb1f95a660babe090', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 16:11:45.580150 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2e920da-eb0d-497e-b694-87baae3b5535 service swift] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:11:45.580436 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2e920da-eb0d-497e-b694-87baae3b5535 service swift] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:11:45.580724 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2e920da-eb0d-497e-b694-87baae3b5535 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:11:45.604011 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2e920da-eb0d-497e-b694-87baae3b5535 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 16:11:45.605819 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2e920da-eb0d-497e-b694-87baae3b5535 service swift] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 16:11:45.615932 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 1054/2085] 10.222.0.15 () {60 vars in 1384 bytes} [Mon Aug 22 16:11:45 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 157 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 22 16:11:46.116934 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG dbcounter [-] [81767] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81767) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 16:11:46.518718 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 16:11:46.623312 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ceaff99-698c-4b39-b276-211100ec5b5d None None] Authenticating user token {{(pid=81767) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 16:11:46.624937 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.fernet_utils [None req-6ceaff99-698c-4b39-b276-211100ec5b5d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81767) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 16:11:46.695625 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ceaff99-698c-4b39-b276-211100ec5b5d None None] Validating token access rules against request {{(pid=81767) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 16:11:46.701583 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30bacc81-7fa4-4caf-89e4-954a9b2faa62 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a98b1e4431940ab8d1fc95df2858a6c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81767) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 16:11:46.702357 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30bacc81-7fa4-4caf-89e4-954a9b2faa62 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:11:46.702755 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30bacc81-7fa4-4caf-89e4-954a9b2faa62 admin admin] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:11:46.703070 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30bacc81-7fa4-4caf-89e4-954a9b2faa62 admin admin] PATH_INFO: `/v3/users/9a95d57d9e7d4f8c81a2b6f038448784` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:11:46.704280 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30bacc81-7fa4-4caf-89e4-954a9b2faa62 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9a95d57d9e7d4f8c81a2b6f038448784)` {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 16:11:46.707112 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30bacc81-7fa4-4caf-89e4-954a9b2faa62 admin admin] RBAC: Authorization granted {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 16:11:46.967732 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.notifications [None req-30bacc81-7fa4-4caf-89e4-954a9b2faa62 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9a95d57d9e7d4f8c81a2b6f038448784'} {{(pid=81767) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 16:11:46.968368 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.notifications [None req-30bacc81-7fa4-4caf-89e4-954a9b2faa62 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9a95d57d9e7d4f8c81a2b6f038448784'} {{(pid=81767) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 16:11:47.026996 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.notifications [None req-30bacc81-7fa4-4caf-89e4-954a9b2faa62 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9a95d57d9e7d4f8c81a2b6f038448784'} {{(pid=81767) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 16:11:47.035118 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.notifications [None req-30bacc81-7fa4-4caf-89e4-954a9b2faa62 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9a95d57d9e7d4f8c81a2b6f038448784'} {{(pid=81767) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 16:11:47.048623 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.notifications [None req-30bacc81-7fa4-4caf-89e4-954a9b2faa62 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9a95d57d9e7d4f8c81a2b6f038448784'} {{(pid=81767) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 16:11:47.055442 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 1032/2086] 10.222.0.15 () {58 vars in 1224 bytes} [Mon Aug 22 16:11:46 2022] DELETE /identity/v3/users/9a95d57d9e7d4f8c81a2b6f038448784 => generated 0 bytes in 433 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 16:11:47.062532 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e2d2b63-b33d-488c-bd41-9b588ece41a4 None None] REQUEST_METHOD: `POST` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:11:47.062532 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e2d2b63-b33d-488c-bd41-9b588ece41a4 None None] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:11:47.062532 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e2d2b63-b33d-488c-bd41-9b588ece41a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:11:47.083043 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.auth.core [None req-7e2d2b63-b33d-488c-bd41-9b588ece41a4 None None] MFA Rules not processed for user `6b2199ae0fb0480aa917ada391f5ec54`. Rule list: `[]` (Enabled: `True`). {{(pid=81768) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 16:11:47.084701 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-7e2d2b63-b33d-488c-bd41-9b588ece41a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 16:11:47.162321 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 1055/2087] 10.222.0.15 () {58 vars in 958 bytes} [Mon Aug 22 16:11:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 16:11:47.172468 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30bacc81-7fa4-4caf-89e4-954a9b2faa62 admin admin] Authenticating user token {{(pid=81767) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 16:11:47.174108 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.fernet_utils [None req-30bacc81-7fa4-4caf-89e4-954a9b2faa62 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81767) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 16:11:47.246197 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30bacc81-7fa4-4caf-89e4-954a9b2faa62 admin admin] Validating token access rules against request {{(pid=81767) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 16:11:47.250208 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c731ab42-5cfb-4865-a260-2d052b671592 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '234687adad594f1aa45775a565ffbdc7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4ee0413f9ef4d4fb1f95a660babe090', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81767) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 16:11:47.250805 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c731ab42-5cfb-4865-a260-2d052b671592 service neutron] REQUEST_METHOD: `GET` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:11:47.251082 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c731ab42-5cfb-4865-a260-2d052b671592 service neutron] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:11:47.251386 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c731ab42-5cfb-4865-a260-2d052b671592 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:11:47.261236 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c731ab42-5cfb-4865-a260-2d052b671592 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 16:11:47.262665 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c731ab42-5cfb-4865-a260-2d052b671592 service neutron] RBAC: Authorization granted {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 16:11:47.273006 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 1033/2088] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Aug 22 16:11:47 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 16:11:47.413488 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG dbcounter [-] [81767] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81767) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 16:11:47.768189 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 16:11:47.775389 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e2d2b63-b33d-488c-bd41-9b588ece41a4 None None] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 16:11:47.776793 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-7e2d2b63-b33d-488c-bd41-9b588ece41a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 16:11:47.792015 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e2d2b63-b33d-488c-bd41-9b588ece41a4 None None] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 16:11:47.797554 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-315fdd99-8c74-4661-8818-d7dc3abbedec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a98b1e4431940ab8d1fc95df2858a6c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 16:11:47.798253 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-315fdd99-8c74-4661-8818-d7dc3abbedec admin admin] REQUEST_METHOD: `DELETE` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:11:47.798666 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-315fdd99-8c74-4661-8818-d7dc3abbedec admin admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:11:47.798953 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-315fdd99-8c74-4661-8818-d7dc3abbedec admin admin] PATH_INFO: `/v3/projects/7f63a07aba354511b3839295aaaf1c0e` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:11:47.801292 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-315fdd99-8c74-4661-8818-d7dc3abbedec admin admin] RBAC: Authorizing `identity:delete_project(project_id=7f63a07aba354511b3839295aaaf1c0e)` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 16:11:47.803618 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-315fdd99-8c74-4661-8818-d7dc3abbedec admin admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 16:11:48.107839 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-315fdd99-8c74-4661-8818-d7dc3abbedec admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7f63a07aba354511b3839295aaaf1c0e', 'request_id': 'req-315fdd99-8c74-4661-8818-d7dc3abbedec'} {{(pid=81768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 16:11:48.113833 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-315fdd99-8c74-4661-8818-d7dc3abbedec admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7f63a07aba354511b3839295aaaf1c0e', 'request_id': 'req-315fdd99-8c74-4661-8818-d7dc3abbedec'} {{(pid=81768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 16:11:48.213622 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-315fdd99-8c74-4661-8818-d7dc3abbedec admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7f63a07aba354511b3839295aaaf1c0e', 'request_id': 'req-315fdd99-8c74-4661-8818-d7dc3abbedec'} {{(pid=81768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 16:11:48.226181 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-315fdd99-8c74-4661-8818-d7dc3abbedec admin admin] The token cache is being invalidate because project 7f63a07aba354511b3839295aaaf1c0e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81768) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 22 16:11:48.226467 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-315fdd99-8c74-4661-8818-d7dc3abbedec admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 16:11:48.227106 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-315fdd99-8c74-4661-8818-d7dc3abbedec admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 16:11:48.228670 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 1056/2089] 10.222.0.15 () {58 vars in 1233 bytes} [Mon Aug 22 16:11:47 2022] DELETE /identity/v3/projects/7f63a07aba354511b3839295aaaf1c0e => generated 0 bytes in 454 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 16:11:48.233879 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b2016e3-578d-4854-a8c7-fb6ab2d92a04 None None] REQUEST_METHOD: `POST` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:11:48.234408 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b2016e3-578d-4854-a8c7-fb6ab2d92a04 None None] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:11:48.234408 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b2016e3-578d-4854-a8c7-fb6ab2d92a04 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:11:48.258861 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.auth.core [None req-4b2016e3-578d-4854-a8c7-fb6ab2d92a04 None None] MFA Rules not processed for user `6b2199ae0fb0480aa917ada391f5ec54`. Rule list: `[]` (Enabled: `True`). {{(pid=81767) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 16:11:48.260241 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.fernet_utils [None req-4b2016e3-578d-4854-a8c7-fb6ab2d92a04 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81767) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 16:11:48.326498 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 1034/2090] 10.222.0.15 () {58 vars in 958 bytes} [Mon Aug 22 16:11:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 22 16:11:48.335917 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aacd5e37-bc7e-4e9d-9bbf-3428efa0de53 None None] REQUEST_METHOD: `POST` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:11:48.336626 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aacd5e37-bc7e-4e9d-9bbf-3428efa0de53 None None] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:11:48.337637 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aacd5e37-bc7e-4e9d-9bbf-3428efa0de53 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:11:48.357490 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.auth.core [None req-aacd5e37-bc7e-4e9d-9bbf-3428efa0de53 None None] MFA Rules not processed for user `6b2199ae0fb0480aa917ada391f5ec54`. Rule list: `[]` (Enabled: `True`). {{(pid=81768) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 16:11:48.359392 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-aacd5e37-bc7e-4e9d-9bbf-3428efa0de53 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 16:11:48.370584 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 1057/2091] 10.222.0.15 () {58 vars in 958 bytes} [Mon Aug 22 16:11:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 16:11:48.375455 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b2016e3-578d-4854-a8c7-fb6ab2d92a04 None None] Authenticating user token {{(pid=81767) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 16:11:48.387572 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b2016e3-578d-4854-a8c7-fb6ab2d92a04 None None] Validating token access rules against request {{(pid=81767) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 16:11:48.398249 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e622817-1cdc-497e-ad1c-4050aace6f72 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a98b1e4431940ab8d1fc95df2858a6c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81767) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 16:11:48.398249 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e622817-1cdc-497e-ad1c-4050aace6f72 admin admin] REQUEST_METHOD: `GET` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:11:48.398249 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e622817-1cdc-497e-ad1c-4050aace6f72 admin admin] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:11:48.398249 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e622817-1cdc-497e-ad1c-4050aace6f72 admin admin] PATH_INFO: `/v3/domains` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:11:48.398836 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e622817-1cdc-497e-ad1c-4050aace6f72 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81767) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 16:11:48.398836 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e622817-1cdc-497e-ad1c-4050aace6f72 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 16:11:48.398836 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e622817-1cdc-497e-ad1c-4050aace6f72 admin admin] RBAC: Authorization granted {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 16:11:48.415088 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 1035/2092] 10.222.0.15 () {58 vars in 1153 bytes} [Mon Aug 22 16:11:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 22 16:11:48.421084 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aacd5e37-bc7e-4e9d-9bbf-3428efa0de53 None None] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 16:11:48.431374 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aacd5e37-bc7e-4e9d-9bbf-3428efa0de53 None None] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 16:11:48.438078 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e30ff91-e42c-45d6-a772-f0ac37c4cfad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a98b1e4431940ab8d1fc95df2858a6c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 16:11:48.438078 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e30ff91-e42c-45d6-a772-f0ac37c4cfad admin admin] REQUEST_METHOD: `POST` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:11:48.438078 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e30ff91-e42c-45d6-a772-f0ac37c4cfad admin admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:11:48.438078 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e30ff91-e42c-45d6-a772-f0ac37c4cfad admin admin] PATH_INFO: `/v3/projects` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:11:48.438508 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e30ff91-e42c-45d6-a772-f0ac37c4cfad admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 16:11:48.440419 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e30ff91-e42c-45d6-a772-f0ac37c4cfad admin admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 16:11:48.558269 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 1058/2093] 10.222.0.15 () {60 vars in 1153 bytes} [Mon Aug 22 16:11:48 2022] POST /identity/v3/projects => generated 357 bytes in 135 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 22 16:11:48.559564 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e622817-1cdc-497e-ad1c-4050aace6f72 admin admin] Authenticating user token {{(pid=81767) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 16:11:48.571482 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e622817-1cdc-497e-ad1c-4050aace6f72 admin admin] Validating token access rules against request {{(pid=81767) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 16:11:48.661099 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7387dc0-5077-4788-9f6a-07df84a3858f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a98b1e4431940ab8d1fc95df2858a6c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81767) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 16:11:48.661834 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7387dc0-5077-4788-9f6a-07df84a3858f admin admin] REQUEST_METHOD: `POST` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:11:48.662115 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7387dc0-5077-4788-9f6a-07df84a3858f admin admin] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:11:48.662428 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7387dc0-5077-4788-9f6a-07df84a3858f admin admin] PATH_INFO: `/v3/users` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:11:48.663089 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7387dc0-5077-4788-9f6a-07df84a3858f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 16:11:48.665214 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7387dc0-5077-4788-9f6a-07df84a3858f admin admin] RBAC: Authorization granted {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 16:11:48.791646 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 1036/2094] 10.222.0.15 () {60 vars in 1144 bytes} [Mon Aug 22 16:11:48 2022] POST /identity/v3/users => generated 279 bytes in 229 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 22 16:11:48.793077 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e30ff91-e42c-45d6-a772-f0ac37c4cfad admin admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 16:11:48.804529 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e30ff91-e42c-45d6-a772-f0ac37c4cfad admin admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 16:11:48.818970 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91ea5e17-480b-4391-ae3c-e0db1a9fcdd7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a98b1e4431940ab8d1fc95df2858a6c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 16:11:48.821029 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91ea5e17-480b-4391-ae3c-e0db1a9fcdd7 admin admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:11:48.821029 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91ea5e17-480b-4391-ae3c-e0db1a9fcdd7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:11:48.821029 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91ea5e17-480b-4391-ae3c-e0db1a9fcdd7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:11:48.821445 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91ea5e17-480b-4391-ae3c-e0db1a9fcdd7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 16:11:48.823063 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91ea5e17-480b-4391-ae3c-e0db1a9fcdd7 admin admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 16:11:48.839548 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 1059/2095] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Aug 22 16:11:48 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 22 16:11:48.843624 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7387dc0-5077-4788-9f6a-07df84a3858f admin admin] Authenticating user token {{(pid=81767) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 16:11:48.854614 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7387dc0-5077-4788-9f6a-07df84a3858f admin admin] Validating token access rules against request {{(pid=81767) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 16:11:48.859529 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fbfee20-4f6d-43d4-b374-4856e4489886 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a98b1e4431940ab8d1fc95df2858a6c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81767) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 16:11:48.863161 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fbfee20-4f6d-43d4-b374-4856e4489886 admin admin] REQUEST_METHOD: `GET` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:11:48.863161 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fbfee20-4f6d-43d4-b374-4856e4489886 admin admin] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:11:48.863161 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fbfee20-4f6d-43d4-b374-4856e4489886 admin admin] PATH_INFO: `/v3/roles` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:11:48.863161 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fbfee20-4f6d-43d4-b374-4856e4489886 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 16:11:48.863623 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fbfee20-4f6d-43d4-b374-4856e4489886 admin admin] RBAC: Authorization granted {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 16:11:48.874283 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 1037/2096] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Aug 22 16:11:48 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 22 16:11:48.878568 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91ea5e17-480b-4391-ae3c-e0db1a9fcdd7 admin admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 16:11:48.888741 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91ea5e17-480b-4391-ae3c-e0db1a9fcdd7 admin admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 16:11:48.893482 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b84a34f7-3cf4-4157-87ef-aba6058e026f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a98b1e4431940ab8d1fc95df2858a6c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 16:11:48.894212 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b84a34f7-3cf4-4157-87ef-aba6058e026f admin admin] REQUEST_METHOD: `PUT` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:11:48.894424 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b84a34f7-3cf4-4157-87ef-aba6058e026f admin admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:11:48.894691 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b84a34f7-3cf4-4157-87ef-aba6058e026f admin admin] PATH_INFO: `/v3/projects/f9b446c6190e410b895b1def7f4b0b28/users/e9cc163981474902b7de3f04e69afd8a/roles/e6fc1ee3fb974125ba9b4fe55b12b45e` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:11:48.910096 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b84a34f7-3cf4-4157-87ef-aba6058e026f admin admin] RBAC: Authorizing `identity:create_grant(project_id=f9b446c6190e410b895b1def7f4b0b28, user_id=e9cc163981474902b7de3f04e69afd8a, role_id=e6fc1ee3fb974125ba9b4fe55b12b45e)` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 16:11:48.912776 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b84a34f7-3cf4-4157-87ef-aba6058e026f admin admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 16:11:48.930014 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG dbcounter [-] [81767] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81767) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 16:11:49.019485 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 1060/2097] 10.222.0.15 () {60 vars in 1483 bytes} [Mon Aug 22 16:11:48 2022] PUT /identity/v3/projects/f9b446c6190e410b895b1def7f4b0b28/users/e9cc163981474902b7de3f04e69afd8a/roles/e6fc1ee3fb974125ba9b4fe55b12b45e => generated 0 bytes in 141 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 22 16:11:49.030890 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af8fa49a-3d1c-4216-9cfd-e5ea9f78f2d1 None None] REQUEST_METHOD: `POST` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:11:49.031226 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af8fa49a-3d1c-4216-9cfd-e5ea9f78f2d1 None None] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:11:49.031588 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af8fa49a-3d1c-4216-9cfd-e5ea9f78f2d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:11:49.065377 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.auth.core [None req-af8fa49a-3d1c-4216-9cfd-e5ea9f78f2d1 None None] MFA Rules not processed for user `e9cc163981474902b7de3f04e69afd8a`. Rule list: `[]` (Enabled: `True`). {{(pid=81767) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 16:11:49.067344 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.fernet_utils [None req-af8fa49a-3d1c-4216-9cfd-e5ea9f78f2d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81767) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 16:11:49.141038 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 1038/2098] 10.222.0.15 () {58 vars in 958 bytes} [Mon Aug 22 16:11:49 2022] POST /identity/v3/auth/tokens => generated 3234 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 16:11:49.335352 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 16:11:49.397028 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b84a34f7-3cf4-4157-87ef-aba6058e026f admin admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 16:11:49.398996 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-b84a34f7-3cf4-4157-87ef-aba6058e026f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 16:11:49.472322 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b84a34f7-3cf4-4157-87ef-aba6058e026f admin admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 16:11:49.475980 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f723b3b-9f23-4ef1-a310-0d449babc7c5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79706d480f864998b9f8594be407ceab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4ee0413f9ef4d4fb1f95a660babe090', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 16:11:49.476574 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f723b3b-9f23-4ef1-a310-0d449babc7c5 service cinder] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:11:49.476863 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f723b3b-9f23-4ef1-a310-0d449babc7c5 service cinder] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:11:49.477136 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f723b3b-9f23-4ef1-a310-0d449babc7c5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:11:49.487967 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f723b3b-9f23-4ef1-a310-0d449babc7c5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 16:11:49.496723 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f723b3b-9f23-4ef1-a310-0d449babc7c5 service cinder] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 16:11:49.500390 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 1061/2099] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Aug 22 16:11:49 2022] GET /identity/v3/auth/tokens => generated 3234 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 16:11:49.516764 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af8fa49a-3d1c-4216-9cfd-e5ea9f78f2d1 None None] Authenticating user token {{(pid=81767) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 16:11:49.589089 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af8fa49a-3d1c-4216-9cfd-e5ea9f78f2d1 None None] Validating token access rules against request {{(pid=81767) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 16:11:49.593751 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f234074b-aa07-4398-8896-778c6e8bc29c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a98b1e4431940ab8d1fc95df2858a6c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81767) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 16:11:49.594381 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f234074b-aa07-4398-8896-778c6e8bc29c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:11:49.594654 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f234074b-aa07-4398-8896-778c6e8bc29c admin admin] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:11:49.594951 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f234074b-aa07-4398-8896-778c6e8bc29c admin admin] PATH_INFO: `/v3/users/e9cc163981474902b7de3f04e69afd8a` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:11:49.596132 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f234074b-aa07-4398-8896-778c6e8bc29c admin admin] RBAC: Authorizing `identity:delete_user(user_id=e9cc163981474902b7de3f04e69afd8a)` {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 16:11:49.598308 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f234074b-aa07-4398-8896-778c6e8bc29c admin admin] RBAC: Authorization granted {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 16:11:50.004569 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.notifications [None req-f234074b-aa07-4398-8896-778c6e8bc29c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e9cc163981474902b7de3f04e69afd8a'} {{(pid=81767) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 16:11:50.005100 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.notifications [None req-f234074b-aa07-4398-8896-778c6e8bc29c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e9cc163981474902b7de3f04e69afd8a'} {{(pid=81767) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 16:11:50.087242 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.notifications [None req-f234074b-aa07-4398-8896-778c6e8bc29c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e9cc163981474902b7de3f04e69afd8a'} {{(pid=81767) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 16:11:50.096203 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.notifications [None req-f234074b-aa07-4398-8896-778c6e8bc29c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e9cc163981474902b7de3f04e69afd8a'} {{(pid=81767) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 16:11:50.110990 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.notifications [None req-f234074b-aa07-4398-8896-778c6e8bc29c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e9cc163981474902b7de3f04e69afd8a'} {{(pid=81767) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 16:11:50.126421 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 1039/2100] 10.222.0.15 () {58 vars in 1224 bytes} [Mon Aug 22 16:11:49 2022] DELETE /identity/v3/users/e9cc163981474902b7de3f04e69afd8a => generated 0 bytes in 602 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 16:11:50.131547 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de6aba61-08f3-46f9-bab4-c5ecc49c9917 None None] REQUEST_METHOD: `POST` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:11:50.132063 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de6aba61-08f3-46f9-bab4-c5ecc49c9917 None None] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:11:50.132165 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de6aba61-08f3-46f9-bab4-c5ecc49c9917 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:11:50.157023 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.auth.core [None req-de6aba61-08f3-46f9-bab4-c5ecc49c9917 None None] MFA Rules not processed for user `6b2199ae0fb0480aa917ada391f5ec54`. Rule list: `[]` (Enabled: `True`). {{(pid=81768) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 16:11:50.157296 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-de6aba61-08f3-46f9-bab4-c5ecc49c9917 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 16:11:50.234920 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 1062/2101] 10.222.0.15 () {58 vars in 958 bytes} [Mon Aug 22 16:11:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 16:11:50.244965 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f234074b-aa07-4398-8896-778c6e8bc29c admin admin] Authenticating user token {{(pid=81767) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 16:11:50.246489 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.fernet_utils [None req-f234074b-aa07-4398-8896-778c6e8bc29c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81767) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 16:11:50.308183 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f234074b-aa07-4398-8896-778c6e8bc29c admin admin] Validating token access rules against request {{(pid=81767) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 16:11:50.312048 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6461e2d-4c3e-4515-bb32-28fcd807abb8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '234687adad594f1aa45775a565ffbdc7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4ee0413f9ef4d4fb1f95a660babe090', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81767) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 16:11:50.312652 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6461e2d-4c3e-4515-bb32-28fcd807abb8 service neutron] REQUEST_METHOD: `GET` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:11:50.312927 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6461e2d-4c3e-4515-bb32-28fcd807abb8 service neutron] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:11:50.313198 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6461e2d-4c3e-4515-bb32-28fcd807abb8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:11:50.332213 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6461e2d-4c3e-4515-bb32-28fcd807abb8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 16:11:50.332213 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6461e2d-4c3e-4515-bb32-28fcd807abb8 service neutron] RBAC: Authorization granted {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 16:11:50.340186 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 1040/2102] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Aug 22 16:11:50 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 16:11:50.405987 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de6aba61-08f3-46f9-bab4-c5ecc49c9917 None None] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 16:11:50.410123 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-de6aba61-08f3-46f9-bab4-c5ecc49c9917 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 16:11:50.427048 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de6aba61-08f3-46f9-bab4-c5ecc49c9917 None None] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 16:11:50.433797 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0633df3-ea86-4d9e-afb5-882e1d7a81b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a98b1e4431940ab8d1fc95df2858a6c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 16:11:50.434482 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0633df3-ea86-4d9e-afb5-882e1d7a81b9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:11:50.434765 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0633df3-ea86-4d9e-afb5-882e1d7a81b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:11:50.435033 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0633df3-ea86-4d9e-afb5-882e1d7a81b9 admin admin] PATH_INFO: `/v3/users/5c9035bfd8c4475fae8a4895804e94a8` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:11:50.436922 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0633df3-ea86-4d9e-afb5-882e1d7a81b9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5c9035bfd8c4475fae8a4895804e94a8)` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 16:11:50.439261 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0633df3-ea86-4d9e-afb5-882e1d7a81b9 admin admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 16:11:50.493521 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG dbcounter [-] [81767] Writing DB stats keystone:SELECT=100 {{(pid=81767) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 16:11:50.653437 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-d0633df3-ea86-4d9e-afb5-882e1d7a81b9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5c9035bfd8c4475fae8a4895804e94a8'} {{(pid=81768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 16:11:50.654314 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-d0633df3-ea86-4d9e-afb5-882e1d7a81b9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5c9035bfd8c4475fae8a4895804e94a8'} {{(pid=81768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 16:11:50.654853 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-d0633df3-ea86-4d9e-afb5-882e1d7a81b9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5c9035bfd8c4475fae8a4895804e94a8'} {{(pid=81768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 16:11:50.663056 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-d0633df3-ea86-4d9e-afb5-882e1d7a81b9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5c9035bfd8c4475fae8a4895804e94a8'} {{(pid=81768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 16:11:50.698407 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 16:11:50.786737 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-d0633df3-ea86-4d9e-afb5-882e1d7a81b9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5c9035bfd8c4475fae8a4895804e94a8'} {{(pid=81768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 16:11:50.805529 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 1063/2103] 10.222.0.15 () {58 vars in 1224 bytes} [Mon Aug 22 16:11:50 2022] DELETE /identity/v3/users/5c9035bfd8c4475fae8a4895804e94a8 => generated 0 bytes in 400 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 16:11:50.810094 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db0db99d-eed9-4633-9cd9-3f32c41a04c5 None None] REQUEST_METHOD: `POST` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:11:50.810626 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db0db99d-eed9-4633-9cd9-3f32c41a04c5 None None] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:11:50.810741 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db0db99d-eed9-4633-9cd9-3f32c41a04c5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:11:50.830207 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.auth.core [None req-db0db99d-eed9-4633-9cd9-3f32c41a04c5 None None] MFA Rules not processed for user `6b2199ae0fb0480aa917ada391f5ec54`. Rule list: `[]` (Enabled: `True`). {{(pid=81767) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 16:11:50.831474 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.fernet_utils [None req-db0db99d-eed9-4633-9cd9-3f32c41a04c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81767) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 16:11:50.891684 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 1041/2104] 10.222.0.15 () {58 vars in 958 bytes} [Mon Aug 22 16:11:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 16:11:50.901570 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0633df3-ea86-4d9e-afb5-882e1d7a81b9 admin admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 16:11:50.903497 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-d0633df3-ea86-4d9e-afb5-882e1d7a81b9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 16:11:50.927798 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db0db99d-eed9-4633-9cd9-3f32c41a04c5 None None] Authenticating user token {{(pid=81767) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 16:11:50.931076 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.fernet_utils [None req-db0db99d-eed9-4633-9cd9-3f32c41a04c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81767) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 16:11:50.932602 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG dbcounter [-] [81767] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81767) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 16:11:50.948640 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db0db99d-eed9-4633-9cd9-3f32c41a04c5 None None] Validating token access rules against request {{(pid=81767) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 16:11:50.962855 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f62ed4c-b66e-4ec8-be6c-7b1dab78ebfe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a98b1e4431940ab8d1fc95df2858a6c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81767) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 16:11:50.962855 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f62ed4c-b66e-4ec8-be6c-7b1dab78ebfe admin admin] REQUEST_METHOD: `DELETE` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:11:50.962855 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f62ed4c-b66e-4ec8-be6c-7b1dab78ebfe admin admin] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:11:50.962855 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f62ed4c-b66e-4ec8-be6c-7b1dab78ebfe admin admin] PATH_INFO: `/v3/projects/f9b446c6190e410b895b1def7f4b0b28` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:11:50.963436 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f62ed4c-b66e-4ec8-be6c-7b1dab78ebfe admin admin] RBAC: Authorizing `identity:delete_project(project_id=f9b446c6190e410b895b1def7f4b0b28)` {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 16:11:50.963436 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f62ed4c-b66e-4ec8-be6c-7b1dab78ebfe admin admin] RBAC: Authorization granted {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 16:11:50.978827 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0633df3-ea86-4d9e-afb5-882e1d7a81b9 admin admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 16:11:50.982644 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42c7bef1-d2c0-4584-afc7-7f20a00deb36 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '234687adad594f1aa45775a565ffbdc7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4ee0413f9ef4d4fb1f95a660babe090', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 16:11:50.983440 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42c7bef1-d2c0-4584-afc7-7f20a00deb36 service neutron] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:11:50.983621 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42c7bef1-d2c0-4584-afc7-7f20a00deb36 service neutron] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:11:50.983881 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42c7bef1-d2c0-4584-afc7-7f20a00deb36 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:11:50.994758 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42c7bef1-d2c0-4584-afc7-7f20a00deb36 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 16:11:50.996888 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42c7bef1-d2c0-4584-afc7-7f20a00deb36 service neutron] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 16:11:51.007730 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 1064/2105] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Aug 22 16:11:50 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 16:11:51.105387 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.notifications [None req-8f62ed4c-b66e-4ec8-be6c-7b1dab78ebfe admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f9b446c6190e410b895b1def7f4b0b28', 'request_id': 'req-8f62ed4c-b66e-4ec8-be6c-7b1dab78ebfe'} {{(pid=81767) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 16:11:51.115386 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.notifications [None req-8f62ed4c-b66e-4ec8-be6c-7b1dab78ebfe admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f9b446c6190e410b895b1def7f4b0b28', 'request_id': 'req-8f62ed4c-b66e-4ec8-be6c-7b1dab78ebfe'} {{(pid=81767) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 16:11:51.208450 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.notifications [None req-8f62ed4c-b66e-4ec8-be6c-7b1dab78ebfe admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f9b446c6190e410b895b1def7f4b0b28', 'request_id': 'req-8f62ed4c-b66e-4ec8-be6c-7b1dab78ebfe'} {{(pid=81767) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 16:11:51.223408 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.notifications [None req-8f62ed4c-b66e-4ec8-be6c-7b1dab78ebfe admin admin] The token cache is being invalidate because project f9b446c6190e410b895b1def7f4b0b28 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81767) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 22 16:11:51.223780 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.notifications [None req-8f62ed4c-b66e-4ec8-be6c-7b1dab78ebfe admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81767) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 16:11:51.224659 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.notifications [None req-8f62ed4c-b66e-4ec8-be6c-7b1dab78ebfe admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81767) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 16:11:51.226386 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 1042/2106] 10.222.0.15 () {58 vars in 1233 bytes} [Mon Aug 22 16:11:50 2022] DELETE /identity/v3/projects/f9b446c6190e410b895b1def7f4b0b28 => generated 0 bytes in 299 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 16:11:51.233829 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e850bfe1-e533-487f-88c4-681166512a9e None None] REQUEST_METHOD: `POST` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:11:51.234329 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e850bfe1-e533-487f-88c4-681166512a9e None None] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:11:51.235119 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e850bfe1-e533-487f-88c4-681166512a9e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:11:51.256373 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.auth.core [None req-e850bfe1-e533-487f-88c4-681166512a9e None None] MFA Rules not processed for user `6b2199ae0fb0480aa917ada391f5ec54`. Rule list: `[]` (Enabled: `True`). {{(pid=81768) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 16:11:51.257671 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-e850bfe1-e533-487f-88c4-681166512a9e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 16:11:51.328463 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 1065/2107] 10.222.0.15 () {58 vars in 958 bytes} [Mon Aug 22 16:11:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 16:11:51.337607 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d70c0a3-95da-4e15-892b-64d22c506919 None None] REQUEST_METHOD: `POST` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:11:51.338249 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d70c0a3-95da-4e15-892b-64d22c506919 None None] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:11:51.338619 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d70c0a3-95da-4e15-892b-64d22c506919 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:11:51.359917 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.auth.core [None req-6d70c0a3-95da-4e15-892b-64d22c506919 None None] MFA Rules not processed for user `6b2199ae0fb0480aa917ada391f5ec54`. Rule list: `[]` (Enabled: `True`). {{(pid=81767) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 16:11:51.361324 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.fernet_utils [None req-6d70c0a3-95da-4e15-892b-64d22c506919 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81767) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 16:11:51.370158 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 1043/2108] 10.222.0.15 () {58 vars in 958 bytes} [Mon Aug 22 16:11:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 16:11:51.374509 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e850bfe1-e533-487f-88c4-681166512a9e None None] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 16:11:51.384528 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e850bfe1-e533-487f-88c4-681166512a9e None None] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 16:11:51.389302 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f17b409c-8939-4857-bbe3-cf4766a9e1fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a98b1e4431940ab8d1fc95df2858a6c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 16:11:51.389906 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f17b409c-8939-4857-bbe3-cf4766a9e1fa admin admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:11:51.390181 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f17b409c-8939-4857-bbe3-cf4766a9e1fa admin admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:11:51.390453 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f17b409c-8939-4857-bbe3-cf4766a9e1fa admin admin] PATH_INFO: `/v3/domains` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:11:51.391100 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f17b409c-8939-4857-bbe3-cf4766a9e1fa admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81768) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 16:11:51.391432 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f17b409c-8939-4857-bbe3-cf4766a9e1fa admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 16:11:51.393065 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f17b409c-8939-4857-bbe3-cf4766a9e1fa admin admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 16:11:51.402008 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 16:11:51.407262 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 1066/2109] 10.222.0.15 () {58 vars in 1153 bytes} [Mon Aug 22 16:11:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 16:11:51.412384 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d70c0a3-95da-4e15-892b-64d22c506919 None None] Authenticating user token {{(pid=81767) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 16:11:51.421584 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d70c0a3-95da-4e15-892b-64d22c506919 None None] Validating token access rules against request {{(pid=81767) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 16:11:51.426370 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4df8308-52a7-4c46-a85a-c87403d29437 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a98b1e4431940ab8d1fc95df2858a6c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81767) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 16:11:51.426958 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4df8308-52a7-4c46-a85a-c87403d29437 admin admin] REQUEST_METHOD: `POST` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:11:51.427230 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4df8308-52a7-4c46-a85a-c87403d29437 admin admin] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:11:51.427530 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4df8308-52a7-4c46-a85a-c87403d29437 admin admin] PATH_INFO: `/v3/projects` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:11:51.428148 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4df8308-52a7-4c46-a85a-c87403d29437 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 16:11:51.430072 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4df8308-52a7-4c46-a85a-c87403d29437 admin admin] RBAC: Authorization granted {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 16:11:51.491171 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f17b409c-8939-4857-bbe3-cf4766a9e1fa admin admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 16:11:51.493182 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-f17b409c-8939-4857-bbe3-cf4766a9e1fa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 16:11:51.511111 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f17b409c-8939-4857-bbe3-cf4766a9e1fa admin admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 16:11:51.518408 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 1044/2110] 10.222.0.15 () {60 vars in 1153 bytes} [Mon Aug 22 16:11:51 2022] POST /identity/v3/projects => generated 373 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 22 16:11:51.518539 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dcfa314-81ef-4e67-bb5f-0da5f7ee4072 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a98b1e4431940ab8d1fc95df2858a6c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 16:11:51.518539 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dcfa314-81ef-4e67-bb5f-0da5f7ee4072 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:11:51.518539 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dcfa314-81ef-4e67-bb5f-0da5f7ee4072 admin admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:11:51.518539 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dcfa314-81ef-4e67-bb5f-0da5f7ee4072 admin admin] PATH_INFO: `/v3/projects/1cb989001cdd4a40b26c33a399448ee7` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:11:51.519535 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dcfa314-81ef-4e67-bb5f-0da5f7ee4072 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1cb989001cdd4a40b26c33a399448ee7)` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 16:11:51.520782 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dcfa314-81ef-4e67-bb5f-0da5f7ee4072 admin admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 16:11:51.522188 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4df8308-52a7-4c46-a85a-c87403d29437 admin admin] Authenticating user token {{(pid=81767) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 16:11:51.592805 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4df8308-52a7-4c46-a85a-c87403d29437 admin admin] Validating token access rules against request {{(pid=81767) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 16:11:51.598256 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bfced5b-dba9-40e7-8992-d1e8e16a19e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a98b1e4431940ab8d1fc95df2858a6c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81767) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 16:11:51.598862 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bfced5b-dba9-40e7-8992-d1e8e16a19e7 admin admin] REQUEST_METHOD: `POST` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:11:51.599142 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bfced5b-dba9-40e7-8992-d1e8e16a19e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:11:51.599447 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bfced5b-dba9-40e7-8992-d1e8e16a19e7 admin admin] PATH_INFO: `/v3/users` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:11:51.600067 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bfced5b-dba9-40e7-8992-d1e8e16a19e7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 16:11:51.601973 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bfced5b-dba9-40e7-8992-d1e8e16a19e7 admin admin] RBAC: Authorization granted {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 16:11:51.645547 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-9dcfa314-81ef-4e67-bb5f-0da5f7ee4072 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1cb989001cdd4a40b26c33a399448ee7', 'request_id': 'req-9dcfa314-81ef-4e67-bb5f-0da5f7ee4072'} {{(pid=81768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 16:11:51.651346 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-9dcfa314-81ef-4e67-bb5f-0da5f7ee4072 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1cb989001cdd4a40b26c33a399448ee7', 'request_id': 'req-9dcfa314-81ef-4e67-bb5f-0da5f7ee4072'} {{(pid=81768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 16:11:51.722401 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 1045/2111] 10.222.0.15 () {60 vars in 1144 bytes} [Mon Aug 22 16:11:51 2022] POST /identity/v3/users => generated 287 bytes in 201 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 22 16:11:51.726712 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bfced5b-dba9-40e7-8992-d1e8e16a19e7 admin admin] Authenticating user token {{(pid=81767) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 16:11:51.736420 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bfced5b-dba9-40e7-8992-d1e8e16a19e7 admin admin] Validating token access rules against request {{(pid=81767) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 16:11:51.767839 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-9dcfa314-81ef-4e67-bb5f-0da5f7ee4072 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1cb989001cdd4a40b26c33a399448ee7', 'request_id': 'req-9dcfa314-81ef-4e67-bb5f-0da5f7ee4072'} {{(pid=81768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 16:11:51.781707 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-9dcfa314-81ef-4e67-bb5f-0da5f7ee4072 admin admin] The token cache is being invalidate because project 1cb989001cdd4a40b26c33a399448ee7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81768) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 22 16:11:51.781975 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-9dcfa314-81ef-4e67-bb5f-0da5f7ee4072 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 16:11:51.782646 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-9dcfa314-81ef-4e67-bb5f-0da5f7ee4072 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 16:11:51.784226 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 1067/2112] 10.222.0.15 () {58 vars in 1233 bytes} [Mon Aug 22 16:11:51 2022] DELETE /identity/v3/projects/1cb989001cdd4a40b26c33a399448ee7 => generated 0 bytes in 294 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 22 16:11:51.788089 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dcfa314-81ef-4e67-bb5f-0da5f7ee4072 admin admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 16:11:51.789577 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-9dcfa314-81ef-4e67-bb5f-0da5f7ee4072 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 16:11:51.796882 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8e0617d-fe81-4775-b247-2d8b1ef1e46a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a98b1e4431940ab8d1fc95df2858a6c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81767) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 16:11:51.797494 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8e0617d-fe81-4775-b247-2d8b1ef1e46a admin admin] REQUEST_METHOD: `GET` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:11:51.797771 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8e0617d-fe81-4775-b247-2d8b1ef1e46a admin admin] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:11:51.798040 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8e0617d-fe81-4775-b247-2d8b1ef1e46a admin admin] PATH_INFO: `/v3/roles` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:11:51.798685 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8e0617d-fe81-4775-b247-2d8b1ef1e46a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 16:11:51.800483 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8e0617d-fe81-4775-b247-2d8b1ef1e46a admin admin] RBAC: Authorization granted {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 16:11:51.804869 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dcfa314-81ef-4e67-bb5f-0da5f7ee4072 admin admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 16:11:51.810214 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f38c7707-dcdc-4873-99ce-29207973190b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a98b1e4431940ab8d1fc95df2858a6c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 16:11:51.810978 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f38c7707-dcdc-4873-99ce-29207973190b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:11:51.811112 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 1046/2113] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Aug 22 16:11:51 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 22 16:11:51.814317 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f38c7707-dcdc-4873-99ce-29207973190b admin admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:11:51.814609 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f38c7707-dcdc-4873-99ce-29207973190b admin admin] PATH_INFO: `/v3/users/5ec894c74be14c469e3458fc0f53603d` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:11:51.815088 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8e0617d-fe81-4775-b247-2d8b1ef1e46a admin admin] Authenticating user token {{(pid=81767) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 16:11:51.816417 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f38c7707-dcdc-4873-99ce-29207973190b admin admin] RBAC: Authorizing `identity:delete_user(user_id=5ec894c74be14c469e3458fc0f53603d)` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 16:11:51.817274 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.fernet_utils [None req-a8e0617d-fe81-4775-b247-2d8b1ef1e46a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81767) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 16:11:51.818348 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f38c7707-dcdc-4873-99ce-29207973190b admin admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 16:11:51.829484 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8e0617d-fe81-4775-b247-2d8b1ef1e46a admin admin] Validating token access rules against request {{(pid=81767) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 16:11:51.835119 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a13b7eb-e7ab-4b18-a435-14224c3dda18 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a98b1e4431940ab8d1fc95df2858a6c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81767) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 16:11:51.835750 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a13b7eb-e7ab-4b18-a435-14224c3dda18 admin admin] REQUEST_METHOD: `GET` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:11:51.836030 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a13b7eb-e7ab-4b18-a435-14224c3dda18 admin admin] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:11:51.836307 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a13b7eb-e7ab-4b18-a435-14224c3dda18 admin admin] PATH_INFO: `/v3/roles` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:11:51.836952 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a13b7eb-e7ab-4b18-a435-14224c3dda18 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 16:11:51.838547 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a13b7eb-e7ab-4b18-a435-14224c3dda18 admin admin] RBAC: Authorization granted {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 16:11:51.851668 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 1047/2114] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Aug 22 16:11:51 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 22 16:11:51.854415 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a13b7eb-e7ab-4b18-a435-14224c3dda18 admin admin] Authenticating user token {{(pid=81767) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 16:11:51.864327 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a13b7eb-e7ab-4b18-a435-14224c3dda18 admin admin] Validating token access rules against request {{(pid=81767) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 16:11:51.869031 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d20ffea9-d28f-4fe4-898f-757516b5291b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a98b1e4431940ab8d1fc95df2858a6c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81767) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 16:11:51.869660 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d20ffea9-d28f-4fe4-898f-757516b5291b admin admin] REQUEST_METHOD: `PUT` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:11:51.870163 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d20ffea9-d28f-4fe4-898f-757516b5291b admin admin] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:11:51.870671 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d20ffea9-d28f-4fe4-898f-757516b5291b admin admin] PATH_INFO: `/v3/projects/581d22bece1544e5addcc7f7205093ec/users/04529e2a334542fcaa4b64a26ae74ed7/roles/e6fc1ee3fb974125ba9b4fe55b12b45e` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:11:51.872707 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG dbcounter [-] [81767] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81767) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 16:11:51.887540 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d20ffea9-d28f-4fe4-898f-757516b5291b admin admin] RBAC: Authorizing `identity:create_grant(project_id=581d22bece1544e5addcc7f7205093ec, user_id=04529e2a334542fcaa4b64a26ae74ed7, role_id=e6fc1ee3fb974125ba9b4fe55b12b45e)` {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 16:11:51.890190 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d20ffea9-d28f-4fe4-898f-757516b5291b admin admin] RBAC: Authorization granted {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 16:11:52.008677 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 1048/2115] 10.222.0.15 () {60 vars in 1483 bytes} [Mon Aug 22 16:11:51 2022] PUT /identity/v3/projects/581d22bece1544e5addcc7f7205093ec/users/04529e2a334542fcaa4b64a26ae74ed7/roles/e6fc1ee3fb974125ba9b4fe55b12b45e => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 22 16:11:52.022219 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=93,keystone:DELETE=6,keystone:INSERT=1 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 16:11:52.028824 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc63080-f1f1-4b7c-b21a-a4bc1c143cfa None None] REQUEST_METHOD: `POST` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:11:52.029331 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc63080-f1f1-4b7c-b21a-a4bc1c143cfa None None] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:11:52.029761 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc63080-f1f1-4b7c-b21a-a4bc1c143cfa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:11:52.035691 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-f38c7707-dcdc-4873-99ce-29207973190b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5ec894c74be14c469e3458fc0f53603d'} {{(pid=81768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 16:11:52.036343 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-f38c7707-dcdc-4873-99ce-29207973190b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5ec894c74be14c469e3458fc0f53603d'} {{(pid=81768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 16:11:52.036977 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-f38c7707-dcdc-4873-99ce-29207973190b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5ec894c74be14c469e3458fc0f53603d'} {{(pid=81768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 16:11:52.043983 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-f38c7707-dcdc-4873-99ce-29207973190b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5ec894c74be14c469e3458fc0f53603d'} {{(pid=81768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 16:11:52.058039 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.auth.core [None req-5dc63080-f1f1-4b7c-b21a-a4bc1c143cfa None None] MFA Rules not processed for user `04529e2a334542fcaa4b64a26ae74ed7`. Rule list: `[]` (Enabled: `True`). {{(pid=81767) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 16:11:52.059296 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.fernet_utils [None req-5dc63080-f1f1-4b7c-b21a-a4bc1c143cfa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81767) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 16:11:52.117516 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-f38c7707-dcdc-4873-99ce-29207973190b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5ec894c74be14c469e3458fc0f53603d'} {{(pid=81768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 16:11:52.132867 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 1049/2116] 10.222.0.15 () {58 vars in 958 bytes} [Mon Aug 22 16:11:52 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 22 16:11:52.140046 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 1068/2117] 10.222.0.15 () {58 vars in 1224 bytes} [Mon Aug 22 16:11:51 2022] DELETE /identity/v3/users/5ec894c74be14c469e3458fc0f53603d => generated 0 bytes in 352 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 22 16:11:52.180562 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dc63080-f1f1-4b7c-b21a-a4bc1c143cfa None None] Authenticating user token {{(pid=81767) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 16:11:52.182986 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.fernet_utils [None req-5dc63080-f1f1-4b7c-b21a-a4bc1c143cfa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81767) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 16:11:52.273932 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dc63080-f1f1-4b7c-b21a-a4bc1c143cfa None None] Validating token access rules against request {{(pid=81767) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 16:11:52.324530 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d00c97db-c78e-43df-b148-12ba09ce8c33 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79706d480f864998b9f8594be407ceab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4ee0413f9ef4d4fb1f95a660babe090', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81767) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 16:11:52.325144 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d00c97db-c78e-43df-b148-12ba09ce8c33 service cinder] REQUEST_METHOD: `GET` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:11:52.325417 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d00c97db-c78e-43df-b148-12ba09ce8c33 service cinder] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:11:52.325684 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d00c97db-c78e-43df-b148-12ba09ce8c33 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:11:52.388697 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d00c97db-c78e-43df-b148-12ba09ce8c33 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 16:11:52.391020 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d00c97db-c78e-43df-b148-12ba09ce8c33 service cinder] RBAC: Authorization granted {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 16:11:52.400619 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 1050/2118] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Aug 22 16:11:52 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 221 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 16:11:52.584088 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f38c7707-dcdc-4873-99ce-29207973190b admin admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 16:11:52.585544 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-f38c7707-dcdc-4873-99ce-29207973190b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 16:11:52.604579 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG dbcounter [-] [81767] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81767) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 16:11:52.659881 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f38c7707-dcdc-4873-99ce-29207973190b admin admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 16:11:52.665230 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c942e8e3-7c63-4c22-b164-62adaf2e8021 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a98b1e4431940ab8d1fc95df2858a6c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 16:11:52.665861 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c942e8e3-7c63-4c22-b164-62adaf2e8021 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:11:52.666141 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c942e8e3-7c63-4c22-b164-62adaf2e8021 admin admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:11:52.666415 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c942e8e3-7c63-4c22-b164-62adaf2e8021 admin admin] PATH_INFO: `/v3/projects/c95c499d811044d6b57903b4f764e5a0` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:11:52.667594 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c942e8e3-7c63-4c22-b164-62adaf2e8021 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c95c499d811044d6b57903b4f764e5a0)` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 16:11:52.670696 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c942e8e3-7c63-4c22-b164-62adaf2e8021 admin admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 16:11:52.814100 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-c942e8e3-7c63-4c22-b164-62adaf2e8021 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c95c499d811044d6b57903b4f764e5a0', 'request_id': 'req-c942e8e3-7c63-4c22-b164-62adaf2e8021'} {{(pid=81768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 16:11:52.820656 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-c942e8e3-7c63-4c22-b164-62adaf2e8021 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c95c499d811044d6b57903b4f764e5a0', 'request_id': 'req-c942e8e3-7c63-4c22-b164-62adaf2e8021'} {{(pid=81768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 16:11:52.825914 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:DELETE=8,keystone:SELECT=89,keystone:INSERT=3 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 16:11:52.921594 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-c942e8e3-7c63-4c22-b164-62adaf2e8021 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c95c499d811044d6b57903b4f764e5a0', 'request_id': 'req-c942e8e3-7c63-4c22-b164-62adaf2e8021'} {{(pid=81768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 16:11:52.933794 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-c942e8e3-7c63-4c22-b164-62adaf2e8021 admin admin] The token cache is being invalidate because project c95c499d811044d6b57903b4f764e5a0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81768) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 22 16:11:52.934084 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-c942e8e3-7c63-4c22-b164-62adaf2e8021 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 16:11:52.934716 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-c942e8e3-7c63-4c22-b164-62adaf2e8021 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 16:11:52.936241 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 1069/2119] 10.222.0.15 () {58 vars in 1233 bytes} [Mon Aug 22 16:11:52 2022] DELETE /identity/v3/projects/c95c499d811044d6b57903b4f764e5a0 => generated 0 bytes in 353 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 16:11:52.941850 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22446681-3be1-4472-94da-695bf669f99e None None] REQUEST_METHOD: `POST` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:11:52.942356 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22446681-3be1-4472-94da-695bf669f99e None None] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:11:52.942356 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22446681-3be1-4472-94da-695bf669f99e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:11:52.962753 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.auth.core [None req-22446681-3be1-4472-94da-695bf669f99e None None] MFA Rules not processed for user `6b2199ae0fb0480aa917ada391f5ec54`. Rule list: `[]` (Enabled: `True`). {{(pid=81767) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 16:11:52.964420 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.fernet_utils [None req-22446681-3be1-4472-94da-695bf669f99e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81767) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 16:11:53.023651 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 1051/2120] 10.222.0.15 () {58 vars in 958 bytes} [Mon Aug 22 16:11:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 16:11:53.031084 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cdc6bd2-fe6b-4358-9372-0047a5926964 None None] REQUEST_METHOD: `POST` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:11:53.031404 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cdc6bd2-fe6b-4358-9372-0047a5926964 None None] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:11:53.031680 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cdc6bd2-fe6b-4358-9372-0047a5926964 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:11:53.052061 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.auth.core [None req-1cdc6bd2-fe6b-4358-9372-0047a5926964 None None] MFA Rules not processed for user `6b2199ae0fb0480aa917ada391f5ec54`. Rule list: `[]` (Enabled: `True`). {{(pid=81768) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 16:11:53.053371 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-1cdc6bd2-fe6b-4358-9372-0047a5926964 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 16:11:53.062182 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 1070/2121] 10.222.0.15 () {58 vars in 958 bytes} [Mon Aug 22 16:11:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 16:11:53.066507 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22446681-3be1-4472-94da-695bf669f99e None None] Authenticating user token {{(pid=81767) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 16:11:53.076925 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22446681-3be1-4472-94da-695bf669f99e None None] Validating token access rules against request {{(pid=81767) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 16:11:53.082405 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b04afc7-d6a5-484b-9b98-b26ce33a7f16 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a98b1e4431940ab8d1fc95df2858a6c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81767) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 16:11:53.084335 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b04afc7-d6a5-484b-9b98-b26ce33a7f16 admin admin] REQUEST_METHOD: `GET` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:11:53.084335 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b04afc7-d6a5-484b-9b98-b26ce33a7f16 admin admin] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:11:53.084335 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b04afc7-d6a5-484b-9b98-b26ce33a7f16 admin admin] PATH_INFO: `/v3/domains` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:11:53.084585 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b04afc7-d6a5-484b-9b98-b26ce33a7f16 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81767) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 16:11:53.084887 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b04afc7-d6a5-484b-9b98-b26ce33a7f16 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 16:11:53.086449 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b04afc7-d6a5-484b-9b98-b26ce33a7f16 admin admin] RBAC: Authorization granted {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 16:11:53.099433 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 1052/2122] 10.222.0.15 () {58 vars in 1153 bytes} [Mon Aug 22 16:11:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 16:11:53.104164 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cdc6bd2-fe6b-4358-9372-0047a5926964 None None] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 16:11:53.118673 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cdc6bd2-fe6b-4358-9372-0047a5926964 None None] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 16:11:53.127059 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfc4d5d6-464b-46b1-b1d6-9dbc7bd9fb54 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a98b1e4431940ab8d1fc95df2858a6c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 16:11:53.128815 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfc4d5d6-464b-46b1-b1d6-9dbc7bd9fb54 admin admin] REQUEST_METHOD: `POST` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:11:53.130175 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfc4d5d6-464b-46b1-b1d6-9dbc7bd9fb54 admin admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:11:53.130834 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfc4d5d6-464b-46b1-b1d6-9dbc7bd9fb54 admin admin] PATH_INFO: `/v3/projects` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:11:53.132333 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfc4d5d6-464b-46b1-b1d6-9dbc7bd9fb54 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 16:11:53.136449 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfc4d5d6-464b-46b1-b1d6-9dbc7bd9fb54 admin admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 16:11:53.293403 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 1071/2123] 10.222.0.15 () {60 vars in 1153 bytes} [Mon Aug 22 16:11:53 2022] POST /identity/v3/projects => generated 387 bytes in 189 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 22 16:11:53.297388 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b04afc7-d6a5-484b-9b98-b26ce33a7f16 admin admin] Authenticating user token {{(pid=81767) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 16:11:53.309329 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b04afc7-d6a5-484b-9b98-b26ce33a7f16 admin admin] Validating token access rules against request {{(pid=81767) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 16:11:53.369679 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cb1b9b1-b29a-4aa4-a53b-c77ea45550f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a98b1e4431940ab8d1fc95df2858a6c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81767) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 16:11:53.370275 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb1b9b1-b29a-4aa4-a53b-c77ea45550f4 admin admin] REQUEST_METHOD: `POST` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:11:53.370582 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb1b9b1-b29a-4aa4-a53b-c77ea45550f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:11:53.370824 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb1b9b1-b29a-4aa4-a53b-c77ea45550f4 admin admin] PATH_INFO: `/v3/users` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:11:53.371486 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cb1b9b1-b29a-4aa4-a53b-c77ea45550f4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 16:11:53.373442 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cb1b9b1-b29a-4aa4-a53b-c77ea45550f4 admin admin] RBAC: Authorization granted {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 16:11:53.478700 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 1053/2124] 10.222.0.15 () {60 vars in 1144 bytes} [Mon Aug 22 16:11:53 2022] POST /identity/v3/users => generated 294 bytes in 182 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 22 16:11:53.482919 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfc4d5d6-464b-46b1-b1d6-9dbc7bd9fb54 admin admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 16:11:53.493930 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfc4d5d6-464b-46b1-b1d6-9dbc7bd9fb54 admin admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 16:11:53.498661 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d98c30e-8cf6-400b-b134-c2e43896b970 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a98b1e4431940ab8d1fc95df2858a6c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 16:11:53.499261 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d98c30e-8cf6-400b-b134-c2e43896b970 admin admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:11:53.499563 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d98c30e-8cf6-400b-b134-c2e43896b970 admin admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:11:53.499877 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d98c30e-8cf6-400b-b134-c2e43896b970 admin admin] PATH_INFO: `/v3/roles` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:11:53.500516 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d98c30e-8cf6-400b-b134-c2e43896b970 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 16:11:53.502422 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d98c30e-8cf6-400b-b134-c2e43896b970 admin admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 16:11:53.513706 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 1072/2125] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Aug 22 16:11:53 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 22 16:11:53.517743 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cb1b9b1-b29a-4aa4-a53b-c77ea45550f4 admin admin] Authenticating user token {{(pid=81767) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 16:11:53.528208 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cb1b9b1-b29a-4aa4-a53b-c77ea45550f4 admin admin] Validating token access rules against request {{(pid=81767) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 16:11:53.533113 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fbefb90-6b8d-4c2a-8aea-5031dc34a8b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a98b1e4431940ab8d1fc95df2858a6c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81767) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 16:11:53.533727 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fbefb90-6b8d-4c2a-8aea-5031dc34a8b2 admin admin] REQUEST_METHOD: `GET` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:11:53.535169 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG dbcounter [-] [81767] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81767) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 16:11:53.535648 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fbefb90-6b8d-4c2a-8aea-5031dc34a8b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:11:53.535984 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fbefb90-6b8d-4c2a-8aea-5031dc34a8b2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:11:53.536695 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fbefb90-6b8d-4c2a-8aea-5031dc34a8b2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 16:11:53.538354 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fbefb90-6b8d-4c2a-8aea-5031dc34a8b2 admin admin] RBAC: Authorization granted {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 16:11:53.551249 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 1054/2126] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Aug 22 16:11:53 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 22 16:11:53.555326 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d98c30e-8cf6-400b-b134-c2e43896b970 admin admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 16:11:53.566388 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d98c30e-8cf6-400b-b134-c2e43896b970 admin admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 16:11:53.570893 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47e4613b-8a76-4a47-a04e-47301331fa36 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a98b1e4431940ab8d1fc95df2858a6c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 16:11:53.571600 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e4613b-8a76-4a47-a04e-47301331fa36 admin admin] REQUEST_METHOD: `PUT` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:11:53.571878 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e4613b-8a76-4a47-a04e-47301331fa36 admin admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:11:53.572151 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e4613b-8a76-4a47-a04e-47301331fa36 admin admin] PATH_INFO: `/v3/projects/b2885b636fe94c119d5d23f7f2ef11ff/users/f08228e7c8ca441e8c75ca450b3fe2c4/roles/e6fc1ee3fb974125ba9b4fe55b12b45e` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:11:53.586999 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47e4613b-8a76-4a47-a04e-47301331fa36 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b2885b636fe94c119d5d23f7f2ef11ff, user_id=f08228e7c8ca441e8c75ca450b3fe2c4, role_id=e6fc1ee3fb974125ba9b4fe55b12b45e)` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 16:11:53.589743 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47e4613b-8a76-4a47-a04e-47301331fa36 admin admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 16:11:53.661347 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 1073/2127] 10.222.0.15 () {60 vars in 1483 bytes} [Mon Aug 22 16:11:53 2022] PUT /identity/v3/projects/b2885b636fe94c119d5d23f7f2ef11ff/users/f08228e7c8ca441e8c75ca450b3fe2c4/roles/e6fc1ee3fb974125ba9b4fe55b12b45e => generated 0 bytes in 106 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 16:11:53.672364 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca0dd10-eb93-45ec-a07a-254eebbb8b5c None None] REQUEST_METHOD: `POST` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:11:53.672718 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca0dd10-eb93-45ec-a07a-254eebbb8b5c None None] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:11:53.674277 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca0dd10-eb93-45ec-a07a-254eebbb8b5c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:11:53.704407 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.auth.core [None req-9ca0dd10-eb93-45ec-a07a-254eebbb8b5c None None] MFA Rules not processed for user `f08228e7c8ca441e8c75ca450b3fe2c4`. Rule list: `[]` (Enabled: `True`). {{(pid=81767) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 16:11:53.706103 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.fernet_utils [None req-9ca0dd10-eb93-45ec-a07a-254eebbb8b5c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81767) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 16:11:53.773145 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 1055/2128] 10.222.0.15 () {58 vars in 958 bytes} [Mon Aug 22 16:11:53 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 16:11:53.777969 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47e4613b-8a76-4a47-a04e-47301331fa36 admin admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 16:11:53.779829 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=85,keystone:INSERT=5,keystone:DELETE=10 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 16:11:53.845327 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47e4613b-8a76-4a47-a04e-47301331fa36 admin admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 16:11:53.850513 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f84dbd0-d7eb-4165-a8fb-efe87d75241b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a98b1e4431940ab8d1fc95df2858a6c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 16:11:53.851111 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f84dbd0-d7eb-4165-a8fb-efe87d75241b admin admin] REQUEST_METHOD: `POST` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:11:53.851483 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f84dbd0-d7eb-4165-a8fb-efe87d75241b admin admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:11:53.851687 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f84dbd0-d7eb-4165-a8fb-efe87d75241b admin admin] PATH_INFO: `/v3/projects` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:11:53.852300 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f84dbd0-d7eb-4165-a8fb-efe87d75241b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 16:11:53.854201 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f84dbd0-d7eb-4165-a8fb-efe87d75241b admin admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 16:11:53.963914 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 1074/2129] 10.222.0.15 () {60 vars in 1153 bytes} [Mon Aug 22 16:11:53 2022] POST /identity/v3/projects => generated 387 bytes in 187 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 22 16:11:53.967884 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ca0dd10-eb93-45ec-a07a-254eebbb8b5c None None] Authenticating user token {{(pid=81767) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 16:11:53.979599 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ca0dd10-eb93-45ec-a07a-254eebbb8b5c None None] Validating token access rules against request {{(pid=81767) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 16:11:54.039895 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fff1303-0601-4864-90a2-2fcc941613cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a98b1e4431940ab8d1fc95df2858a6c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81767) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 16:11:54.040508 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fff1303-0601-4864-90a2-2fcc941613cf admin admin] REQUEST_METHOD: `POST` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:11:54.040784 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fff1303-0601-4864-90a2-2fcc941613cf admin admin] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:11:54.041055 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fff1303-0601-4864-90a2-2fcc941613cf admin admin] PATH_INFO: `/v3/users` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:11:54.041694 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fff1303-0601-4864-90a2-2fcc941613cf admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 16:11:54.043658 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fff1303-0601-4864-90a2-2fcc941613cf admin admin] RBAC: Authorization granted {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 16:11:54.193195 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 1056/2130] 10.222.0.15 () {60 vars in 1144 bytes} [Mon Aug 22 16:11:53 2022] POST /identity/v3/users => generated 300 bytes in 226 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 22 16:11:54.197886 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f84dbd0-d7eb-4165-a8fb-efe87d75241b admin admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 16:11:54.209039 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f84dbd0-d7eb-4165-a8fb-efe87d75241b admin admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 16:11:54.213724 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27f964c0-1096-4b1a-a74f-0fa47a3ef400 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a98b1e4431940ab8d1fc95df2858a6c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 16:11:54.214584 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27f964c0-1096-4b1a-a74f-0fa47a3ef400 admin admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:11:54.214860 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27f964c0-1096-4b1a-a74f-0fa47a3ef400 admin admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:11:54.215194 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27f964c0-1096-4b1a-a74f-0fa47a3ef400 admin admin] PATH_INFO: `/v3/roles` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:11:54.215876 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27f964c0-1096-4b1a-a74f-0fa47a3ef400 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 16:11:54.217463 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27f964c0-1096-4b1a-a74f-0fa47a3ef400 admin admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 16:11:54.235689 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 1075/2131] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Aug 22 16:11:54 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 22 16:11:54.243299 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG dbcounter [-] [81767] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81767) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 16:11:54.246863 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fff1303-0601-4864-90a2-2fcc941613cf admin admin] Authenticating user token {{(pid=81767) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 16:11:54.257478 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fff1303-0601-4864-90a2-2fcc941613cf admin admin] Validating token access rules against request {{(pid=81767) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 16:11:54.263192 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-745d32e0-a81c-4baa-a14f-17cb65ac7909 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a98b1e4431940ab8d1fc95df2858a6c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81767) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 16:11:54.263961 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-745d32e0-a81c-4baa-a14f-17cb65ac7909 admin admin] REQUEST_METHOD: `PUT` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:11:54.264316 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-745d32e0-a81c-4baa-a14f-17cb65ac7909 admin admin] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:11:54.264840 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-745d32e0-a81c-4baa-a14f-17cb65ac7909 admin admin] PATH_INFO: `/v3/projects/3f49c673dfa840518fef7df4911d8e44/users/b7ab0430063b49dbac86812b29d1638c/roles/9d9848cfd22e4b089db443c350e56670` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:11:54.281191 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-745d32e0-a81c-4baa-a14f-17cb65ac7909 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3f49c673dfa840518fef7df4911d8e44, user_id=b7ab0430063b49dbac86812b29d1638c, role_id=9d9848cfd22e4b089db443c350e56670)` {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 16:11:54.283994 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-745d32e0-a81c-4baa-a14f-17cb65ac7909 admin admin] RBAC: Authorization granted {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 16:11:54.387043 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 1057/2132] 10.222.0.15 () {60 vars in 1483 bytes} [Mon Aug 22 16:11:54 2022] PUT /identity/v3/projects/3f49c673dfa840518fef7df4911d8e44/users/b7ab0430063b49dbac86812b29d1638c/roles/9d9848cfd22e4b089db443c350e56670 => generated 0 bytes in 141 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 22 16:11:54.391201 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27f964c0-1096-4b1a-a74f-0fa47a3ef400 admin admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 16:11:54.402091 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27f964c0-1096-4b1a-a74f-0fa47a3ef400 admin admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 16:11:54.457894 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64501154-2ade-439a-bfb2-e045874a072f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a98b1e4431940ab8d1fc95df2858a6c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 16:11:54.458492 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64501154-2ade-439a-bfb2-e045874a072f admin admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:11:54.459030 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64501154-2ade-439a-bfb2-e045874a072f admin admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:11:54.459326 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64501154-2ade-439a-bfb2-e045874a072f admin admin] PATH_INFO: `/v3/roles` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:11:54.459967 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64501154-2ade-439a-bfb2-e045874a072f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 16:11:54.461563 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64501154-2ade-439a-bfb2-e045874a072f admin admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 16:11:54.471253 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 1076/2133] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Aug 22 16:11:54 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 22 16:11:54.475560 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-745d32e0-a81c-4baa-a14f-17cb65ac7909 admin admin] Authenticating user token {{(pid=81767) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 16:11:54.486035 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-745d32e0-a81c-4baa-a14f-17cb65ac7909 admin admin] Validating token access rules against request {{(pid=81767) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 16:11:54.491671 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e09c636-275a-493b-ad32-00295f424558 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a98b1e4431940ab8d1fc95df2858a6c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81767) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 16:11:54.492383 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e09c636-275a-493b-ad32-00295f424558 admin admin] REQUEST_METHOD: `PUT` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:11:54.492661 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e09c636-275a-493b-ad32-00295f424558 admin admin] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:11:54.492927 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e09c636-275a-493b-ad32-00295f424558 admin admin] PATH_INFO: `/v3/projects/3f49c673dfa840518fef7df4911d8e44/users/b7ab0430063b49dbac86812b29d1638c/roles/9d9848cfd22e4b089db443c350e56670` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:11:54.494876 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e09c636-275a-493b-ad32-00295f424558 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3f49c673dfa840518fef7df4911d8e44, user_id=b7ab0430063b49dbac86812b29d1638c, role_id=9d9848cfd22e4b089db443c350e56670)` {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 16:11:54.497508 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e09c636-275a-493b-ad32-00295f424558 admin admin] RBAC: Authorization granted {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 16:11:54.508831 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 1058/2134] 10.222.0.15 () {60 vars in 1483 bytes} [Mon Aug 22 16:11:54 2022] PUT /identity/v3/projects/3f49c673dfa840518fef7df4911d8e44/users/b7ab0430063b49dbac86812b29d1638c/roles/9d9848cfd22e4b089db443c350e56670 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 16:11:54.520239 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eb0e195-26fe-4f91-9c03-8f0ea1fa70ea None None] REQUEST_METHOD: `POST` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:11:54.520529 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eb0e195-26fe-4f91-9c03-8f0ea1fa70ea None None] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:11:54.520804 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eb0e195-26fe-4f91-9c03-8f0ea1fa70ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:11:54.548614 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=100 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 16:11:54.552060 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.auth.core [None req-9eb0e195-26fe-4f91-9c03-8f0ea1fa70ea None None] MFA Rules not processed for user `b7ab0430063b49dbac86812b29d1638c`. Rule list: `[]` (Enabled: `True`). {{(pid=81768) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 16:11:54.553268 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-9eb0e195-26fe-4f91-9c03-8f0ea1fa70ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 16:11:54.578637 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG dbcounter [-] [81767] Writing DB stats keystone:SELECT=93,keystone:INSERT=4,keystone:DELETE=3 {{(pid=81767) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 16:11:54.627964 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 16:11:54.628590 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 1077/2135] 10.222.0.15 () {58 vars in 958 bytes} [Mon Aug 22 16:11:54 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 16:11:54.639308 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e09c636-275a-493b-ad32-00295f424558 admin admin] Authenticating user token {{(pid=81767) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 16:11:54.641000 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.fernet_utils [None req-6e09c636-275a-493b-ad32-00295f424558 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81767) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 16:11:54.701149 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e09c636-275a-493b-ad32-00295f424558 admin admin] Validating token access rules against request {{(pid=81767) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 16:11:54.705140 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9107ac5d-f25d-4177-b8ca-b5dab2d7e416 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79706d480f864998b9f8594be407ceab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4ee0413f9ef4d4fb1f95a660babe090', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81767) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 16:11:54.705796 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9107ac5d-f25d-4177-b8ca-b5dab2d7e416 service cinder] REQUEST_METHOD: `GET` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:11:54.706652 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9107ac5d-f25d-4177-b8ca-b5dab2d7e416 service cinder] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:11:54.706935 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9107ac5d-f25d-4177-b8ca-b5dab2d7e416 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:11:54.716688 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9107ac5d-f25d-4177-b8ca-b5dab2d7e416 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 16:11:54.718117 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9107ac5d-f25d-4177-b8ca-b5dab2d7e416 service cinder] RBAC: Authorization granted {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 16:11:54.731385 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 1059/2136] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Aug 22 16:11:54 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 16:11:54.760292 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eb0e195-26fe-4f91-9c03-8f0ea1fa70ea None None] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 16:11:54.827000 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eb0e195-26fe-4f91-9c03-8f0ea1fa70ea None None] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 16:11:54.832406 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a3fc251-a98f-4c6d-8e25-eefcae064635 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a98b1e4431940ab8d1fc95df2858a6c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 16:11:54.833035 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a3fc251-a98f-4c6d-8e25-eefcae064635 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:11:54.833357 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a3fc251-a98f-4c6d-8e25-eefcae064635 admin admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:11:54.833630 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a3fc251-a98f-4c6d-8e25-eefcae064635 admin admin] PATH_INFO: `/v3/users/f08228e7c8ca441e8c75ca450b3fe2c4` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:11:54.835018 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a3fc251-a98f-4c6d-8e25-eefcae064635 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f08228e7c8ca441e8c75ca450b3fe2c4)` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 16:11:54.837080 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a3fc251-a98f-4c6d-8e25-eefcae064635 admin admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 16:11:54.947228 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG dbcounter [-] [81767] Writing DB stats keystone:SELECT=85,keystone:DELETE=13,keystone:INSERT=2 {{(pid=81767) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 16:11:55.029220 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 16:11:55.043613 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-7a3fc251-a98f-4c6d-8e25-eefcae064635 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f08228e7c8ca441e8c75ca450b3fe2c4'} {{(pid=81768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 16:11:55.044263 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-7a3fc251-a98f-4c6d-8e25-eefcae064635 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f08228e7c8ca441e8c75ca450b3fe2c4'} {{(pid=81768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 16:11:55.044840 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-7a3fc251-a98f-4c6d-8e25-eefcae064635 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f08228e7c8ca441e8c75ca450b3fe2c4'} {{(pid=81768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 16:11:55.051851 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-7a3fc251-a98f-4c6d-8e25-eefcae064635 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f08228e7c8ca441e8c75ca450b3fe2c4'} {{(pid=81768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 16:11:55.173583 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-7a3fc251-a98f-4c6d-8e25-eefcae064635 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f08228e7c8ca441e8c75ca450b3fe2c4'} {{(pid=81768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 16:11:55.191495 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 1078/2137] 10.222.0.15 () {58 vars in 1224 bytes} [Mon Aug 22 16:11:54 2022] DELETE /identity/v3/users/f08228e7c8ca441e8c75ca450b3fe2c4 => generated 0 bytes in 432 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 16:11:55.197153 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b922ff-ce5c-4ce1-a4bf-8ae8ebcb2531 None None] REQUEST_METHOD: `POST` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:11:55.197153 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b922ff-ce5c-4ce1-a4bf-8ae8ebcb2531 None None] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:11:55.197153 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b922ff-ce5c-4ce1-a4bf-8ae8ebcb2531 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:11:55.216055 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.auth.core [None req-23b922ff-ce5c-4ce1-a4bf-8ae8ebcb2531 None None] MFA Rules not processed for user `6b2199ae0fb0480aa917ada391f5ec54`. Rule list: `[]` (Enabled: `True`). {{(pid=81767) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 16:11:55.217270 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.fernet_utils [None req-23b922ff-ce5c-4ce1-a4bf-8ae8ebcb2531 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81767) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 16:11:55.310098 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 1060/2138] 10.222.0.15 () {58 vars in 958 bytes} [Mon Aug 22 16:11:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 16:11:55.319444 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a3fc251-a98f-4c6d-8e25-eefcae064635 admin admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 16:11:55.320859 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-7a3fc251-a98f-4c6d-8e25-eefcae064635 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 16:11:55.391964 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba2cf4b4-5b18-413d-909a-59278dbf0ab3 None None] REQUEST_METHOD: `POST` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:11:55.392471 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba2cf4b4-5b18-413d-909a-59278dbf0ab3 None None] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:11:55.393040 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba2cf4b4-5b18-413d-909a-59278dbf0ab3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:11:55.431775 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.auth.core [None req-ba2cf4b4-5b18-413d-909a-59278dbf0ab3 None None] MFA Rules not processed for user `04529e2a334542fcaa4b64a26ae74ed7`. Rule list: `[]` (Enabled: `True`). {{(pid=81767) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 16:11:55.433282 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.fernet_utils [None req-ba2cf4b4-5b18-413d-909a-59278dbf0ab3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81767) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 16:11:55.437281 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a3fc251-a98f-4c6d-8e25-eefcae064635 admin admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 16:11:55.441827 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9250a2d1-382d-4141-9089-c1b3a59f575a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '234687adad594f1aa45775a565ffbdc7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4ee0413f9ef4d4fb1f95a660babe090', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 16:11:55.442604 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9250a2d1-382d-4141-9089-c1b3a59f575a service neutron] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:11:55.442886 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9250a2d1-382d-4141-9089-c1b3a59f575a service neutron] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:11:55.443157 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9250a2d1-382d-4141-9089-c1b3a59f575a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:11:55.453695 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9250a2d1-382d-4141-9089-c1b3a59f575a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 16:11:55.455152 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9250a2d1-382d-4141-9089-c1b3a59f575a service neutron] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 16:11:55.466980 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 1079/2139] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Aug 22 16:11:55 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 148 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 22 16:11:55.507872 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 1061/2140] 10.222.0.15 () {58 vars in 958 bytes} [Mon Aug 22 16:11:55 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 22 16:11:55.562415 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 16:11:55.637098 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG dbcounter [-] [81767] Writing DB stats keystone:SELECT=90,keystone:DELETE=8,keystone:INSERT=2 {{(pid=81767) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 16:11:56.085903 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9250a2d1-382d-4141-9089-c1b3a59f575a service neutron] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 16:11:56.087668 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-9250a2d1-382d-4141-9089-c1b3a59f575a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 16:11:56.100511 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9250a2d1-382d-4141-9089-c1b3a59f575a service neutron] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 16:11:56.105208 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6078dbf-e244-4e8d-b097-23a30997782e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a98b1e4431940ab8d1fc95df2858a6c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 16:11:56.106193 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6078dbf-e244-4e8d-b097-23a30997782e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:11:56.106686 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6078dbf-e244-4e8d-b097-23a30997782e admin admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:11:56.107019 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6078dbf-e244-4e8d-b097-23a30997782e admin admin] PATH_INFO: `/v3/projects/b2885b636fe94c119d5d23f7f2ef11ff` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:11:56.108275 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6078dbf-e244-4e8d-b097-23a30997782e admin admin] RBAC: Authorizing `identity:delete_project(project_id=b2885b636fe94c119d5d23f7f2ef11ff)` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 16:11:56.110270 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6078dbf-e244-4e8d-b097-23a30997782e admin admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 16:11:56.257171 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-c6078dbf-e244-4e8d-b097-23a30997782e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b2885b636fe94c119d5d23f7f2ef11ff', 'request_id': 'req-c6078dbf-e244-4e8d-b097-23a30997782e'} {{(pid=81768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 16:11:56.268993 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-c6078dbf-e244-4e8d-b097-23a30997782e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b2885b636fe94c119d5d23f7f2ef11ff', 'request_id': 'req-c6078dbf-e244-4e8d-b097-23a30997782e'} {{(pid=81768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 16:11:56.303979 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba2cf4b4-5b18-413d-909a-59278dbf0ab3 None None] Authenticating user token {{(pid=81767) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 16:11:56.305934 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.fernet_utils [None req-ba2cf4b4-5b18-413d-909a-59278dbf0ab3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81767) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 16:11:56.356793 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-c6078dbf-e244-4e8d-b097-23a30997782e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b2885b636fe94c119d5d23f7f2ef11ff', 'request_id': 'req-c6078dbf-e244-4e8d-b097-23a30997782e'} {{(pid=81768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 16:11:56.374318 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-c6078dbf-e244-4e8d-b097-23a30997782e admin admin] The token cache is being invalidate because project b2885b636fe94c119d5d23f7f2ef11ff was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81768) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 22 16:11:56.374864 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-c6078dbf-e244-4e8d-b097-23a30997782e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 16:11:56.375573 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-c6078dbf-e244-4e8d-b097-23a30997782e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 16:11:56.377643 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 1080/2141] 10.222.0.15 () {58 vars in 1233 bytes} [Mon Aug 22 16:11:56 2022] DELETE /identity/v3/projects/b2885b636fe94c119d5d23f7f2ef11ff => generated 0 bytes in 292 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 16:11:56.381634 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6078dbf-e244-4e8d-b097-23a30997782e admin admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 16:11:56.383015 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-c6078dbf-e244-4e8d-b097-23a30997782e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 16:11:56.416489 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba2cf4b4-5b18-413d-909a-59278dbf0ab3 None None] Validating token access rules against request {{(pid=81767) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 16:11:56.473144 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6078dbf-e244-4e8d-b097-23a30997782e admin admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 16:11:56.478603 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06df35a6-0a23-4209-8c73-839a281df23a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a98b1e4431940ab8d1fc95df2858a6c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 16:11:56.479238 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06df35a6-0a23-4209-8c73-839a281df23a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:11:56.479541 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06df35a6-0a23-4209-8c73-839a281df23a admin admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:11:56.479810 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06df35a6-0a23-4209-8c73-839a281df23a admin admin] PATH_INFO: `/v3/users/b7ab0430063b49dbac86812b29d1638c` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:11:56.481080 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06df35a6-0a23-4209-8c73-839a281df23a admin admin] RBAC: Authorizing `identity:delete_user(user_id=b7ab0430063b49dbac86812b29d1638c)` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 16:11:56.483002 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06df35a6-0a23-4209-8c73-839a281df23a admin admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 16:11:56.504934 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd801c1d-94b3-4ab0-983f-f27a2c71fc73 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '08159bf8edda4246bb463f7b11b0894b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4ee0413f9ef4d4fb1f95a660babe090', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81767) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 16:11:56.505563 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd801c1d-94b3-4ab0-983f-f27a2c71fc73 service placement] REQUEST_METHOD: `GET` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:11:56.505840 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd801c1d-94b3-4ab0-983f-f27a2c71fc73 service placement] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:11:56.506109 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd801c1d-94b3-4ab0-983f-f27a2c71fc73 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:11:56.508362 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.fernet_utils [None req-bd801c1d-94b3-4ab0-983f-f27a2c71fc73 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81767) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 16:11:56.529306 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd801c1d-94b3-4ab0-983f-f27a2c71fc73 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 16:11:56.529306 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd801c1d-94b3-4ab0-983f-f27a2c71fc73 service placement] RBAC: Authorization granted {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 16:11:56.537802 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 1062/2142] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Aug 22 16:11:56 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 234 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 22 16:11:56.562322 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd801c1d-94b3-4ab0-983f-f27a2c71fc73 service placement] Authenticating user token {{(pid=81767) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 16:11:56.564088 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.fernet_utils [None req-bd801c1d-94b3-4ab0-983f-f27a2c71fc73 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81767) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 16:11:56.578624 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd801c1d-94b3-4ab0-983f-f27a2c71fc73 service placement] Validating token access rules against request {{(pid=81767) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 16:11:56.585576 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-844dd41c-52b1-4450-ab6f-aeb83335e849 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a98b1e4431940ab8d1fc95df2858a6c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81767) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 16:11:56.586212 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-844dd41c-52b1-4450-ab6f-aeb83335e849 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:11:56.586554 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-844dd41c-52b1-4450-ab6f-aeb83335e849 admin admin] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:11:56.586767 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-844dd41c-52b1-4450-ab6f-aeb83335e849 admin admin] PATH_INFO: `/v3/users/6d4136763ffe47249d865f3ff0e3adb1` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:11:56.588794 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-844dd41c-52b1-4450-ab6f-aeb83335e849 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6d4136763ffe47249d865f3ff0e3adb1)` {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 16:11:56.591908 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-844dd41c-52b1-4450-ab6f-aeb83335e849 admin admin] RBAC: Authorization granted {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 16:11:56.693319 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-06df35a6-0a23-4209-8c73-839a281df23a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b7ab0430063b49dbac86812b29d1638c'} {{(pid=81768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 16:11:56.693970 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-06df35a6-0a23-4209-8c73-839a281df23a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b7ab0430063b49dbac86812b29d1638c'} {{(pid=81768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 16:11:56.694597 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-06df35a6-0a23-4209-8c73-839a281df23a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b7ab0430063b49dbac86812b29d1638c'} {{(pid=81768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 16:11:56.702546 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-06df35a6-0a23-4209-8c73-839a281df23a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b7ab0430063b49dbac86812b29d1638c'} {{(pid=81768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 16:11:56.781707 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-06df35a6-0a23-4209-8c73-839a281df23a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b7ab0430063b49dbac86812b29d1638c'} {{(pid=81768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 16:11:56.802846 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 1081/2143] 10.222.0.15 () {58 vars in 1224 bytes} [Mon Aug 22 16:11:56 2022] DELETE /identity/v3/users/b7ab0430063b49dbac86812b29d1638c => generated 0 bytes in 422 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 22 16:11:56.837424 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.notifications [None req-844dd41c-52b1-4450-ab6f-aeb83335e849 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6d4136763ffe47249d865f3ff0e3adb1'} {{(pid=81767) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 16:11:56.838272 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.notifications [None req-844dd41c-52b1-4450-ab6f-aeb83335e849 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6d4136763ffe47249d865f3ff0e3adb1'} {{(pid=81767) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 16:11:56.947011 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.notifications [None req-844dd41c-52b1-4450-ab6f-aeb83335e849 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6d4136763ffe47249d865f3ff0e3adb1'} {{(pid=81767) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 16:11:56.955346 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.notifications [None req-844dd41c-52b1-4450-ab6f-aeb83335e849 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6d4136763ffe47249d865f3ff0e3adb1'} {{(pid=81767) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 16:11:56.986498 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.notifications [None req-844dd41c-52b1-4450-ab6f-aeb83335e849 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6d4136763ffe47249d865f3ff0e3adb1'} {{(pid=81767) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 16:11:56.996922 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 1063/2144] 10.222.0.15 () {58 vars in 1224 bytes} [Mon Aug 22 16:11:56 2022] DELETE /identity/v3/users/6d4136763ffe47249d865f3ff0e3adb1 => generated 0 bytes in 435 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 16:11:57.001182 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e645a2d-0457-41ca-8450-17ebaea75ce3 None None] REQUEST_METHOD: `POST` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:11:57.001620 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e645a2d-0457-41ca-8450-17ebaea75ce3 None None] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:11:57.001751 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e645a2d-0457-41ca-8450-17ebaea75ce3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:11:57.032133 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.auth.core [None req-4e645a2d-0457-41ca-8450-17ebaea75ce3 None None] MFA Rules not processed for user `6b2199ae0fb0480aa917ada391f5ec54`. Rule list: `[]` (Enabled: `True`). {{(pid=81768) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 16:11:57.034023 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-4e645a2d-0457-41ca-8450-17ebaea75ce3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 16:11:57.097038 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-844dd41c-52b1-4450-ab6f-aeb83335e849 admin admin] Authenticating user token {{(pid=81767) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 16:11:57.098876 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.fernet_utils [None req-844dd41c-52b1-4450-ab6f-aeb83335e849 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81767) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 16:11:57.133183 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 1082/2145] 10.222.0.15 () {58 vars in 958 bytes} [Mon Aug 22 16:11:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 16:11:57.144152 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e645a2d-0457-41ca-8450-17ebaea75ce3 None None] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 16:11:57.145686 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-4e645a2d-0457-41ca-8450-17ebaea75ce3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 16:11:57.198442 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-844dd41c-52b1-4450-ab6f-aeb83335e849 admin admin] Validating token access rules against request {{(pid=81767) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 16:11:57.205730 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf77ad62-9952-4a0d-9e50-aafd3e0de4b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a98b1e4431940ab8d1fc95df2858a6c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81767) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 16:11:57.206452 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf77ad62-9952-4a0d-9e50-aafd3e0de4b9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:11:57.206732 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf77ad62-9952-4a0d-9e50-aafd3e0de4b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:11:57.207005 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf77ad62-9952-4a0d-9e50-aafd3e0de4b9 admin admin] PATH_INFO: `/v3/users/394b669ff68949e8909ba5e7248c8984` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:11:57.212103 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf77ad62-9952-4a0d-9e50-aafd3e0de4b9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=394b669ff68949e8909ba5e7248c8984)` {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 16:11:57.214629 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf77ad62-9952-4a0d-9e50-aafd3e0de4b9 admin admin] RBAC: Authorization granted {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 16:11:57.218907 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 16:11:57.236176 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e645a2d-0457-41ca-8450-17ebaea75ce3 None None] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 16:11:57.240917 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04cba83e-3809-48bd-bc0c-6f3f794d64bc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '234687adad594f1aa45775a565ffbdc7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4ee0413f9ef4d4fb1f95a660babe090', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 16:11:57.243317 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04cba83e-3809-48bd-bc0c-6f3f794d64bc service neutron] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:11:57.243783 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04cba83e-3809-48bd-bc0c-6f3f794d64bc service neutron] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:11:57.244204 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04cba83e-3809-48bd-bc0c-6f3f794d64bc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:11:57.263229 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04cba83e-3809-48bd-bc0c-6f3f794d64bc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 16:11:57.264859 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04cba83e-3809-48bd-bc0c-6f3f794d64bc service neutron] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 16:11:57.280901 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 1083/2146] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Aug 22 16:11:57 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 137 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 22 16:11:57.307433 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04cba83e-3809-48bd-bc0c-6f3f794d64bc service neutron] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 16:11:57.311461 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-04cba83e-3809-48bd-bc0c-6f3f794d64bc service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 16:11:57.345814 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04cba83e-3809-48bd-bc0c-6f3f794d64bc service neutron] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 16:11:57.352683 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d4ee519-6082-44ea-8d83-243bf20561c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a98b1e4431940ab8d1fc95df2858a6c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 16:11:57.353381 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d4ee519-6082-44ea-8d83-243bf20561c7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:11:57.353604 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d4ee519-6082-44ea-8d83-243bf20561c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:11:57.353871 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d4ee519-6082-44ea-8d83-243bf20561c7 admin admin] PATH_INFO: `/v3/projects/3f49c673dfa840518fef7df4911d8e44` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:11:57.361301 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d4ee519-6082-44ea-8d83-243bf20561c7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3f49c673dfa840518fef7df4911d8e44)` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 16:11:57.363907 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d4ee519-6082-44ea-8d83-243bf20561c7 admin admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 16:11:57.420207 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG dbcounter [-] [81767] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81767) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 16:11:57.521442 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-9d4ee519-6082-44ea-8d83-243bf20561c7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3f49c673dfa840518fef7df4911d8e44', 'request_id': 'req-9d4ee519-6082-44ea-8d83-243bf20561c7'} {{(pid=81768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 16:11:57.528947 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.notifications [None req-cf77ad62-9952-4a0d-9e50-aafd3e0de4b9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '394b669ff68949e8909ba5e7248c8984'} {{(pid=81767) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 16:11:57.530021 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.notifications [None req-cf77ad62-9952-4a0d-9e50-aafd3e0de4b9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '394b669ff68949e8909ba5e7248c8984'} {{(pid=81767) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 16:11:57.532905 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-9d4ee519-6082-44ea-8d83-243bf20561c7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3f49c673dfa840518fef7df4911d8e44', 'request_id': 'req-9d4ee519-6082-44ea-8d83-243bf20561c7'} {{(pid=81768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 16:11:57.644366 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.notifications [None req-cf77ad62-9952-4a0d-9e50-aafd3e0de4b9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '394b669ff68949e8909ba5e7248c8984'} {{(pid=81767) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 16:11:57.652609 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.notifications [None req-cf77ad62-9952-4a0d-9e50-aafd3e0de4b9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '394b669ff68949e8909ba5e7248c8984'} {{(pid=81767) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 16:11:57.665470 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.notifications [None req-cf77ad62-9952-4a0d-9e50-aafd3e0de4b9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '394b669ff68949e8909ba5e7248c8984'} {{(pid=81767) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 16:11:57.672711 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 1064/2147] 10.222.0.15 () {58 vars in 1224 bytes} [Mon Aug 22 16:11:57 2022] DELETE /identity/v3/users/394b669ff68949e8909ba5e7248c8984 => generated 0 bytes in 576 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 16:11:57.679029 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8881f2f-5b08-4a8a-88f3-a6ec4c4af1df None None] REQUEST_METHOD: `POST` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:11:57.679334 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8881f2f-5b08-4a8a-88f3-a6ec4c4af1df None None] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:11:57.679872 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8881f2f-5b08-4a8a-88f3-a6ec4c4af1df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:11:57.699636 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.auth.core [None req-e8881f2f-5b08-4a8a-88f3-a6ec4c4af1df None None] MFA Rules not processed for user `6b2199ae0fb0480aa917ada391f5ec54`. Rule list: `[]` (Enabled: `True`). {{(pid=81767) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 16:11:57.700848 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.fernet_utils [None req-e8881f2f-5b08-4a8a-88f3-a6ec4c4af1df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81767) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 16:11:57.770186 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-9d4ee519-6082-44ea-8d83-243bf20561c7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3f49c673dfa840518fef7df4911d8e44', 'request_id': 'req-9d4ee519-6082-44ea-8d83-243bf20561c7'} {{(pid=81768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 16:11:57.788574 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-9d4ee519-6082-44ea-8d83-243bf20561c7 admin admin] The token cache is being invalidate because project 3f49c673dfa840518fef7df4911d8e44 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81768) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 22 16:11:57.789014 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-9d4ee519-6082-44ea-8d83-243bf20561c7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 16:11:57.789813 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-9d4ee519-6082-44ea-8d83-243bf20561c7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 16:11:57.792112 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 1084/2148] 10.222.0.15 () {58 vars in 1233 bytes} [Mon Aug 22 16:11:57 2022] DELETE /identity/v3/projects/3f49c673dfa840518fef7df4911d8e44 => generated 0 bytes in 487 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 16:11:57.793227 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d4ee519-6082-44ea-8d83-243bf20561c7 admin admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 16:11:57.794715 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-9d4ee519-6082-44ea-8d83-243bf20561c7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 16:11:57.794822 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 1065/2149] 10.222.0.15 () {58 vars in 958 bytes} [Mon Aug 22 16:11:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 16:11:57.799023 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d33def21-6209-4502-87b6-4e927bb39562 None None] REQUEST_METHOD: `POST` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:11:57.802788 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d33def21-6209-4502-87b6-4e927bb39562 None None] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:11:57.802788 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d33def21-6209-4502-87b6-4e927bb39562 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:11:57.852405 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.auth.core [None req-d33def21-6209-4502-87b6-4e927bb39562 None None] MFA Rules not processed for user `6b2199ae0fb0480aa917ada391f5ec54`. Rule list: `[]` (Enabled: `True`). {{(pid=81767) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 16:11:57.854520 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.fernet_utils [None req-d33def21-6209-4502-87b6-4e927bb39562 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81767) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 16:11:57.865166 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 1066/2150] 10.222.0.15 () {58 vars in 958 bytes} [Mon Aug 22 16:11:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 67 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 16:11:57.866202 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d33def21-6209-4502-87b6-4e927bb39562 None None] Authenticating user token {{(pid=81767) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 16:11:57.867711 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.fernet_utils [None req-d33def21-6209-4502-87b6-4e927bb39562 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81767) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 16:11:57.904243 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d4ee519-6082-44ea-8d83-243bf20561c7 admin admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 16:11:57.915215 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0614356b-63e4-485b-91ae-0948e427a6ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a98b1e4431940ab8d1fc95df2858a6c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 16:11:57.915987 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0614356b-63e4-485b-91ae-0948e427a6ce admin admin] REQUEST_METHOD: `DELETE` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:11:57.916267 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0614356b-63e4-485b-91ae-0948e427a6ce admin admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:11:57.916614 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0614356b-63e4-485b-91ae-0948e427a6ce admin admin] PATH_INFO: `/v3/projects/49ed116bd33d456ba7f6256f34462ffc` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:11:57.917904 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0614356b-63e4-485b-91ae-0948e427a6ce admin admin] RBAC: Authorizing `identity:delete_project(project_id=49ed116bd33d456ba7f6256f34462ffc)` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 16:11:57.922417 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0614356b-63e4-485b-91ae-0948e427a6ce admin admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 16:11:57.936472 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d33def21-6209-4502-87b6-4e927bb39562 None None] Validating token access rules against request {{(pid=81767) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 16:11:57.956974 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c9993ff-4929-4e64-badc-2ad3c67a691d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '234687adad594f1aa45775a565ffbdc7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4ee0413f9ef4d4fb1f95a660babe090', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81767) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 16:11:57.957635 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c9993ff-4929-4e64-badc-2ad3c67a691d service neutron] REQUEST_METHOD: `GET` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:11:57.957914 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c9993ff-4929-4e64-badc-2ad3c67a691d service neutron] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:11:57.958182 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c9993ff-4929-4e64-badc-2ad3c67a691d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:11:57.960786 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.fernet_utils [None req-9c9993ff-4929-4e64-badc-2ad3c67a691d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81767) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 16:11:57.973914 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c9993ff-4929-4e64-badc-2ad3c67a691d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 16:11:57.979428 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c9993ff-4929-4e64-badc-2ad3c67a691d service neutron] RBAC: Authorization granted {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 16:11:57.986227 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 1067/2151] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Aug 22 16:11:57 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 120 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 16:11:57.988054 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf9c9df2-6b02-4ee2-9f40-a8687d8d750c None None] REQUEST_METHOD: `POST` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:11:57.988385 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf9c9df2-6b02-4ee2-9f40-a8687d8d750c None None] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:11:57.988674 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf9c9df2-6b02-4ee2-9f40-a8687d8d750c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:11:58.009640 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.auth.core [None req-cf9c9df2-6b02-4ee2-9f40-a8687d8d750c None None] MFA Rules not processed for user `6b2199ae0fb0480aa917ada391f5ec54`. Rule list: `[]` (Enabled: `True`). {{(pid=81767) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 16:11:58.010894 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.fernet_utils [None req-cf9c9df2-6b02-4ee2-9f40-a8687d8d750c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81767) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 16:11:58.019263 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 1068/2152] 10.222.0.15 () {58 vars in 958 bytes} [Mon Aug 22 16:11:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 16:11:58.023495 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf9c9df2-6b02-4ee2-9f40-a8687d8d750c None None] Authenticating user token {{(pid=81767) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 16:11:58.037712 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf9c9df2-6b02-4ee2-9f40-a8687d8d750c None None] Validating token access rules against request {{(pid=81767) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 16:11:58.054592 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-0614356b-63e4-485b-91ae-0948e427a6ce admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '49ed116bd33d456ba7f6256f34462ffc', 'request_id': 'req-0614356b-63e4-485b-91ae-0948e427a6ce'} {{(pid=81768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 16:11:58.068029 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-0614356b-63e4-485b-91ae-0948e427a6ce admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '49ed116bd33d456ba7f6256f34462ffc', 'request_id': 'req-0614356b-63e4-485b-91ae-0948e427a6ce'} {{(pid=81768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 16:11:58.129901 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af051df6-cff2-42c0-a99c-b522b30f5742 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a98b1e4431940ab8d1fc95df2858a6c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81767) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 16:11:58.129901 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af051df6-cff2-42c0-a99c-b522b30f5742 admin admin] REQUEST_METHOD: `GET` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:11:58.129901 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af051df6-cff2-42c0-a99c-b522b30f5742 admin admin] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:11:58.129901 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af051df6-cff2-42c0-a99c-b522b30f5742 admin admin] PATH_INFO: `/v3/domains` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:11:58.130249 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af051df6-cff2-42c0-a99c-b522b30f5742 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81767) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 16:11:58.130249 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af051df6-cff2-42c0-a99c-b522b30f5742 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 16:11:58.130249 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af051df6-cff2-42c0-a99c-b522b30f5742 admin admin] RBAC: Authorization granted {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 16:11:58.145969 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-0614356b-63e4-485b-91ae-0948e427a6ce admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '49ed116bd33d456ba7f6256f34462ffc', 'request_id': 'req-0614356b-63e4-485b-91ae-0948e427a6ce'} {{(pid=81768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 16:11:58.164140 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 1069/2153] 10.222.0.15 () {58 vars in 1153 bytes} [Mon Aug 22 16:11:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 16:11:58.168623 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af051df6-cff2-42c0-a99c-b522b30f5742 admin admin] Authenticating user token {{(pid=81767) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 16:11:58.171489 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-0614356b-63e4-485b-91ae-0948e427a6ce admin admin] The token cache is being invalidate because project 49ed116bd33d456ba7f6256f34462ffc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81768) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 22 16:11:58.171489 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-0614356b-63e4-485b-91ae-0948e427a6ce admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 16:11:58.172055 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-0614356b-63e4-485b-91ae-0948e427a6ce admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 16:11:58.173748 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 1085/2154] 10.222.0.15 () {58 vars in 1233 bytes} [Mon Aug 22 16:11:57 2022] DELETE /identity/v3/projects/49ed116bd33d456ba7f6256f34462ffc => generated 0 bytes in 381 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 16:11:58.177456 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0614356b-63e4-485b-91ae-0948e427a6ce admin admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 16:11:58.178787 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-0614356b-63e4-485b-91ae-0948e427a6ce admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 16:11:58.180914 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af051df6-cff2-42c0-a99c-b522b30f5742 admin admin] Validating token access rules against request {{(pid=81767) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 16:11:58.186980 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c869a2d-4373-4768-8154-65fb038c5cfb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a98b1e4431940ab8d1fc95df2858a6c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81767) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 16:11:58.187610 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c869a2d-4373-4768-8154-65fb038c5cfb admin admin] REQUEST_METHOD: `POST` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:11:58.187881 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c869a2d-4373-4768-8154-65fb038c5cfb admin admin] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:11:58.188149 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c869a2d-4373-4768-8154-65fb038c5cfb admin admin] PATH_INFO: `/v3/projects` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:11:58.188765 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c869a2d-4373-4768-8154-65fb038c5cfb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 16:11:58.190686 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c869a2d-4373-4768-8154-65fb038c5cfb admin admin] RBAC: Authorization granted {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 16:11:58.192803 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0614356b-63e4-485b-91ae-0948e427a6ce admin admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 16:11:58.198040 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51bfe6e9-11a6-41de-8164-2eb1c45af484 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a98b1e4431940ab8d1fc95df2858a6c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 16:11:58.198664 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51bfe6e9-11a6-41de-8164-2eb1c45af484 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:11:58.198995 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51bfe6e9-11a6-41de-8164-2eb1c45af484 admin admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:11:58.199295 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51bfe6e9-11a6-41de-8164-2eb1c45af484 admin admin] PATH_INFO: `/v3/users/a88aa134951a4a139562609c27e14bc0` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:11:58.200448 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51bfe6e9-11a6-41de-8164-2eb1c45af484 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a88aa134951a4a139562609c27e14bc0)` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 16:11:58.202632 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51bfe6e9-11a6-41de-8164-2eb1c45af484 admin admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 16:11:58.336402 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 1070/2155] 10.222.0.15 () {60 vars in 1153 bytes} [Mon Aug 22 16:11:58 2022] POST /identity/v3/projects => generated 383 bytes in 168 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 22 16:11:58.342737 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c869a2d-4373-4768-8154-65fb038c5cfb admin admin] Authenticating user token {{(pid=81767) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 16:11:58.348425 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.fernet_utils [None req-0c869a2d-4373-4768-8154-65fb038c5cfb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81767) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 16:11:58.429344 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c869a2d-4373-4768-8154-65fb038c5cfb admin admin] Validating token access rules against request {{(pid=81767) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 16:11:58.434277 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09d716a9-fa13-421e-a255-05d1920e09e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a98b1e4431940ab8d1fc95df2858a6c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81767) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 16:11:58.434885 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09d716a9-fa13-421e-a255-05d1920e09e2 admin admin] REQUEST_METHOD: `POST` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:11:58.435167 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09d716a9-fa13-421e-a255-05d1920e09e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:11:58.435476 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09d716a9-fa13-421e-a255-05d1920e09e2 admin admin] PATH_INFO: `/v3/users` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:11:58.436124 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09d716a9-fa13-421e-a255-05d1920e09e2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 16:11:58.438131 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09d716a9-fa13-421e-a255-05d1920e09e2 admin admin] RBAC: Authorization granted {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 16:11:58.497102 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-51bfe6e9-11a6-41de-8164-2eb1c45af484 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a88aa134951a4a139562609c27e14bc0'} {{(pid=81768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 16:11:58.497970 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-51bfe6e9-11a6-41de-8164-2eb1c45af484 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a88aa134951a4a139562609c27e14bc0'} {{(pid=81768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 16:11:58.498583 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-51bfe6e9-11a6-41de-8164-2eb1c45af484 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a88aa134951a4a139562609c27e14bc0'} {{(pid=81768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 16:11:58.521399 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-51bfe6e9-11a6-41de-8164-2eb1c45af484 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a88aa134951a4a139562609c27e14bc0'} {{(pid=81768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 16:11:58.545384 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 1071/2156] 10.222.0.15 () {60 vars in 1144 bytes} [Mon Aug 22 16:11:58 2022] POST /identity/v3/users => generated 292 bytes in 205 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 22 16:11:58.546515 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09d716a9-fa13-421e-a255-05d1920e09e2 admin admin] Authenticating user token {{(pid=81767) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 16:11:58.548048 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.fernet_utils [None req-09d716a9-fa13-421e-a255-05d1920e09e2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81767) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 16:11:58.579170 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09d716a9-fa13-421e-a255-05d1920e09e2 admin admin] Validating token access rules against request {{(pid=81767) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 16:11:58.584954 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab48bb9b-8164-48dc-93c3-1e9c424e7a34 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a98b1e4431940ab8d1fc95df2858a6c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81767) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 16:11:58.585927 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab48bb9b-8164-48dc-93c3-1e9c424e7a34 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:11:58.586215 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab48bb9b-8164-48dc-93c3-1e9c424e7a34 admin admin] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:11:58.586531 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab48bb9b-8164-48dc-93c3-1e9c424e7a34 admin admin] PATH_INFO: `/v3/projects/ce9c2f7bd3e94da7a3b5c3de403fb67c` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:11:58.587699 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab48bb9b-8164-48dc-93c3-1e9c424e7a34 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ce9c2f7bd3e94da7a3b5c3de403fb67c)` {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 16:11:58.589601 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab48bb9b-8164-48dc-93c3-1e9c424e7a34 admin admin] RBAC: Authorization granted {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 16:11:58.613634 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-51bfe6e9-11a6-41de-8164-2eb1c45af484 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a88aa134951a4a139562609c27e14bc0'} {{(pid=81768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 16:11:58.623425 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 16:11:58.636979 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 1086/2157] 10.222.0.15 () {58 vars in 1224 bytes} [Mon Aug 22 16:11:58 2022] DELETE /identity/v3/users/a88aa134951a4a139562609c27e14bc0 => generated 0 bytes in 460 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 22 16:11:58.637705 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51bfe6e9-11a6-41de-8164-2eb1c45af484 admin admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 16:11:58.645519 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-51bfe6e9-11a6-41de-8164-2eb1c45af484 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 16:11:58.737825 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.notifications [None req-ab48bb9b-8164-48dc-93c3-1e9c424e7a34 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ce9c2f7bd3e94da7a3b5c3de403fb67c', 'request_id': 'req-ab48bb9b-8164-48dc-93c3-1e9c424e7a34'} {{(pid=81767) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 16:11:58.744723 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51bfe6e9-11a6-41de-8164-2eb1c45af484 admin admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 16:11:58.753759 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.notifications [None req-ab48bb9b-8164-48dc-93c3-1e9c424e7a34 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ce9c2f7bd3e94da7a3b5c3de403fb67c', 'request_id': 'req-ab48bb9b-8164-48dc-93c3-1e9c424e7a34'} {{(pid=81767) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 16:11:58.823898 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-473ddd22-0617-4a1b-ae88-d375264eb0ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a98b1e4431940ab8d1fc95df2858a6c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 16:11:58.824528 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-473ddd22-0617-4a1b-ae88-d375264eb0ca admin admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:11:58.824830 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-473ddd22-0617-4a1b-ae88-d375264eb0ca admin admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:11:58.825100 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-473ddd22-0617-4a1b-ae88-d375264eb0ca admin admin] PATH_INFO: `/v3/roles` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:11:58.825746 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-473ddd22-0617-4a1b-ae88-d375264eb0ca admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 16:11:58.827379 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-473ddd22-0617-4a1b-ae88-d375264eb0ca admin admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 16:11:58.837896 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 1087/2158] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Aug 22 16:11:58 2022] GET /identity/v3/roles => generated 1420 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 22 16:11:58.838579 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG dbcounter [-] [81767] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81767) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 16:11:58.842120 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-473ddd22-0617-4a1b-ae88-d375264eb0ca admin admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 16:11:58.852991 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-473ddd22-0617-4a1b-ae88-d375264eb0ca admin admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 16:11:58.857603 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48750a2e-6e5b-44b2-bba3-3c5d9c659651 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a98b1e4431940ab8d1fc95df2858a6c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 16:11:58.858178 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48750a2e-6e5b-44b2-bba3-3c5d9c659651 admin admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:11:58.858466 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48750a2e-6e5b-44b2-bba3-3c5d9c659651 admin admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:11:58.858729 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48750a2e-6e5b-44b2-bba3-3c5d9c659651 admin admin] PATH_INFO: `/v3/roles` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:11:58.862636 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48750a2e-6e5b-44b2-bba3-3c5d9c659651 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 16:11:58.862636 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48750a2e-6e5b-44b2-bba3-3c5d9c659651 admin admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 16:11:58.871006 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 1088/2159] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Aug 22 16:11:58 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 22 16:11:58.875312 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48750a2e-6e5b-44b2-bba3-3c5d9c659651 admin admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 16:11:58.881968 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.notifications [None req-ab48bb9b-8164-48dc-93c3-1e9c424e7a34 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ce9c2f7bd3e94da7a3b5c3de403fb67c', 'request_id': 'req-ab48bb9b-8164-48dc-93c3-1e9c424e7a34'} {{(pid=81767) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 16:11:58.885677 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48750a2e-6e5b-44b2-bba3-3c5d9c659651 admin admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 16:11:58.891151 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f90b424c-8f79-49f4-acd0-7b5a2e9f8136 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a98b1e4431940ab8d1fc95df2858a6c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 16:11:58.891930 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f90b424c-8f79-49f4-acd0-7b5a2e9f8136 admin admin] REQUEST_METHOD: `PUT` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:11:58.892210 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f90b424c-8f79-49f4-acd0-7b5a2e9f8136 admin admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:11:58.892485 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f90b424c-8f79-49f4-acd0-7b5a2e9f8136 admin admin] PATH_INFO: `/v3/projects/4ca6ef9827884b04ad9117c11756fe00/users/256b01ff0c414712bc351d24adf6c2a1/roles/e6fc1ee3fb974125ba9b4fe55b12b45e` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:11:58.904962 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.notifications [None req-ab48bb9b-8164-48dc-93c3-1e9c424e7a34 admin admin] The token cache is being invalidate because project ce9c2f7bd3e94da7a3b5c3de403fb67c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81767) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 22 16:11:58.905367 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.notifications [None req-ab48bb9b-8164-48dc-93c3-1e9c424e7a34 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81767) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 16:11:58.906341 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.notifications [None req-ab48bb9b-8164-48dc-93c3-1e9c424e7a34 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81767) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 16:11:58.912039 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 1072/2160] 10.222.0.15 () {58 vars in 1233 bytes} [Mon Aug 22 16:11:58 2022] DELETE /identity/v3/projects/ce9c2f7bd3e94da7a3b5c3de403fb67c => generated 0 bytes in 362 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 16:11:58.913265 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab48bb9b-8164-48dc-93c3-1e9c424e7a34 admin admin] Authenticating user token {{(pid=81767) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 16:11:58.916976 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.fernet_utils [None req-ab48bb9b-8164-48dc-93c3-1e9c424e7a34 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81767) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 16:11:58.926527 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f90b424c-8f79-49f4-acd0-7b5a2e9f8136 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4ca6ef9827884b04ad9117c11756fe00, user_id=256b01ff0c414712bc351d24adf6c2a1, role_id=e6fc1ee3fb974125ba9b4fe55b12b45e)` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 16:11:58.929651 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f90b424c-8f79-49f4-acd0-7b5a2e9f8136 admin admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 16:11:58.942608 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab48bb9b-8164-48dc-93c3-1e9c424e7a34 admin admin] Validating token access rules against request {{(pid=81767) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 16:11:58.950969 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3307be44-4f9e-4c1d-af7d-782b3d94ab24 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a98b1e4431940ab8d1fc95df2858a6c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81767) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 16:11:58.952016 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3307be44-4f9e-4c1d-af7d-782b3d94ab24 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:11:58.952485 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3307be44-4f9e-4c1d-af7d-782b3d94ab24 admin admin] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:11:58.952859 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3307be44-4f9e-4c1d-af7d-782b3d94ab24 admin admin] PATH_INFO: `/v3/users/ffb8485c36c147239e279087f49735dd` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:11:58.967790 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3307be44-4f9e-4c1d-af7d-782b3d94ab24 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ffb8485c36c147239e279087f49735dd)` {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 16:11:58.970012 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3307be44-4f9e-4c1d-af7d-782b3d94ab24 admin admin] RBAC: Authorization granted {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 16:11:59.005069 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 1089/2161] 10.222.0.15 () {60 vars in 1483 bytes} [Mon Aug 22 16:11:58 2022] PUT /identity/v3/projects/4ca6ef9827884b04ad9117c11756fe00/users/256b01ff0c414712bc351d24adf6c2a1/roles/e6fc1ee3fb974125ba9b4fe55b12b45e => generated 0 bytes in 130 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 16:11:59.016944 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b8772f0-9212-4158-b223-eeab4a529dc6 None None] REQUEST_METHOD: `POST` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:11:59.017229 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b8772f0-9212-4158-b223-eeab4a529dc6 None None] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:11:59.017494 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b8772f0-9212-4158-b223-eeab4a529dc6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:11:59.047734 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.auth.core [None req-0b8772f0-9212-4158-b223-eeab4a529dc6 None None] MFA Rules not processed for user `256b01ff0c414712bc351d24adf6c2a1`. Rule list: `[]` (Enabled: `True`). {{(pid=81768) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 16:11:59.048631 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-0b8772f0-9212-4158-b223-eeab4a529dc6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 16:11:59.116453 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 1090/2162] 10.222.0.15 () {58 vars in 958 bytes} [Mon Aug 22 16:11:59 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 22 16:11:59.120798 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b8772f0-9212-4158-b223-eeab4a529dc6 None None] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 16:11:59.122093 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-0b8772f0-9212-4158-b223-eeab4a529dc6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 16:11:59.189721 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b8772f0-9212-4158-b223-eeab4a529dc6 None None] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 16:11:59.197441 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f92519d-8379-4b09-97ac-0a1a4eb60b3a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a98b1e4431940ab8d1fc95df2858a6c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 16:11:59.198069 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f92519d-8379-4b09-97ac-0a1a4eb60b3a admin admin] REQUEST_METHOD: `POST` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:11:59.198369 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f92519d-8379-4b09-97ac-0a1a4eb60b3a admin admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:11:59.198608 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f92519d-8379-4b09-97ac-0a1a4eb60b3a admin admin] PATH_INFO: `/v3/projects` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:11:59.199235 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f92519d-8379-4b09-97ac-0a1a4eb60b3a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 16:11:59.201414 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f92519d-8379-4b09-97ac-0a1a4eb60b3a admin admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 16:11:59.211128 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.notifications [None req-3307be44-4f9e-4c1d-af7d-782b3d94ab24 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ffb8485c36c147239e279087f49735dd'} {{(pid=81767) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 16:11:59.211826 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.notifications [None req-3307be44-4f9e-4c1d-af7d-782b3d94ab24 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ffb8485c36c147239e279087f49735dd'} {{(pid=81767) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 16:11:59.304499 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.notifications [None req-3307be44-4f9e-4c1d-af7d-782b3d94ab24 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ffb8485c36c147239e279087f49735dd'} {{(pid=81767) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 16:11:59.312152 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 1091/2163] 10.222.0.15 () {60 vars in 1153 bytes} [Mon Aug 22 16:11:59 2022] POST /identity/v3/projects => generated 387 bytes in 192 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 22 16:11:59.313013 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f92519d-8379-4b09-97ac-0a1a4eb60b3a admin admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 16:11:59.314484 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.notifications [None req-3307be44-4f9e-4c1d-af7d-782b3d94ab24 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ffb8485c36c147239e279087f49735dd'} {{(pid=81767) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 16:11:59.314823 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-3f92519d-8379-4b09-97ac-0a1a4eb60b3a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 16:11:59.328577 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.notifications [None req-3307be44-4f9e-4c1d-af7d-782b3d94ab24 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ffb8485c36c147239e279087f49735dd'} {{(pid=81767) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 16:11:59.337603 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 1073/2164] 10.222.0.15 () {58 vars in 1224 bytes} [Mon Aug 22 16:11:58 2022] DELETE /identity/v3/users/ffb8485c36c147239e279087f49735dd => generated 0 bytes in 423 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 16:11:59.337603 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3307be44-4f9e-4c1d-af7d-782b3d94ab24 admin admin] Authenticating user token {{(pid=81767) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 16:11:59.338679 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.fernet_utils [None req-3307be44-4f9e-4c1d-af7d-782b3d94ab24 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81767) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 16:11:59.398830 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f92519d-8379-4b09-97ac-0a1a4eb60b3a admin admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 16:11:59.412562 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3307be44-4f9e-4c1d-af7d-782b3d94ab24 admin admin] Validating token access rules against request {{(pid=81767) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 16:11:59.418448 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b77d6879-41f6-488c-9729-5025d1f2d304 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a98b1e4431940ab8d1fc95df2858a6c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81767) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 16:11:59.419409 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b77d6879-41f6-488c-9729-5025d1f2d304 admin admin] REQUEST_METHOD: `POST` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:11:59.419670 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b77d6879-41f6-488c-9729-5025d1f2d304 admin admin] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:11:59.419939 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b77d6879-41f6-488c-9729-5025d1f2d304 admin admin] PATH_INFO: `/v3/users` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:11:59.420728 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b77d6879-41f6-488c-9729-5025d1f2d304 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 16:11:59.422694 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b77d6879-41f6-488c-9729-5025d1f2d304 admin admin] RBAC: Authorization granted {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 16:11:59.463582 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-927cc904-3747-40f1-a5fd-a6ea2bf8eabe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a98b1e4431940ab8d1fc95df2858a6c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 16:11:59.464251 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-927cc904-3747-40f1-a5fd-a6ea2bf8eabe admin admin] REQUEST_METHOD: `DELETE` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:11:59.464531 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-927cc904-3747-40f1-a5fd-a6ea2bf8eabe admin admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:11:59.464803 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-927cc904-3747-40f1-a5fd-a6ea2bf8eabe admin admin] PATH_INFO: `/v3/projects/cd9cd02faaa64a2d89782aba11c73b22` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:11:59.465932 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-927cc904-3747-40f1-a5fd-a6ea2bf8eabe admin admin] RBAC: Authorizing `identity:delete_project(project_id=cd9cd02faaa64a2d89782aba11c73b22)` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 16:11:59.467940 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-927cc904-3747-40f1-a5fd-a6ea2bf8eabe admin admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 16:11:59.547078 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 1074/2165] 10.222.0.15 () {60 vars in 1144 bytes} [Mon Aug 22 16:11:59 2022] POST /identity/v3/users => generated 300 bytes in 210 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 22 16:11:59.550994 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b77d6879-41f6-488c-9729-5025d1f2d304 admin admin] Authenticating user token {{(pid=81767) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 16:11:59.562243 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b77d6879-41f6-488c-9729-5025d1f2d304 admin admin] Validating token access rules against request {{(pid=81767) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 16:11:59.582802 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-927cc904-3747-40f1-a5fd-a6ea2bf8eabe admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cd9cd02faaa64a2d89782aba11c73b22', 'request_id': 'req-927cc904-3747-40f1-a5fd-a6ea2bf8eabe'} {{(pid=81768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 16:11:59.589387 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-927cc904-3747-40f1-a5fd-a6ea2bf8eabe admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cd9cd02faaa64a2d89782aba11c73b22', 'request_id': 'req-927cc904-3747-40f1-a5fd-a6ea2bf8eabe'} {{(pid=81768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 16:11:59.645241 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d15341f-b93d-43ad-92fe-5119a6501c7c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a98b1e4431940ab8d1fc95df2858a6c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81767) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 16:11:59.645858 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d15341f-b93d-43ad-92fe-5119a6501c7c admin admin] REQUEST_METHOD: `GET` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:11:59.646138 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d15341f-b93d-43ad-92fe-5119a6501c7c admin admin] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:11:59.646410 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d15341f-b93d-43ad-92fe-5119a6501c7c admin admin] PATH_INFO: `/v3/roles` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:11:59.647228 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d15341f-b93d-43ad-92fe-5119a6501c7c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 16:11:59.648881 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d15341f-b93d-43ad-92fe-5119a6501c7c admin admin] RBAC: Authorization granted {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 16:11:59.653102 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-927cc904-3747-40f1-a5fd-a6ea2bf8eabe admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cd9cd02faaa64a2d89782aba11c73b22', 'request_id': 'req-927cc904-3747-40f1-a5fd-a6ea2bf8eabe'} {{(pid=81768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 16:11:59.663588 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 1075/2166] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Aug 22 16:11:59 2022] GET /identity/v3/roles => generated 1420 bytes in 113 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 22 16:11:59.664474 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d15341f-b93d-43ad-92fe-5119a6501c7c admin admin] Authenticating user token {{(pid=81767) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 16:11:59.668708 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-927cc904-3747-40f1-a5fd-a6ea2bf8eabe admin admin] The token cache is being invalidate because project cd9cd02faaa64a2d89782aba11c73b22 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81768) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 22 16:11:59.668961 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-927cc904-3747-40f1-a5fd-a6ea2bf8eabe admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 16:11:59.670358 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-927cc904-3747-40f1-a5fd-a6ea2bf8eabe admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 16:11:59.671922 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 1092/2167] 10.222.0.15 () {58 vars in 1233 bytes} [Mon Aug 22 16:11:59 2022] DELETE /identity/v3/projects/cd9cd02faaa64a2d89782aba11c73b22 => generated 0 bytes in 359 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 16:11:59.674669 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d15341f-b93d-43ad-92fe-5119a6501c7c admin admin] Validating token access rules against request {{(pid=81767) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 16:11:59.676082 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-927cc904-3747-40f1-a5fd-a6ea2bf8eabe admin admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 16:11:59.679580 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-927cc904-3747-40f1-a5fd-a6ea2bf8eabe admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 16:11:59.679807 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-809ebe13-2569-4244-8b68-5ad70737fcdf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a98b1e4431940ab8d1fc95df2858a6c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81767) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 16:11:59.680474 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-809ebe13-2569-4244-8b68-5ad70737fcdf admin admin] REQUEST_METHOD: `PUT` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:11:59.680757 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-809ebe13-2569-4244-8b68-5ad70737fcdf admin admin] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:11:59.681031 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-809ebe13-2569-4244-8b68-5ad70737fcdf admin admin] PATH_INFO: `/v3/projects/a1cf040c8a984da28c85d8d3cea57530/users/08d8e4a8e6fa4cccb860ba64f4c0eacd/roles/9d9848cfd22e4b089db443c350e56670` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:11:59.697370 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-809ebe13-2569-4244-8b68-5ad70737fcdf admin admin] RBAC: Authorizing `identity:create_grant(project_id=a1cf040c8a984da28c85d8d3cea57530, user_id=08d8e4a8e6fa4cccb860ba64f4c0eacd, role_id=9d9848cfd22e4b089db443c350e56670)` {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 16:11:59.698193 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-927cc904-3747-40f1-a5fd-a6ea2bf8eabe admin admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 16:11:59.700550 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-809ebe13-2569-4244-8b68-5ad70737fcdf admin admin] RBAC: Authorization granted {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 16:11:59.704996 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b78e34ba-f842-44b0-aa94-0cec354dc398 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a98b1e4431940ab8d1fc95df2858a6c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 16:11:59.706303 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b78e34ba-f842-44b0-aa94-0cec354dc398 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:11:59.706666 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b78e34ba-f842-44b0-aa94-0cec354dc398 admin admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:11:59.707046 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b78e34ba-f842-44b0-aa94-0cec354dc398 admin admin] PATH_INFO: `/v3/users/b527a92a1be840a19745f067f2729477` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:11:59.708364 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b78e34ba-f842-44b0-aa94-0cec354dc398 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b527a92a1be840a19745f067f2729477)` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 16:11:59.710688 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b78e34ba-f842-44b0-aa94-0cec354dc398 admin admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 16:11:59.809998 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 16:11:59.823200 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 1076/2168] 10.222.0.15 () {60 vars in 1483 bytes} [Mon Aug 22 16:11:59 2022] PUT /identity/v3/projects/a1cf040c8a984da28c85d8d3cea57530/users/08d8e4a8e6fa4cccb860ba64f4c0eacd/roles/9d9848cfd22e4b089db443c350e56670 => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 16:11:59.823200 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-809ebe13-2569-4244-8b68-5ad70737fcdf admin admin] Authenticating user token {{(pid=81767) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 16:11:59.823200 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.fernet_utils [None req-809ebe13-2569-4244-8b68-5ad70737fcdf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81767) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 16:11:59.892237 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-809ebe13-2569-4244-8b68-5ad70737fcdf admin admin] Validating token access rules against request {{(pid=81767) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 16:11:59.897212 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc749b96-0c16-4fb7-be1f-cb314023f1fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a98b1e4431940ab8d1fc95df2858a6c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81767) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 16:11:59.897848 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc749b96-0c16-4fb7-be1f-cb314023f1fe admin admin] REQUEST_METHOD: `DELETE` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:11:59.898121 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc749b96-0c16-4fb7-be1f-cb314023f1fe admin admin] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:11:59.898389 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc749b96-0c16-4fb7-be1f-cb314023f1fe admin admin] PATH_INFO: `/v3/projects/1d9445484db94d76bf676b2f24693c32` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:11:59.899698 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc749b96-0c16-4fb7-be1f-cb314023f1fe admin admin] RBAC: Authorizing `identity:delete_project(project_id=1d9445484db94d76bf676b2f24693c32)` {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 16:11:59.902252 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc749b96-0c16-4fb7-be1f-cb314023f1fe admin admin] RBAC: Authorization granted {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 16:11:59.905082 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG dbcounter [-] [81767] Writing DB stats keystone:INSERT=4,keystone:SELECT=96 {{(pid=81767) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 16:11:59.907212 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-b78e34ba-f842-44b0-aa94-0cec354dc398 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b527a92a1be840a19745f067f2729477'} {{(pid=81768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 16:11:59.907831 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-b78e34ba-f842-44b0-aa94-0cec354dc398 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b527a92a1be840a19745f067f2729477'} {{(pid=81768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 16:11:59.908464 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-b78e34ba-f842-44b0-aa94-0cec354dc398 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b527a92a1be840a19745f067f2729477'} {{(pid=81768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 16:11:59.929402 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-b78e34ba-f842-44b0-aa94-0cec354dc398 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b527a92a1be840a19745f067f2729477'} {{(pid=81768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 16:11:59.998375 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-b78e34ba-f842-44b0-aa94-0cec354dc398 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b527a92a1be840a19745f067f2729477'} {{(pid=81768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 16:12:00.009429 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.notifications [None req-fc749b96-0c16-4fb7-be1f-cb314023f1fe admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1d9445484db94d76bf676b2f24693c32', 'request_id': 'req-fc749b96-0c16-4fb7-be1f-cb314023f1fe'} {{(pid=81767) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 16:12:00.015940 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.notifications [None req-fc749b96-0c16-4fb7-be1f-cb314023f1fe admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1d9445484db94d76bf676b2f24693c32', 'request_id': 'req-fc749b96-0c16-4fb7-be1f-cb314023f1fe'} {{(pid=81767) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 16:12:00.020105 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 1093/2169] 10.222.0.15 () {58 vars in 1224 bytes} [Mon Aug 22 16:11:59 2022] DELETE /identity/v3/users/b527a92a1be840a19745f067f2729477 => generated 0 bytes in 344 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 16:12:00.022634 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b78e34ba-f842-44b0-aa94-0cec354dc398 admin admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 16:12:00.022634 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-b78e34ba-f842-44b0-aa94-0cec354dc398 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 16:12:00.091619 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b78e34ba-f842-44b0-aa94-0cec354dc398 admin admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 16:12:00.097062 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-015c8743-40ac-497f-b05c-ed9d703f135c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a98b1e4431940ab8d1fc95df2858a6c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 16:12:00.097695 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-015c8743-40ac-497f-b05c-ed9d703f135c admin admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:12:00.099429 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-015c8743-40ac-497f-b05c-ed9d703f135c admin admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:12:00.099429 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-015c8743-40ac-497f-b05c-ed9d703f135c admin admin] PATH_INFO: `/v3/roles` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:12:00.099907 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-015c8743-40ac-497f-b05c-ed9d703f135c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 16:12:00.101782 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-015c8743-40ac-497f-b05c-ed9d703f135c admin admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 16:12:00.107130 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.notifications [None req-fc749b96-0c16-4fb7-be1f-cb314023f1fe admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1d9445484db94d76bf676b2f24693c32', 'request_id': 'req-fc749b96-0c16-4fb7-be1f-cb314023f1fe'} {{(pid=81767) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 16:12:00.115348 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 1094/2170] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Aug 22 16:12:00 2022] GET /identity/v3/roles => generated 1420 bytes in 95 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 22 16:12:00.120267 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-015c8743-40ac-497f-b05c-ed9d703f135c admin admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 16:12:00.133062 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-015c8743-40ac-497f-b05c-ed9d703f135c admin admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 16:12:00.133927 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.notifications [None req-fc749b96-0c16-4fb7-be1f-cb314023f1fe admin admin] The token cache is being invalidate because project 1d9445484db94d76bf676b2f24693c32 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81767) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 22 16:12:00.134301 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.notifications [None req-fc749b96-0c16-4fb7-be1f-cb314023f1fe admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81767) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 16:12:00.135715 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.notifications [None req-fc749b96-0c16-4fb7-be1f-cb314023f1fe admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81767) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 16:12:00.139253 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 1077/2171] 10.222.0.15 () {58 vars in 1233 bytes} [Mon Aug 22 16:11:59 2022] DELETE /identity/v3/projects/1d9445484db94d76bf676b2f24693c32 => generated 0 bytes in 321 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 16:12:00.141226 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bd92a0c-7304-47db-a12c-4b3c49cfaab9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a98b1e4431940ab8d1fc95df2858a6c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 16:12:00.144659 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bd92a0c-7304-47db-a12c-4b3c49cfaab9 admin admin] REQUEST_METHOD: `PUT` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:12:00.144659 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bd92a0c-7304-47db-a12c-4b3c49cfaab9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:12:00.144659 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bd92a0c-7304-47db-a12c-4b3c49cfaab9 admin admin] PATH_INFO: `/v3/projects/a1cf040c8a984da28c85d8d3cea57530/users/08d8e4a8e6fa4cccb860ba64f4c0eacd/roles/9d9848cfd22e4b089db443c350e56670` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:12:00.145436 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bd92a0c-7304-47db-a12c-4b3c49cfaab9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a1cf040c8a984da28c85d8d3cea57530, user_id=08d8e4a8e6fa4cccb860ba64f4c0eacd, role_id=9d9848cfd22e4b089db443c350e56670)` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 16:12:00.148698 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d81683bb-d92c-44a5-94f7-408bd89cd284 None None] REQUEST_METHOD: `POST` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:12:00.149693 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d81683bb-d92c-44a5-94f7-408bd89cd284 None None] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:12:00.150141 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d81683bb-d92c-44a5-94f7-408bd89cd284 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:12:00.151046 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bd92a0c-7304-47db-a12c-4b3c49cfaab9 admin admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 16:12:00.163551 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 1095/2172] 10.222.0.15 () {60 vars in 1483 bytes} [Mon Aug 22 16:12:00 2022] PUT /identity/v3/projects/a1cf040c8a984da28c85d8d3cea57530/users/08d8e4a8e6fa4cccb860ba64f4c0eacd/roles/9d9848cfd22e4b089db443c350e56670 => generated 0 bytes in 44 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 16:12:00.171908 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.auth.core [None req-d81683bb-d92c-44a5-94f7-408bd89cd284 None None] MFA Rules not processed for user `6b2199ae0fb0480aa917ada391f5ec54`. Rule list: `[]` (Enabled: `True`). {{(pid=81767) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 16:12:00.173155 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.fernet_utils [None req-d81683bb-d92c-44a5-94f7-408bd89cd284 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81767) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 16:12:00.177532 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31318099-24e5-4aa8-a7c8-bd77ecf6ea5a None None] REQUEST_METHOD: `POST` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:12:00.177832 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31318099-24e5-4aa8-a7c8-bd77ecf6ea5a None None] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:12:00.178087 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31318099-24e5-4aa8-a7c8-bd77ecf6ea5a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:12:00.211614 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.auth.core [None req-31318099-24e5-4aa8-a7c8-bd77ecf6ea5a None None] MFA Rules not processed for user `08d8e4a8e6fa4cccb860ba64f4c0eacd`. Rule list: `[]` (Enabled: `True`). {{(pid=81768) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 16:12:00.213567 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-31318099-24e5-4aa8-a7c8-bd77ecf6ea5a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 16:12:00.243784 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 1078/2173] 10.222.0.15 () {58 vars in 958 bytes} [Mon Aug 22 16:12:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 16:12:00.251203 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ee55d3d-83a7-4422-a72d-2dff013890bb None None] REQUEST_METHOD: `POST` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:12:00.252722 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ee55d3d-83a7-4422-a72d-2dff013890bb None None] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:12:00.252879 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ee55d3d-83a7-4422-a72d-2dff013890bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:12:00.286651 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.auth.core [None req-6ee55d3d-83a7-4422-a72d-2dff013890bb None None] MFA Rules not processed for user `6b2199ae0fb0480aa917ada391f5ec54`. Rule list: `[]` (Enabled: `True`). {{(pid=81767) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 16:12:00.287992 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.fernet_utils [None req-6ee55d3d-83a7-4422-a72d-2dff013890bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81767) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 16:12:00.303431 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 1079/2174] 10.222.0.15 () {58 vars in 958 bytes} [Mon Aug 22 16:12:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 53 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 16:12:00.306949 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ee55d3d-83a7-4422-a72d-2dff013890bb None None] Authenticating user token {{(pid=81767) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 16:12:00.317076 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 1096/2175] 10.222.0.15 () {58 vars in 958 bytes} [Mon Aug 22 16:12:00 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 22 16:12:00.321049 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ee55d3d-83a7-4422-a72d-2dff013890bb None None] Validating token access rules against request {{(pid=81767) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 16:12:00.331670 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31318099-24e5-4aa8-a7c8-bd77ecf6ea5a None None] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 16:12:00.337408 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-31318099-24e5-4aa8-a7c8-bd77ecf6ea5a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 16:12:00.337566 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e6c2292-4237-4a61-ab60-cb73bffbb731 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a98b1e4431940ab8d1fc95df2858a6c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81767) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 16:12:00.337695 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e6c2292-4237-4a61-ab60-cb73bffbb731 admin admin] REQUEST_METHOD: `GET` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:12:00.338007 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e6c2292-4237-4a61-ab60-cb73bffbb731 admin admin] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:12:00.338286 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e6c2292-4237-4a61-ab60-cb73bffbb731 admin admin] PATH_INFO: `/v3/domains` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:12:00.338940 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e6c2292-4237-4a61-ab60-cb73bffbb731 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81767) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 16:12:00.339220 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e6c2292-4237-4a61-ab60-cb73bffbb731 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 16:12:00.341868 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e6c2292-4237-4a61-ab60-cb73bffbb731 admin admin] RBAC: Authorization granted {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 16:12:00.360311 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 1080/2176] 10.222.0.15 () {58 vars in 1153 bytes} [Mon Aug 22 16:12:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 16:12:00.361614 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e6c2292-4237-4a61-ab60-cb73bffbb731 admin admin] Authenticating user token {{(pid=81767) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 16:12:00.375521 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e6c2292-4237-4a61-ab60-cb73bffbb731 admin admin] Validating token access rules against request {{(pid=81767) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 16:12:00.383876 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99593983-c73c-4096-b052-b3fd72f43ffb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a98b1e4431940ab8d1fc95df2858a6c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81767) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 16:12:00.384492 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99593983-c73c-4096-b052-b3fd72f43ffb admin admin] REQUEST_METHOD: `POST` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:12:00.384767 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99593983-c73c-4096-b052-b3fd72f43ffb admin admin] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:12:00.385034 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99593983-c73c-4096-b052-b3fd72f43ffb admin admin] PATH_INFO: `/v3/projects` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:12:00.385874 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99593983-c73c-4096-b052-b3fd72f43ffb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 16:12:00.387885 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99593983-c73c-4096-b052-b3fd72f43ffb admin admin] RBAC: Authorization granted {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 16:12:00.399062 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 16:12:00.409967 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31318099-24e5-4aa8-a7c8-bd77ecf6ea5a None None] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 16:12:00.413963 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd51f4d6-8db5-463d-af94-2f008d41e382 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79706d480f864998b9f8594be407ceab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4ee0413f9ef4d4fb1f95a660babe090', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 16:12:00.414652 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd51f4d6-8db5-463d-af94-2f008d41e382 service cinder] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:12:00.414930 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd51f4d6-8db5-463d-af94-2f008d41e382 service cinder] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:12:00.415200 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd51f4d6-8db5-463d-af94-2f008d41e382 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:12:00.425059 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd51f4d6-8db5-463d-af94-2f008d41e382 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 16:12:00.426495 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd51f4d6-8db5-463d-af94-2f008d41e382 service cinder] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 16:12:00.436158 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 1097/2177] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Aug 22 16:12:00 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 22 16:12:00.491815 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd51f4d6-8db5-463d-af94-2f008d41e382 service cinder] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 16:12:00.493559 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-bd51f4d6-8db5-463d-af94-2f008d41e382 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 16:12:00.514680 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd51f4d6-8db5-463d-af94-2f008d41e382 service cinder] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 16:12:00.519779 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a6cbe80-bd2f-47a9-a2c2-94fa91be299f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a98b1e4431940ab8d1fc95df2858a6c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 16:12:00.520896 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a6cbe80-bd2f-47a9-a2c2-94fa91be299f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:12:00.521353 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a6cbe80-bd2f-47a9-a2c2-94fa91be299f admin admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:12:00.521791 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a6cbe80-bd2f-47a9-a2c2-94fa91be299f admin admin] PATH_INFO: `/v3/projects/a23886ab1d2a48f3a0a21077d585f1a0` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:12:00.523591 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a6cbe80-bd2f-47a9-a2c2-94fa91be299f admin admin] RBAC: Authorizing `identity:delete_project(project_id=a23886ab1d2a48f3a0a21077d585f1a0)` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 16:12:00.525196 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 1081/2178] 10.222.0.15 () {60 vars in 1153 bytes} [Mon Aug 22 16:12:00 2022] POST /identity/v3/projects => generated 377 bytes in 164 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 22 16:12:00.525827 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a6cbe80-bd2f-47a9-a2c2-94fa91be299f admin admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 16:12:00.525929 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99593983-c73c-4096-b052-b3fd72f43ffb admin admin] Authenticating user token {{(pid=81767) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 16:12:00.527618 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.fernet_utils [None req-99593983-c73c-4096-b052-b3fd72f43ffb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81767) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 16:12:00.597513 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99593983-c73c-4096-b052-b3fd72f43ffb admin admin] Validating token access rules against request {{(pid=81767) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 16:12:00.603412 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-283b4714-2755-4009-9f0c-83673605606f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a98b1e4431940ab8d1fc95df2858a6c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81767) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 16:12:00.604079 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-283b4714-2755-4009-9f0c-83673605606f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:12:00.604358 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-283b4714-2755-4009-9f0c-83673605606f admin admin] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:12:00.604634 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-283b4714-2755-4009-9f0c-83673605606f admin admin] PATH_INFO: `/v3/users/256b01ff0c414712bc351d24adf6c2a1` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:12:00.605859 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-283b4714-2755-4009-9f0c-83673605606f admin admin] RBAC: Authorizing `identity:delete_user(user_id=256b01ff0c414712bc351d24adf6c2a1)` {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 16:12:00.608116 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-283b4714-2755-4009-9f0c-83673605606f admin admin] RBAC: Authorization granted {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 16:12:00.660834 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-2a6cbe80-bd2f-47a9-a2c2-94fa91be299f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a23886ab1d2a48f3a0a21077d585f1a0', 'request_id': 'req-2a6cbe80-bd2f-47a9-a2c2-94fa91be299f'} {{(pid=81768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 16:12:00.666198 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-2a6cbe80-bd2f-47a9-a2c2-94fa91be299f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a23886ab1d2a48f3a0a21077d585f1a0', 'request_id': 'req-2a6cbe80-bd2f-47a9-a2c2-94fa91be299f'} {{(pid=81768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 16:12:00.671753 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG dbcounter [-] [81767] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=81767) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 16:12:00.762180 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-2a6cbe80-bd2f-47a9-a2c2-94fa91be299f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a23886ab1d2a48f3a0a21077d585f1a0', 'request_id': 'req-2a6cbe80-bd2f-47a9-a2c2-94fa91be299f'} {{(pid=81768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 16:12:00.776624 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-2a6cbe80-bd2f-47a9-a2c2-94fa91be299f admin admin] The token cache is being invalidate because project a23886ab1d2a48f3a0a21077d585f1a0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81768) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 22 16:12:00.776918 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-2a6cbe80-bd2f-47a9-a2c2-94fa91be299f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 16:12:00.777618 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-2a6cbe80-bd2f-47a9-a2c2-94fa91be299f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 16:12:00.779268 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 1098/2179] 10.222.0.15 () {58 vars in 1233 bytes} [Mon Aug 22 16:12:00 2022] DELETE /identity/v3/projects/a23886ab1d2a48f3a0a21077d585f1a0 => generated 0 bytes in 288 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 22 16:12:00.783991 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a6cbe80-bd2f-47a9-a2c2-94fa91be299f admin admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 16:12:00.783991 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-2a6cbe80-bd2f-47a9-a2c2-94fa91be299f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 16:12:00.854499 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a6cbe80-bd2f-47a9-a2c2-94fa91be299f admin admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 16:12:00.859261 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df68d7ff-758e-45a3-b4eb-dfa110cd1f0e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a98b1e4431940ab8d1fc95df2858a6c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 16:12:00.859900 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df68d7ff-758e-45a3-b4eb-dfa110cd1f0e admin admin] REQUEST_METHOD: `POST` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:12:00.860179 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df68d7ff-758e-45a3-b4eb-dfa110cd1f0e admin admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:12:00.860452 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df68d7ff-758e-45a3-b4eb-dfa110cd1f0e admin admin] PATH_INFO: `/v3/users` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:12:00.861068 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df68d7ff-758e-45a3-b4eb-dfa110cd1f0e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 16:12:00.863065 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df68d7ff-758e-45a3-b4eb-dfa110cd1f0e admin admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 16:12:00.896772 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.notifications [None req-283b4714-2755-4009-9f0c-83673605606f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '256b01ff0c414712bc351d24adf6c2a1'} {{(pid=81767) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 16:12:00.897422 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.notifications [None req-283b4714-2755-4009-9f0c-83673605606f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '256b01ff0c414712bc351d24adf6c2a1'} {{(pid=81767) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 16:12:01.011370 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 1099/2180] 10.222.0.15 () {60 vars in 1144 bytes} [Mon Aug 22 16:12:00 2022] POST /identity/v3/users => generated 289 bytes in 232 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 22 16:12:01.013054 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74c2fc04-cd89-4d94-86ad-65d99a45ea55 None None] REQUEST_METHOD: `POST` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:12:01.013499 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74c2fc04-cd89-4d94-86ad-65d99a45ea55 None None] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:12:01.013909 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74c2fc04-cd89-4d94-86ad-65d99a45ea55 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:12:01.041416 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.notifications [None req-283b4714-2755-4009-9f0c-83673605606f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '256b01ff0c414712bc351d24adf6c2a1'} {{(pid=81767) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 16:12:01.047521 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.auth.core [None req-74c2fc04-cd89-4d94-86ad-65d99a45ea55 None None] MFA Rules not processed for user `6b2199ae0fb0480aa917ada391f5ec54`. Rule list: `[]` (Enabled: `True`). {{(pid=81768) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 16:12:01.048980 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-74c2fc04-cd89-4d94-86ad-65d99a45ea55 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 16:12:01.049428 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.notifications [None req-283b4714-2755-4009-9f0c-83673605606f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '256b01ff0c414712bc351d24adf6c2a1'} {{(pid=81767) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 16:12:01.065042 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 1100/2181] 10.222.0.15 () {58 vars in 958 bytes} [Mon Aug 22 16:12:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 53 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 16:12:01.066593 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74c2fc04-cd89-4d94-86ad-65d99a45ea55 None None] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 16:12:01.068200 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-74c2fc04-cd89-4d94-86ad-65d99a45ea55 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 16:12:01.072391 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.notifications [None req-283b4714-2755-4009-9f0c-83673605606f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '256b01ff0c414712bc351d24adf6c2a1'} {{(pid=81767) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 16:12:01.081551 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74c2fc04-cd89-4d94-86ad-65d99a45ea55 None None] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 16:12:01.082809 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 1082/2182] 10.222.0.15 () {58 vars in 1224 bytes} [Mon Aug 22 16:12:00 2022] DELETE /identity/v3/users/256b01ff0c414712bc351d24adf6c2a1 => generated 0 bytes in 557 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 16:12:01.084524 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd327f36-7bec-42bd-87be-231e0ec3cc04 None None] REQUEST_METHOD: `POST` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:12:01.084808 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd327f36-7bec-42bd-87be-231e0ec3cc04 None None] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:12:01.085076 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd327f36-7bec-42bd-87be-231e0ec3cc04 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:12:01.107293 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.auth.core [None req-fd327f36-7bec-42bd-87be-231e0ec3cc04 None None] MFA Rules not processed for user `6b2199ae0fb0480aa917ada391f5ec54`. Rule list: `[]` (Enabled: `True`). {{(pid=81767) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 16:12:01.108303 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.fernet_utils [None req-fd327f36-7bec-42bd-87be-231e0ec3cc04 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81767) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 16:12:01.150517 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ede2922-a303-4229-a32a-bb146b3ab4f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a98b1e4431940ab8d1fc95df2858a6c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 16:12:01.151155 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ede2922-a303-4229-a32a-bb146b3ab4f6 admin admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:12:01.151465 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ede2922-a303-4229-a32a-bb146b3ab4f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:12:01.151911 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ede2922-a303-4229-a32a-bb146b3ab4f6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:12:01.152636 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ede2922-a303-4229-a32a-bb146b3ab4f6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 16:12:01.154247 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ede2922-a303-4229-a32a-bb146b3ab4f6 admin admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 16:12:01.165417 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 1101/2183] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Aug 22 16:12:01 2022] GET /identity/v3/roles => generated 1420 bytes in 100 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 22 16:12:01.166679 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94afe4d7-9b82-4852-848d-c79bef825866 None None] REQUEST_METHOD: `POST` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:12:01.167560 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94afe4d7-9b82-4852-848d-c79bef825866 None None] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:12:01.167560 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94afe4d7-9b82-4852-848d-c79bef825866 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:12:01.177666 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 1083/2184] 10.222.0.15 () {58 vars in 958 bytes} [Mon Aug 22 16:12:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 16:12:01.178987 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd327f36-7bec-42bd-87be-231e0ec3cc04 None None] Authenticating user token {{(pid=81767) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 16:12:01.189941 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd327f36-7bec-42bd-87be-231e0ec3cc04 None None] Validating token access rules against request {{(pid=81767) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 16:12:01.197842 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.auth.core [None req-94afe4d7-9b82-4852-848d-c79bef825866 None None] MFA Rules not processed for user `6b2199ae0fb0480aa917ada391f5ec54`. Rule list: `[]` (Enabled: `True`). {{(pid=81768) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 16:12:01.199110 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-94afe4d7-9b82-4852-848d-c79bef825866 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 16:12:01.209744 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c0d6104-cb99-4886-b699-61a878977c60 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a98b1e4431940ab8d1fc95df2858a6c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81767) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 16:12:01.210316 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c0d6104-cb99-4886-b699-61a878977c60 admin admin] REQUEST_METHOD: `GET` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:12:01.210590 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c0d6104-cb99-4886-b699-61a878977c60 admin admin] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:12:01.210855 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c0d6104-cb99-4886-b699-61a878977c60 admin admin] PATH_INFO: `/v3/roles` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:12:01.211375 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 1102/2185] 10.222.0.15 () {58 vars in 958 bytes} [Mon Aug 22 16:12:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 45 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 16:12:01.211550 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c0d6104-cb99-4886-b699-61a878977c60 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 16:12:01.213171 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c0d6104-cb99-4886-b699-61a878977c60 admin admin] RBAC: Authorization granted {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 16:12:01.216251 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94afe4d7-9b82-4852-848d-c79bef825866 None None] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 16:12:01.225012 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 1084/2186] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Aug 22 16:12:01 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 22 16:12:01.229040 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c0d6104-cb99-4886-b699-61a878977c60 admin admin] Authenticating user token {{(pid=81767) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 16:12:01.231765 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.fernet_utils [None req-3c0d6104-cb99-4886-b699-61a878977c60 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81767) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 16:12:01.236483 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94afe4d7-9b82-4852-848d-c79bef825866 None None] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 16:12:01.238539 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-381c4d50-c1d3-4a4f-9f30-89233d13a1b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a98b1e4431940ab8d1fc95df2858a6c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 16:12:01.239298 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-381c4d50-c1d3-4a4f-9f30-89233d13a1b4 admin admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:12:01.239604 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-381c4d50-c1d3-4a4f-9f30-89233d13a1b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:12:01.239945 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-381c4d50-c1d3-4a4f-9f30-89233d13a1b4 admin admin] PATH_INFO: `/v3/domains` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:12:01.240605 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-381c4d50-c1d3-4a4f-9f30-89233d13a1b4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81768) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 16:12:01.240910 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-381c4d50-c1d3-4a4f-9f30-89233d13a1b4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 16:12:01.242635 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-381c4d50-c1d3-4a4f-9f30-89233d13a1b4 admin admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 16:12:01.258190 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 1103/2187] 10.222.0.15 () {58 vars in 1153 bytes} [Mon Aug 22 16:12:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 16:12:01.259265 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-381c4d50-c1d3-4a4f-9f30-89233d13a1b4 admin admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 16:12:01.274835 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-381c4d50-c1d3-4a4f-9f30-89233d13a1b4 admin admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 16:12:01.280223 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14aa838f-581f-4077-9878-6cd16b2142e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a98b1e4431940ab8d1fc95df2858a6c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 16:12:01.280909 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14aa838f-581f-4077-9878-6cd16b2142e2 admin admin] REQUEST_METHOD: `PUT` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:12:01.281181 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14aa838f-581f-4077-9878-6cd16b2142e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:12:01.281449 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14aa838f-581f-4077-9878-6cd16b2142e2 admin admin] PATH_INFO: `/v3/projects/15b380b8d48c41ba87da5f9a329b25fe/users/352157e88c864b3dbc3f3baa8350fea9/roles/e6fc1ee3fb974125ba9b4fe55b12b45e` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:12:01.303257 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14aa838f-581f-4077-9878-6cd16b2142e2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=15b380b8d48c41ba87da5f9a329b25fe, user_id=352157e88c864b3dbc3f3baa8350fea9, role_id=e6fc1ee3fb974125ba9b4fe55b12b45e)` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 16:12:01.306045 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14aa838f-581f-4077-9878-6cd16b2142e2 admin admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 16:12:01.322579 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c0d6104-cb99-4886-b699-61a878977c60 admin admin] Validating token access rules against request {{(pid=81767) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 16:12:01.325220 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de6db080-e578-4fc3-9d71-2eb73419f9bc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '234687adad594f1aa45775a565ffbdc7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4ee0413f9ef4d4fb1f95a660babe090', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81767) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 16:12:01.326101 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de6db080-e578-4fc3-9d71-2eb73419f9bc service neutron] REQUEST_METHOD: `GET` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:12:01.326886 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de6db080-e578-4fc3-9d71-2eb73419f9bc service neutron] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:12:01.327336 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de6db080-e578-4fc3-9d71-2eb73419f9bc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:12:01.360934 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de6db080-e578-4fc3-9d71-2eb73419f9bc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 16:12:01.360934 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de6db080-e578-4fc3-9d71-2eb73419f9bc service neutron] RBAC: Authorization granted {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 16:12:01.364406 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 1085/2188] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Aug 22 16:12:01 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 136 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 16:12:01.365307 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de6db080-e578-4fc3-9d71-2eb73419f9bc service neutron] Authenticating user token {{(pid=81767) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 16:12:01.374885 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de6db080-e578-4fc3-9d71-2eb73419f9bc service neutron] Validating token access rules against request {{(pid=81767) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 16:12:01.381751 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6749976-c831-4b31-9b55-b977d11e703f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a98b1e4431940ab8d1fc95df2858a6c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81767) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 16:12:01.382353 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6749976-c831-4b31-9b55-b977d11e703f admin admin] REQUEST_METHOD: `POST` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:12:01.382648 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6749976-c831-4b31-9b55-b977d11e703f admin admin] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:12:01.382904 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6749976-c831-4b31-9b55-b977d11e703f admin admin] PATH_INFO: `/v3/projects` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:12:01.383548 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6749976-c831-4b31-9b55-b977d11e703f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 16:12:01.385459 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6749976-c831-4b31-9b55-b977d11e703f admin admin] RBAC: Authorization granted {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 16:12:01.415890 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 1104/2189] 10.222.0.15 () {60 vars in 1483 bytes} [Mon Aug 22 16:12:01 2022] PUT /identity/v3/projects/15b380b8d48c41ba87da5f9a329b25fe/users/352157e88c864b3dbc3f3baa8350fea9/roles/e6fc1ee3fb974125ba9b4fe55b12b45e => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 16:12:01.426732 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42fcb75e-219a-4290-9c9b-6e5744702621 None None] REQUEST_METHOD: `POST` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:12:01.427084 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42fcb75e-219a-4290-9c9b-6e5744702621 None None] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:12:01.427460 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42fcb75e-219a-4290-9c9b-6e5744702621 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:12:01.456641 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.auth.core [None req-42fcb75e-219a-4290-9c9b-6e5744702621 None None] MFA Rules not processed for user `352157e88c864b3dbc3f3baa8350fea9`. Rule list: `[]` (Enabled: `True`). {{(pid=81768) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 16:12:01.458040 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-42fcb75e-219a-4290-9c9b-6e5744702621 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 16:12:01.524514 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 1105/2190] 10.222.0.15 () {58 vars in 958 bytes} [Mon Aug 22 16:12:01 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 22 16:12:01.525598 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 1086/2191] 10.222.0.15 () {60 vars in 1153 bytes} [Mon Aug 22 16:12:01 2022] POST /identity/v3/projects => generated 365 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 22 16:12:01.529196 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42fcb75e-219a-4290-9c9b-6e5744702621 None None] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 16:12:01.530493 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6749976-c831-4b31-9b55-b977d11e703f admin admin] Authenticating user token {{(pid=81767) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 16:12:01.619768 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6749976-c831-4b31-9b55-b977d11e703f admin admin] Validating token access rules against request {{(pid=81767) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 16:12:01.620954 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42fcb75e-219a-4290-9c9b-6e5744702621 None None] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 16:12:01.622511 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c240c00-fd8b-4820-8dc7-44f050a9d36c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a98b1e4431940ab8d1fc95df2858a6c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81767) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 16:12:01.623126 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c240c00-fd8b-4820-8dc7-44f050a9d36c admin admin] REQUEST_METHOD: `POST` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:12:01.623436 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c240c00-fd8b-4820-8dc7-44f050a9d36c admin admin] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:12:01.623746 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c240c00-fd8b-4820-8dc7-44f050a9d36c admin admin] PATH_INFO: `/v3/projects` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:12:01.624830 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c240c00-fd8b-4820-8dc7-44f050a9d36c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 16:12:01.626814 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c240c00-fd8b-4820-8dc7-44f050a9d36c admin admin] RBAC: Authorization granted {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 16:12:01.626993 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67c018cb-4d53-49a7-82d3-41c57ba87bd7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a98b1e4431940ab8d1fc95df2858a6c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 16:12:01.628355 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c018cb-4d53-49a7-82d3-41c57ba87bd7 admin admin] REQUEST_METHOD: `POST` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:12:01.628724 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c018cb-4d53-49a7-82d3-41c57ba87bd7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:12:01.629705 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c018cb-4d53-49a7-82d3-41c57ba87bd7 admin admin] PATH_INFO: `/v3/users` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:12:01.629705 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67c018cb-4d53-49a7-82d3-41c57ba87bd7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 16:12:01.632203 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67c018cb-4d53-49a7-82d3-41c57ba87bd7 admin admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 16:12:01.766541 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 1106/2192] 10.222.0.15 () {60 vars in 1144 bytes} [Mon Aug 22 16:12:01 2022] POST /identity/v3/users => generated 283 bytes in 238 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 22 16:12:01.766866 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 1087/2193] 10.222.0.15 () {60 vars in 1153 bytes} [Mon Aug 22 16:12:01 2022] POST /identity/v3/projects => generated 379 bytes in 237 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 22 16:12:01.771017 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67c018cb-4d53-49a7-82d3-41c57ba87bd7 admin admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 16:12:01.774241 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c240c00-fd8b-4820-8dc7-44f050a9d36c admin admin] Authenticating user token {{(pid=81767) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 16:12:01.786670 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67c018cb-4d53-49a7-82d3-41c57ba87bd7 admin admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 16:12:01.863816 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c240c00-fd8b-4820-8dc7-44f050a9d36c admin admin] Validating token access rules against request {{(pid=81767) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 16:12:01.865460 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efa0c70e-a057-4574-80b8-0ec75fcc9b05 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a98b1e4431940ab8d1fc95df2858a6c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 16:12:01.866075 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efa0c70e-a057-4574-80b8-0ec75fcc9b05 admin admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:12:01.866347 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efa0c70e-a057-4574-80b8-0ec75fcc9b05 admin admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:12:01.866614 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efa0c70e-a057-4574-80b8-0ec75fcc9b05 admin admin] PATH_INFO: `/v3/roles` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:12:01.867299 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efa0c70e-a057-4574-80b8-0ec75fcc9b05 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 16:12:01.868919 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efa0c70e-a057-4574-80b8-0ec75fcc9b05 admin admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 16:12:01.869283 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-782077b3-dbff-407f-86b0-826b4787786c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a98b1e4431940ab8d1fc95df2858a6c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81767) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 16:12:01.869967 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-782077b3-dbff-407f-86b0-826b4787786c admin admin] REQUEST_METHOD: `POST` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:12:01.870348 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-782077b3-dbff-407f-86b0-826b4787786c admin admin] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:12:01.870684 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-782077b3-dbff-407f-86b0-826b4787786c admin admin] PATH_INFO: `/v3/users` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:12:01.871437 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-782077b3-dbff-407f-86b0-826b4787786c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 16:12:01.873553 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-782077b3-dbff-407f-86b0-826b4787786c admin admin] RBAC: Authorization granted {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 16:12:01.880202 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 1107/2194] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Aug 22 16:12:01 2022] GET /identity/v3/roles => generated 1420 bytes in 110 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 22 16:12:01.884415 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efa0c70e-a057-4574-80b8-0ec75fcc9b05 admin admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 16:12:01.894771 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efa0c70e-a057-4574-80b8-0ec75fcc9b05 admin admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 16:12:01.900107 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62e3b580-a668-4fb2-9af8-55bebdc423b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a98b1e4431940ab8d1fc95df2858a6c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 16:12:01.900706 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62e3b580-a668-4fb2-9af8-55bebdc423b0 admin admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:12:01.907112 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62e3b580-a668-4fb2-9af8-55bebdc423b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:12:01.907112 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62e3b580-a668-4fb2-9af8-55bebdc423b0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:12:01.907112 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62e3b580-a668-4fb2-9af8-55bebdc423b0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 16:12:01.907112 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62e3b580-a668-4fb2-9af8-55bebdc423b0 admin admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 16:12:01.915653 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 1108/2195] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Aug 22 16:12:01 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 22 16:12:01.920401 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62e3b580-a668-4fb2-9af8-55bebdc423b0 admin admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 16:12:01.931064 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62e3b580-a668-4fb2-9af8-55bebdc423b0 admin admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 16:12:01.936177 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-055952a9-4c12-475b-9898-cf217bceb308 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a98b1e4431940ab8d1fc95df2858a6c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 16:12:01.936835 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-055952a9-4c12-475b-9898-cf217bceb308 admin admin] REQUEST_METHOD: `PUT` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:12:01.937127 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-055952a9-4c12-475b-9898-cf217bceb308 admin admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:12:01.937372 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-055952a9-4c12-475b-9898-cf217bceb308 admin admin] PATH_INFO: `/v3/projects/169c9dcaffbe4ede89fbf23768666ad2/users/371289161199421ea28a7688ef40d5ad/roles/e6fc1ee3fb974125ba9b4fe55b12b45e` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:12:01.951223 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG dbcounter [-] [81767] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81767) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 16:12:01.978912 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-055952a9-4c12-475b-9898-cf217bceb308 admin admin] RBAC: Authorizing `identity:create_grant(project_id=169c9dcaffbe4ede89fbf23768666ad2, user_id=371289161199421ea28a7688ef40d5ad, role_id=e6fc1ee3fb974125ba9b4fe55b12b45e)` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 16:12:01.987507 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-055952a9-4c12-475b-9898-cf217bceb308 admin admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 16:12:02.008396 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 1088/2196] 10.222.0.15 () {60 vars in 1144 bytes} [Mon Aug 22 16:12:01 2022] POST /identity/v3/users => generated 296 bytes in 235 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 22 16:12:02.008396 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-782077b3-dbff-407f-86b0-826b4787786c admin admin] Authenticating user token {{(pid=81767) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 16:12:02.009096 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.fernet_utils [None req-782077b3-dbff-407f-86b0-826b4787786c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81767) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 16:12:02.033322 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-782077b3-dbff-407f-86b0-826b4787786c admin admin] Validating token access rules against request {{(pid=81767) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 16:12:02.033322 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-666d663e-8a5f-4874-8c20-006ea536402f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a98b1e4431940ab8d1fc95df2858a6c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81767) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 16:12:02.033628 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-666d663e-8a5f-4874-8c20-006ea536402f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:12:02.033873 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-666d663e-8a5f-4874-8c20-006ea536402f admin admin] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:12:02.034159 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-666d663e-8a5f-4874-8c20-006ea536402f admin admin] PATH_INFO: `/v3/projects/4ca6ef9827884b04ad9117c11756fe00` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:12:02.035416 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-666d663e-8a5f-4874-8c20-006ea536402f admin admin] RBAC: Authorizing `identity:delete_project(project_id=4ca6ef9827884b04ad9117c11756fe00)` {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 16:12:02.037394 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-666d663e-8a5f-4874-8c20-006ea536402f admin admin] RBAC: Authorization granted {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 16:12:02.064723 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 1109/2197] 10.222.0.15 () {60 vars in 1483 bytes} [Mon Aug 22 16:12:01 2022] PUT /identity/v3/projects/169c9dcaffbe4ede89fbf23768666ad2/users/371289161199421ea28a7688ef40d5ad/roles/e6fc1ee3fb974125ba9b4fe55b12b45e => generated 0 bytes in 143 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 22 16:12:02.064723 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-055952a9-4c12-475b-9898-cf217bceb308 admin admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 16:12:02.139987 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 16:12:02.150699 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-055952a9-4c12-475b-9898-cf217bceb308 admin admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 16:12:02.155542 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db976fce-e76b-4efa-a636-8a64eee773f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a98b1e4431940ab8d1fc95df2858a6c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 16:12:02.156148 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db976fce-e76b-4efa-a636-8a64eee773f0 admin admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:12:02.156425 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db976fce-e76b-4efa-a636-8a64eee773f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:12:02.156697 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db976fce-e76b-4efa-a636-8a64eee773f0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:12:02.157372 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db976fce-e76b-4efa-a636-8a64eee773f0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 16:12:02.159554 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db976fce-e76b-4efa-a636-8a64eee773f0 admin admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 16:12:02.172987 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 1110/2198] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Aug 22 16:12:02 2022] GET /identity/v3/roles => generated 1420 bytes in 106 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 22 16:12:02.172987 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e53a967-20c8-4f9a-afb4-14203b20a83d None None] REQUEST_METHOD: `POST` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:12:02.172987 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e53a967-20c8-4f9a-afb4-14203b20a83d None None] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:12:02.172987 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e53a967-20c8-4f9a-afb4-14203b20a83d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:12:02.199937 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.notifications [None req-666d663e-8a5f-4874-8c20-006ea536402f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4ca6ef9827884b04ad9117c11756fe00', 'request_id': 'req-666d663e-8a5f-4874-8c20-006ea536402f'} {{(pid=81767) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 16:12:02.206102 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.auth.core [None req-2e53a967-20c8-4f9a-afb4-14203b20a83d None None] MFA Rules not processed for user `371289161199421ea28a7688ef40d5ad`. Rule list: `[]` (Enabled: `True`). {{(pid=81768) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 16:12:02.211294 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.notifications [None req-666d663e-8a5f-4874-8c20-006ea536402f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4ca6ef9827884b04ad9117c11756fe00', 'request_id': 'req-666d663e-8a5f-4874-8c20-006ea536402f'} {{(pid=81767) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 16:12:02.211408 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-2e53a967-20c8-4f9a-afb4-14203b20a83d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 16:12:02.286848 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 1111/2199] 10.222.0.15 () {58 vars in 958 bytes} [Mon Aug 22 16:12:02 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 16:12:02.287690 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e53a967-20c8-4f9a-afb4-14203b20a83d None None] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 16:12:02.291085 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.notifications [None req-666d663e-8a5f-4874-8c20-006ea536402f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4ca6ef9827884b04ad9117c11756fe00', 'request_id': 'req-666d663e-8a5f-4874-8c20-006ea536402f'} {{(pid=81767) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 16:12:02.304400 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.notifications [None req-666d663e-8a5f-4874-8c20-006ea536402f admin admin] The token cache is being invalidate because project 4ca6ef9827884b04ad9117c11756fe00 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81767) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 22 16:12:02.304691 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.notifications [None req-666d663e-8a5f-4874-8c20-006ea536402f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81767) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 16:12:02.305370 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.notifications [None req-666d663e-8a5f-4874-8c20-006ea536402f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81767) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 16:12:02.306883 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 1089/2200] 10.222.0.15 () {58 vars in 1233 bytes} [Mon Aug 22 16:12:02 2022] DELETE /identity/v3/projects/4ca6ef9827884b04ad9117c11756fe00 => generated 0 bytes in 301 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 16:12:02.307675 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-666d663e-8a5f-4874-8c20-006ea536402f admin admin] Authenticating user token {{(pid=81767) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 16:12:02.308965 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.fernet_utils [None req-666d663e-8a5f-4874-8c20-006ea536402f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81767) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 16:12:02.361455 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e53a967-20c8-4f9a-afb4-14203b20a83d None None] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 16:12:02.367227 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8417400-cf2e-44cd-937c-2c1067582f33 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a98b1e4431940ab8d1fc95df2858a6c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 16:12:02.367865 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8417400-cf2e-44cd-937c-2c1067582f33 admin admin] REQUEST_METHOD: `PUT` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:12:02.368144 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8417400-cf2e-44cd-937c-2c1067582f33 admin admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:12:02.368415 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8417400-cf2e-44cd-937c-2c1067582f33 admin admin] PATH_INFO: `/v3/projects/3af9d400743147ed957a276423b43f39/users/81ec415f5f004eaa959c87f13933df51/roles/9d9848cfd22e4b089db443c350e56670` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:12:02.386955 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-666d663e-8a5f-4874-8c20-006ea536402f admin admin] Validating token access rules against request {{(pid=81767) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 16:12:02.387322 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8417400-cf2e-44cd-937c-2c1067582f33 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3af9d400743147ed957a276423b43f39, user_id=81ec415f5f004eaa959c87f13933df51, role_id=9d9848cfd22e4b089db443c350e56670)` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 16:12:02.390502 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8417400-cf2e-44cd-937c-2c1067582f33 admin admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 16:12:02.391937 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6556d8dd-0d8d-471b-9a5a-eadd8b92f5ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a98b1e4431940ab8d1fc95df2858a6c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81767) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 16:12:02.392613 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6556d8dd-0d8d-471b-9a5a-eadd8b92f5ce admin admin] REQUEST_METHOD: `POST` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:12:02.392894 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6556d8dd-0d8d-471b-9a5a-eadd8b92f5ce admin admin] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:12:02.393169 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6556d8dd-0d8d-471b-9a5a-eadd8b92f5ce admin admin] PATH_INFO: `/v3/projects` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:12:02.393801 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6556d8dd-0d8d-471b-9a5a-eadd8b92f5ce admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 16:12:02.395754 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6556d8dd-0d8d-471b-9a5a-eadd8b92f5ce admin admin] RBAC: Authorization granted {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 16:12:02.464946 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 1112/2201] 10.222.0.15 () {60 vars in 1483 bytes} [Mon Aug 22 16:12:02 2022] PUT /identity/v3/projects/3af9d400743147ed957a276423b43f39/users/81ec415f5f004eaa959c87f13933df51/roles/9d9848cfd22e4b089db443c350e56670 => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 16:12:02.465922 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8417400-cf2e-44cd-937c-2c1067582f33 admin admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 16:12:02.467561 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-c8417400-cf2e-44cd-937c-2c1067582f33 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 16:12:02.495510 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 1090/2202] 10.222.0.15 () {60 vars in 1153 bytes} [Mon Aug 22 16:12:02 2022] POST /identity/v3/projects => generated 367 bytes in 188 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 22 16:12:02.496352 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6556d8dd-0d8d-471b-9a5a-eadd8b92f5ce admin admin] Authenticating user token {{(pid=81767) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 16:12:02.498094 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.fernet_utils [None req-6556d8dd-0d8d-471b-9a5a-eadd8b92f5ce admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81767) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 16:12:02.536797 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8417400-cf2e-44cd-937c-2c1067582f33 admin admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 16:12:02.565617 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6556d8dd-0d8d-471b-9a5a-eadd8b92f5ce admin admin] Validating token access rules against request {{(pid=81767) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 16:12:02.570273 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c50f781c-25cf-4a4c-b82b-51284dbb0a15 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a98b1e4431940ab8d1fc95df2858a6c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81767) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 16:12:02.570882 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c50f781c-25cf-4a4c-b82b-51284dbb0a15 admin admin] REQUEST_METHOD: `GET` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:12:02.571158 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c50f781c-25cf-4a4c-b82b-51284dbb0a15 admin admin] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:12:02.571488 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c50f781c-25cf-4a4c-b82b-51284dbb0a15 admin admin] PATH_INFO: `/v3/roles` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:12:02.572191 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c50f781c-25cf-4a4c-b82b-51284dbb0a15 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 16:12:02.573789 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c50f781c-25cf-4a4c-b82b-51284dbb0a15 admin admin] RBAC: Authorization granted {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 16:12:02.585956 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 1091/2203] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Aug 22 16:12:02 2022] GET /identity/v3/roles => generated 1420 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 22 16:12:02.585956 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c50f781c-25cf-4a4c-b82b-51284dbb0a15 admin admin] Authenticating user token {{(pid=81767) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 16:12:02.595788 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93923316-f188-4dbe-ac68-94dc7d0a9472 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a98b1e4431940ab8d1fc95df2858a6c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 16:12:02.597059 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93923316-f188-4dbe-ac68-94dc7d0a9472 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:12:02.597345 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93923316-f188-4dbe-ac68-94dc7d0a9472 admin admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:12:02.597843 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93923316-f188-4dbe-ac68-94dc7d0a9472 admin admin] PATH_INFO: `/v3/users/08d8e4a8e6fa4cccb860ba64f4c0eacd` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:12:02.602458 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93923316-f188-4dbe-ac68-94dc7d0a9472 admin admin] RBAC: Authorizing `identity:delete_user(user_id=08d8e4a8e6fa4cccb860ba64f4c0eacd)` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 16:12:02.602584 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c50f781c-25cf-4a4c-b82b-51284dbb0a15 admin admin] Validating token access rules against request {{(pid=81767) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 16:12:02.602667 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93923316-f188-4dbe-ac68-94dc7d0a9472 admin admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 16:12:02.604713 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20cf6a3b-65a2-494c-b390-1b3972e24fc1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a98b1e4431940ab8d1fc95df2858a6c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81767) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 16:12:02.605240 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20cf6a3b-65a2-494c-b390-1b3972e24fc1 admin admin] REQUEST_METHOD: `POST` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:12:02.605520 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20cf6a3b-65a2-494c-b390-1b3972e24fc1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:12:02.605808 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20cf6a3b-65a2-494c-b390-1b3972e24fc1 admin admin] PATH_INFO: `/v3/users` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:12:02.606420 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20cf6a3b-65a2-494c-b390-1b3972e24fc1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 16:12:02.608547 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20cf6a3b-65a2-494c-b390-1b3972e24fc1 admin admin] RBAC: Authorization granted {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 16:12:02.650513 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG dbcounter [-] [81767] Writing DB stats keystone:SELECT=91,keystone:DELETE=5,keystone:INSERT=4 {{(pid=81767) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 16:12:02.726996 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 1092/2204] 10.222.0.15 () {60 vars in 1144 bytes} [Mon Aug 22 16:12:02 2022] POST /identity/v3/users => generated 290 bytes in 139 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 22 16:12:02.726996 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20cf6a3b-65a2-494c-b390-1b3972e24fc1 admin admin] Authenticating user token {{(pid=81767) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 16:12:02.743959 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20cf6a3b-65a2-494c-b390-1b3972e24fc1 admin admin] Validating token access rules against request {{(pid=81767) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 16:12:02.743959 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f467322-2d70-4192-9326-1616de00744d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a98b1e4431940ab8d1fc95df2858a6c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81767) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 16:12:02.743959 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f467322-2d70-4192-9326-1616de00744d admin admin] REQUEST_METHOD: `PUT` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:12:02.743959 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f467322-2d70-4192-9326-1616de00744d admin admin] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:12:02.744284 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f467322-2d70-4192-9326-1616de00744d admin admin] PATH_INFO: `/v3/projects/3af9d400743147ed957a276423b43f39/users/81ec415f5f004eaa959c87f13933df51/roles/9d9848cfd22e4b089db443c350e56670` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:12:02.745017 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f467322-2d70-4192-9326-1616de00744d admin admin] RBAC: Authorizing `identity:create_grant(project_id=3af9d400743147ed957a276423b43f39, user_id=81ec415f5f004eaa959c87f13933df51, role_id=9d9848cfd22e4b089db443c350e56670)` {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 16:12:02.748013 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f467322-2d70-4192-9326-1616de00744d admin admin] RBAC: Authorization granted {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 16:12:02.760839 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 1093/2205] 10.222.0.15 () {60 vars in 1483 bytes} [Mon Aug 22 16:12:02 2022] PUT /identity/v3/projects/3af9d400743147ed957a276423b43f39/users/81ec415f5f004eaa959c87f13933df51/roles/9d9848cfd22e4b089db443c350e56670 => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 16:12:02.761663 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f467322-2d70-4192-9326-1616de00744d admin admin] Authenticating user token {{(pid=81767) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 16:12:02.839558 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f467322-2d70-4192-9326-1616de00744d admin admin] Validating token access rules against request {{(pid=81767) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 16:12:02.845883 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b8a2549-8502-4d1d-a5eb-c8462233a746 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a98b1e4431940ab8d1fc95df2858a6c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81767) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 16:12:02.845883 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b8a2549-8502-4d1d-a5eb-c8462233a746 admin admin] REQUEST_METHOD: `GET` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:12:02.845883 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b8a2549-8502-4d1d-a5eb-c8462233a746 admin admin] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:12:02.845883 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b8a2549-8502-4d1d-a5eb-c8462233a746 admin admin] PATH_INFO: `/v3/roles` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:12:02.846298 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b8a2549-8502-4d1d-a5eb-c8462233a746 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 16:12:02.848026 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b8a2549-8502-4d1d-a5eb-c8462233a746 admin admin] RBAC: Authorization granted {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 16:12:02.857996 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 1094/2206] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Aug 22 16:12:02 2022] GET /identity/v3/roles => generated 1420 bytes in 97 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 22 16:12:02.859198 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1491180-8c55-4beb-bb8d-d3356529f7c5 None None] REQUEST_METHOD: `POST` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:12:02.859527 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1491180-8c55-4beb-bb8d-d3356529f7c5 None None] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:12:02.859795 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1491180-8c55-4beb-bb8d-d3356529f7c5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:12:02.864037 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 16:12:02.888715 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.auth.core [None req-e1491180-8c55-4beb-bb8d-d3356529f7c5 None None] MFA Rules not processed for user `81ec415f5f004eaa959c87f13933df51`. Rule list: `[]` (Enabled: `True`). {{(pid=81767) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 16:12:02.890069 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.fernet_utils [None req-e1491180-8c55-4beb-bb8d-d3356529f7c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81767) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 16:12:02.924536 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-93923316-f188-4dbe-ac68-94dc7d0a9472 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '08d8e4a8e6fa4cccb860ba64f4c0eacd'} {{(pid=81768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 16:12:02.926026 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-93923316-f188-4dbe-ac68-94dc7d0a9472 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '08d8e4a8e6fa4cccb860ba64f4c0eacd'} {{(pid=81768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 16:12:02.926697 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-93923316-f188-4dbe-ac68-94dc7d0a9472 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '08d8e4a8e6fa4cccb860ba64f4c0eacd'} {{(pid=81768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 16:12:02.934861 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-93923316-f188-4dbe-ac68-94dc7d0a9472 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '08d8e4a8e6fa4cccb860ba64f4c0eacd'} {{(pid=81768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 16:12:02.991112 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 1095/2207] 10.222.0.15 () {58 vars in 958 bytes} [Mon Aug 22 16:12:02 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 16:12:02.992155 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1491180-8c55-4beb-bb8d-d3356529f7c5 None None] Authenticating user token {{(pid=81767) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 16:12:02.994465 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.fernet_utils [None req-e1491180-8c55-4beb-bb8d-d3356529f7c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81767) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 16:12:03.007375 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-93923316-f188-4dbe-ac68-94dc7d0a9472 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '08d8e4a8e6fa4cccb860ba64f4c0eacd'} {{(pid=81768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 16:12:03.009692 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1491180-8c55-4beb-bb8d-d3356529f7c5 None None] Validating token access rules against request {{(pid=81767) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 16:12:03.017824 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4724cb6-a2a7-4300-8560-4b5dd5edc52b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a98b1e4431940ab8d1fc95df2858a6c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81767) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 16:12:03.017824 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4724cb6-a2a7-4300-8560-4b5dd5edc52b admin admin] REQUEST_METHOD: `PUT` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:12:03.017824 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4724cb6-a2a7-4300-8560-4b5dd5edc52b admin admin] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:12:03.017824 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4724cb6-a2a7-4300-8560-4b5dd5edc52b admin admin] PATH_INFO: `/v3/projects/42eb15f24a664ac997543e7df9fb0205/users/622e68277ba74bd98a0fce9a20e6991b/roles/9d9848cfd22e4b089db443c350e56670` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:12:03.036089 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 1113/2208] 10.222.0.15 () {58 vars in 1224 bytes} [Mon Aug 22 16:12:02 2022] DELETE /identity/v3/users/08d8e4a8e6fa4cccb860ba64f4c0eacd => generated 0 bytes in 570 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 16:12:03.037292 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93923316-f188-4dbe-ac68-94dc7d0a9472 admin admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 16:12:03.038678 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-93923316-f188-4dbe-ac68-94dc7d0a9472 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 16:12:03.061990 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4724cb6-a2a7-4300-8560-4b5dd5edc52b admin admin] RBAC: Authorizing `identity:create_grant(project_id=42eb15f24a664ac997543e7df9fb0205, user_id=622e68277ba74bd98a0fce9a20e6991b, role_id=9d9848cfd22e4b089db443c350e56670)` {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 16:12:03.064709 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4724cb6-a2a7-4300-8560-4b5dd5edc52b admin admin] RBAC: Authorization granted {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 16:12:03.122431 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93923316-f188-4dbe-ac68-94dc7d0a9472 admin admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 16:12:03.128005 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b524b492-363a-4cfc-bfd4-55aaa7fe65da service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79706d480f864998b9f8594be407ceab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4ee0413f9ef4d4fb1f95a660babe090', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 16:12:03.129336 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b524b492-363a-4cfc-bfd4-55aaa7fe65da service cinder] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:12:03.129676 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b524b492-363a-4cfc-bfd4-55aaa7fe65da service cinder] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:12:03.129948 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b524b492-363a-4cfc-bfd4-55aaa7fe65da service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:12:03.159375 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 1096/2209] 10.222.0.15 () {60 vars in 1483 bytes} [Mon Aug 22 16:12:02 2022] PUT /identity/v3/projects/42eb15f24a664ac997543e7df9fb0205/users/622e68277ba74bd98a0fce9a20e6991b/roles/9d9848cfd22e4b089db443c350e56670 => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 16:12:03.163125 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4724cb6-a2a7-4300-8560-4b5dd5edc52b admin admin] Authenticating user token {{(pid=81767) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 16:12:03.234462 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b524b492-363a-4cfc-bfd4-55aaa7fe65da service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 16:12:03.236658 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b524b492-363a-4cfc-bfd4-55aaa7fe65da service cinder] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 16:12:03.261019 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4724cb6-a2a7-4300-8560-4b5dd5edc52b admin admin] Validating token access rules against request {{(pid=81767) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 16:12:03.269412 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 1114/2210] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Aug 22 16:12:03 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 233 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 16:12:03.286494 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f5a8a93-f87c-4cb6-ac2a-256c439827e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a98b1e4431940ab8d1fc95df2858a6c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81767) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 16:12:03.288028 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f5a8a93-f87c-4cb6-ac2a-256c439827e9 admin admin] REQUEST_METHOD: `GET` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:12:03.290303 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f5a8a93-f87c-4cb6-ac2a-256c439827e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:12:03.290766 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f5a8a93-f87c-4cb6-ac2a-256c439827e9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:12:03.291815 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f5a8a93-f87c-4cb6-ac2a-256c439827e9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 16:12:03.294673 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f5a8a93-f87c-4cb6-ac2a-256c439827e9 admin admin] RBAC: Authorization granted {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 16:12:03.315586 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 1097/2211] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Aug 22 16:12:03 2022] GET /identity/v3/roles => generated 1420 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 22 16:12:03.319628 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b524b492-363a-4cfc-bfd4-55aaa7fe65da service cinder] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 16:12:03.429872 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b524b492-363a-4cfc-bfd4-55aaa7fe65da service cinder] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 16:12:03.435045 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66095f6f-f034-42ef-8447-131a97af1211 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a98b1e4431940ab8d1fc95df2858a6c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 16:12:03.450010 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66095f6f-f034-42ef-8447-131a97af1211 admin admin] REQUEST_METHOD: `PUT` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:12:03.450010 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66095f6f-f034-42ef-8447-131a97af1211 admin admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:12:03.450010 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66095f6f-f034-42ef-8447-131a97af1211 admin admin] PATH_INFO: `/v3/projects/42eb15f24a664ac997543e7df9fb0205/users/622e68277ba74bd98a0fce9a20e6991b/roles/9d9848cfd22e4b089db443c350e56670` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:12:03.450010 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66095f6f-f034-42ef-8447-131a97af1211 admin admin] RBAC: Authorizing `identity:create_grant(project_id=42eb15f24a664ac997543e7df9fb0205, user_id=622e68277ba74bd98a0fce9a20e6991b, role_id=9d9848cfd22e4b089db443c350e56670)` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 16:12:03.450010 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66095f6f-f034-42ef-8447-131a97af1211 admin admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 16:12:03.468132 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 1115/2212] 10.222.0.15 () {60 vars in 1483 bytes} [Mon Aug 22 16:12:03 2022] PUT /identity/v3/projects/42eb15f24a664ac997543e7df9fb0205/users/622e68277ba74bd98a0fce9a20e6991b/roles/9d9848cfd22e4b089db443c350e56670 => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 16:12:03.479133 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f6afbae-111a-410b-a4ac-873b0f907781 None None] REQUEST_METHOD: `POST` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:12:03.479598 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f6afbae-111a-410b-a4ac-873b0f907781 None None] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:12:03.480047 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f6afbae-111a-410b-a4ac-873b0f907781 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:12:03.519368 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG dbcounter [-] [81767] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81767) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 16:12:03.526269 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.auth.core [None req-0f6afbae-111a-410b-a4ac-873b0f907781 None None] MFA Rules not processed for user `622e68277ba74bd98a0fce9a20e6991b`. Rule list: `[]` (Enabled: `True`). {{(pid=81767) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 16:12:03.527840 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.fernet_utils [None req-0f6afbae-111a-410b-a4ac-873b0f907781 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81767) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 16:12:03.639729 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bfc01b4-69d4-4bc3-8a01-615c9da451cf None None] REQUEST_METHOD: `POST` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:12:03.640010 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bfc01b4-69d4-4bc3-8a01-615c9da451cf None None] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:12:03.640992 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bfc01b4-69d4-4bc3-8a01-615c9da451cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:12:03.645735 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 16:12:03.674988 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.auth.core [None req-0bfc01b4-69d4-4bc3-8a01-615c9da451cf None None] MFA Rules not processed for user `058e76354dce412cb78b2875c88f2791`. Rule list: `[]` (Enabled: `True`). {{(pid=81768) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 16:12:03.677475 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-0bfc01b4-69d4-4bc3-8a01-615c9da451cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 16:12:03.703602 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 1098/2213] 10.222.0.15 () {58 vars in 958 bytes} [Mon Aug 22 16:12:03 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 221 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 22 16:12:03.723169 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f6afbae-111a-410b-a4ac-873b0f907781 None None] Authenticating user token {{(pid=81767) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 16:12:03.724808 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.fernet_utils [None req-0f6afbae-111a-410b-a4ac-873b0f907781 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81767) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 16:12:03.764136 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 1116/2214] 10.222.0.15 () {58 vars in 1009 bytes} [Mon Aug 22 16:12:03 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 16:12:03.765102 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bfc01b4-69d4-4bc3-8a01-615c9da451cf None None] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 16:12:03.766820 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-0bfc01b4-69d4-4bc3-8a01-615c9da451cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 16:12:03.839539 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f6afbae-111a-410b-a4ac-873b0f907781 None None] Validating token access rules against request {{(pid=81767) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 16:12:03.840722 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bfc01b4-69d4-4bc3-8a01-615c9da451cf None None] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 16:12:03.844442 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-217d65a1-4827-467b-9424-6a53aa8c5f24 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79706d480f864998b9f8594be407ceab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4ee0413f9ef4d4fb1f95a660babe090', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 16:12:03.844589 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0b9e67c-4ef8-47bf-9558-902a1847623f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a98b1e4431940ab8d1fc95df2858a6c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81767) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 16:12:03.845088 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-217d65a1-4827-467b-9424-6a53aa8c5f24 service cinder] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:12:03.845202 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0b9e67c-4ef8-47bf-9558-902a1847623f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:12:03.845476 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-217d65a1-4827-467b-9424-6a53aa8c5f24 service cinder] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:12:03.845589 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0b9e67c-4ef8-47bf-9558-902a1847623f admin admin] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:12:03.845681 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-217d65a1-4827-467b-9424-6a53aa8c5f24 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:12:03.845821 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0b9e67c-4ef8-47bf-9558-902a1847623f admin admin] PATH_INFO: `/v3/projects/a1cf040c8a984da28c85d8d3cea57530` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:12:03.847150 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0b9e67c-4ef8-47bf-9558-902a1847623f admin admin] RBAC: Authorizing `identity:delete_project(project_id=a1cf040c8a984da28c85d8d3cea57530)` {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 16:12:03.849464 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0b9e67c-4ef8-47bf-9558-902a1847623f admin admin] RBAC: Authorization granted {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 16:12:03.856218 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-217d65a1-4827-467b-9424-6a53aa8c5f24 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 16:12:03.858311 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-217d65a1-4827-467b-9424-6a53aa8c5f24 service cinder] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 16:12:03.871773 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 1117/2215] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Aug 22 16:12:03 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 16:12:03.872734 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-217d65a1-4827-467b-9424-6a53aa8c5f24 service cinder] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 16:12:03.874884 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-217d65a1-4827-467b-9424-6a53aa8c5f24 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 16:12:04.050128 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-217d65a1-4827-467b-9424-6a53aa8c5f24 service cinder] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 16:12:04.050128 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d7b9a75-57e6-40bb-a5c6-2f102dc91918 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a09d128dee9438283633e51f4834643', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4ee0413f9ef4d4fb1f95a660babe090', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 16:12:04.050128 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d7b9a75-57e6-40bb-a5c6-2f102dc91918 service swift] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:12:04.050128 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d7b9a75-57e6-40bb-a5c6-2f102dc91918 service swift] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:12:04.052361 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d7b9a75-57e6-40bb-a5c6-2f102dc91918 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:12:04.052361 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d7b9a75-57e6-40bb-a5c6-2f102dc91918 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 16:12:04.052361 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d7b9a75-57e6-40bb-a5c6-2f102dc91918 service swift] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 16:12:04.052361 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 1118/2216] 10.222.0.15 () {60 vars in 1384 bytes} [Mon Aug 22 16:12:03 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 22 16:12:04.460409 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.notifications [None req-f0b9e67c-4ef8-47bf-9558-902a1847623f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a1cf040c8a984da28c85d8d3cea57530', 'request_id': 'req-f0b9e67c-4ef8-47bf-9558-902a1847623f'} {{(pid=81767) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 16:12:04.484097 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.notifications [None req-f0b9e67c-4ef8-47bf-9558-902a1847623f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a1cf040c8a984da28c85d8d3cea57530', 'request_id': 'req-f0b9e67c-4ef8-47bf-9558-902a1847623f'} {{(pid=81767) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 16:12:04.570005 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.notifications [None req-f0b9e67c-4ef8-47bf-9558-902a1847623f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a1cf040c8a984da28c85d8d3cea57530', 'request_id': 'req-f0b9e67c-4ef8-47bf-9558-902a1847623f'} {{(pid=81767) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 16:12:04.599482 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.notifications [None req-f0b9e67c-4ef8-47bf-9558-902a1847623f admin admin] The token cache is being invalidate because project a1cf040c8a984da28c85d8d3cea57530 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81767) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 22 16:12:04.601472 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.notifications [None req-f0b9e67c-4ef8-47bf-9558-902a1847623f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81767) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 16:12:04.603655 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.notifications [None req-f0b9e67c-4ef8-47bf-9558-902a1847623f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81767) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 16:12:04.614066 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 1099/2217] 10.222.0.15 () {58 vars in 1233 bytes} [Mon Aug 22 16:12:03 2022] DELETE /identity/v3/projects/a1cf040c8a984da28c85d8d3cea57530 => generated 0 bytes in 892 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 16:12:04.620393 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27eb4f35-f977-4ed1-b380-22290e7848a6 None None] REQUEST_METHOD: `POST` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:12:04.621035 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27eb4f35-f977-4ed1-b380-22290e7848a6 None None] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:12:04.621035 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27eb4f35-f977-4ed1-b380-22290e7848a6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:12:04.679544 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG dbcounter [-] [81767] Writing DB stats keystone:INSERT=3,keystone:SELECT=97 {{(pid=81767) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 16:12:04.699443 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.auth.core [None req-27eb4f35-f977-4ed1-b380-22290e7848a6 None None] MFA Rules not processed for user `6b2199ae0fb0480aa917ada391f5ec54`. Rule list: `[]` (Enabled: `True`). {{(pid=81768) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 16:12:04.701566 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-27eb4f35-f977-4ed1-b380-22290e7848a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 16:12:04.834843 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 1119/2218] 10.222.0.15 () {58 vars in 958 bytes} [Mon Aug 22 16:12:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 216 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 16:12:04.842956 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70eb573f-e5b1-4510-b162-a4d8a22b7a97 None None] REQUEST_METHOD: `POST` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:12:04.843531 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70eb573f-e5b1-4510-b162-a4d8a22b7a97 None None] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:12:04.843971 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70eb573f-e5b1-4510-b162-a4d8a22b7a97 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:12:04.877668 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.auth.core [None req-70eb573f-e5b1-4510-b162-a4d8a22b7a97 None None] MFA Rules not processed for user `6b2199ae0fb0480aa917ada391f5ec54`. Rule list: `[]` (Enabled: `True`). {{(pid=81767) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 16:12:04.879841 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.fernet_utils [None req-70eb573f-e5b1-4510-b162-a4d8a22b7a97 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81767) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 16:12:04.893768 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 1100/2219] 10.222.0.15 () {58 vars in 958 bytes} [Mon Aug 22 16:12:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 51 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 22 16:12:04.897732 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27eb4f35-f977-4ed1-b380-22290e7848a6 None None] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 16:12:04.911842 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27eb4f35-f977-4ed1-b380-22290e7848a6 None None] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 16:12:04.918685 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f775341-4cc0-4fb0-8794-649a7ac33c37 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a98b1e4431940ab8d1fc95df2858a6c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 16:12:04.919497 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f775341-4cc0-4fb0-8794-649a7ac33c37 admin admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:12:04.919780 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f775341-4cc0-4fb0-8794-649a7ac33c37 admin admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:12:04.920053 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f775341-4cc0-4fb0-8794-649a7ac33c37 admin admin] PATH_INFO: `/v3/domains` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:12:04.921709 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f775341-4cc0-4fb0-8794-649a7ac33c37 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81768) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 16:12:04.922170 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f775341-4cc0-4fb0-8794-649a7ac33c37 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 16:12:04.924784 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f775341-4cc0-4fb0-8794-649a7ac33c37 admin admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 16:12:04.940726 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 1120/2220] 10.222.0.15 () {58 vars in 1153 bytes} [Mon Aug 22 16:12:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 22 16:12:04.944909 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70eb573f-e5b1-4510-b162-a4d8a22b7a97 None None] Authenticating user token {{(pid=81767) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 16:12:04.966532 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70eb573f-e5b1-4510-b162-a4d8a22b7a97 None None] Validating token access rules against request {{(pid=81767) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 16:12:04.978520 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b863e0b-5d6b-46bd-a3c2-fd0220f7ebc0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a98b1e4431940ab8d1fc95df2858a6c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81767) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 16:12:04.979905 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b863e0b-5d6b-46bd-a3c2-fd0220f7ebc0 admin admin] REQUEST_METHOD: `POST` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:12:04.980205 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b863e0b-5d6b-46bd-a3c2-fd0220f7ebc0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:12:04.994509 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b863e0b-5d6b-46bd-a3c2-fd0220f7ebc0 admin admin] PATH_INFO: `/v3/projects` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:12:04.996732 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b863e0b-5d6b-46bd-a3c2-fd0220f7ebc0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 16:12:05.003945 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b863e0b-5d6b-46bd-a3c2-fd0220f7ebc0 admin admin] RBAC: Authorization granted {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 16:12:05.117292 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 16:12:05.165884 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 1101/2221] 10.222.0.15 () {60 vars in 1153 bytes} [Mon Aug 22 16:12:04 2022] POST /identity/v3/projects => generated 381 bytes in 221 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 22 16:12:05.173546 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f775341-4cc0-4fb0-8794-649a7ac33c37 admin admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 16:12:05.188933 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f775341-4cc0-4fb0-8794-649a7ac33c37 admin admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 16:12:05.291163 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG dbcounter [-] [81767] Writing DB stats keystone:SELECT=93,keystone:DELETE=7 {{(pid=81767) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 16:12:05.319052 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6f6d20d-c056-4059-be43-5ee457f174cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a98b1e4431940ab8d1fc95df2858a6c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 16:12:05.320948 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f6d20d-c056-4059-be43-5ee457f174cf admin admin] REQUEST_METHOD: `POST` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:12:05.320948 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f6d20d-c056-4059-be43-5ee457f174cf admin admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:12:05.320948 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f6d20d-c056-4059-be43-5ee457f174cf admin admin] PATH_INFO: `/v3/users` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:12:05.321235 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6f6d20d-c056-4059-be43-5ee457f174cf admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 16:12:05.323515 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6f6d20d-c056-4059-be43-5ee457f174cf admin admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 16:12:05.388356 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b863e0b-5d6b-46bd-a3c2-fd0220f7ebc0 admin admin] Authenticating user token {{(pid=81767) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 16:12:05.390837 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.fernet_utils [None req-0b863e0b-5d6b-46bd-a3c2-fd0220f7ebc0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81767) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 16:12:05.421781 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 1121/2222] 10.222.0.15 () {60 vars in 1144 bytes} [Mon Aug 22 16:12:05 2022] POST /identity/v3/users => generated 291 bytes in 248 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 22 16:12:05.427433 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6f6d20d-c056-4059-be43-5ee457f174cf admin admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 16:12:05.438528 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6f6d20d-c056-4059-be43-5ee457f174cf admin admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 16:12:05.444366 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83eda01c-ad9a-4343-bb3f-5711c8c362f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a98b1e4431940ab8d1fc95df2858a6c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 16:12:05.444873 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83eda01c-ad9a-4343-bb3f-5711c8c362f9 admin admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:12:05.445211 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83eda01c-ad9a-4343-bb3f-5711c8c362f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:12:05.445626 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83eda01c-ad9a-4343-bb3f-5711c8c362f9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:12:05.446347 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83eda01c-ad9a-4343-bb3f-5711c8c362f9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 16:12:05.459262 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83eda01c-ad9a-4343-bb3f-5711c8c362f9 admin admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 16:12:05.460764 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 1122/2223] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Aug 22 16:12:05 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 22 16:12:05.465079 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b863e0b-5d6b-46bd-a3c2-fd0220f7ebc0 admin admin] Validating token access rules against request {{(pid=81767) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 16:12:05.468682 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83eda01c-ad9a-4343-bb3f-5711c8c362f9 admin admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 16:12:05.469052 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71769442-f157-436b-b328-a072185a5d63 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79706d480f864998b9f8594be407ceab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4ee0413f9ef4d4fb1f95a660babe090', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81767) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 16:12:05.469715 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71769442-f157-436b-b328-a072185a5d63 service cinder] REQUEST_METHOD: `GET` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:12:05.469945 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71769442-f157-436b-b328-a072185a5d63 service cinder] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:12:05.470384 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71769442-f157-436b-b328-a072185a5d63 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:12:05.472607 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.fernet_utils [None req-71769442-f157-436b-b328-a072185a5d63 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81767) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 16:12:05.480254 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83eda01c-ad9a-4343-bb3f-5711c8c362f9 admin admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 16:12:05.485409 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11d8ba04-4be8-4c6e-929f-a8dc1857f8aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a98b1e4431940ab8d1fc95df2858a6c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 16:12:05.488207 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d8ba04-4be8-4c6e-929f-a8dc1857f8aa admin admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:12:05.488207 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d8ba04-4be8-4c6e-929f-a8dc1857f8aa admin admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:12:05.488207 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d8ba04-4be8-4c6e-929f-a8dc1857f8aa admin admin] PATH_INFO: `/v3/roles` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:12:05.488207 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11d8ba04-4be8-4c6e-929f-a8dc1857f8aa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 16:12:05.489460 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11d8ba04-4be8-4c6e-929f-a8dc1857f8aa admin admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 16:12:05.500045 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 1123/2224] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Aug 22 16:12:05 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 22 16:12:05.504194 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11d8ba04-4be8-4c6e-929f-a8dc1857f8aa admin admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 16:12:05.533003 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11d8ba04-4be8-4c6e-929f-a8dc1857f8aa admin admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 16:12:05.553807 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71769442-f157-436b-b328-a072185a5d63 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 16:12:05.555643 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71769442-f157-436b-b328-a072185a5d63 service cinder] RBAC: Authorization granted {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 16:12:05.558463 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9b3820d-71d2-4c4f-a192-848d482b75b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a98b1e4431940ab8d1fc95df2858a6c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 16:12:05.559591 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9b3820d-71d2-4c4f-a192-848d482b75b5 admin admin] REQUEST_METHOD: `PUT` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:12:05.559874 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9b3820d-71d2-4c4f-a192-848d482b75b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:12:05.560246 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9b3820d-71d2-4c4f-a192-848d482b75b5 admin admin] PATH_INFO: `/v3/projects/0f9b4fe7ee53496db2381767acec2fed/users/6e6d487b67864d79b72a3291894c3015/roles/e6fc1ee3fb974125ba9b4fe55b12b45e` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:12:05.569941 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 1102/2225] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Aug 22 16:12:05 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 182 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 16:12:05.616494 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9b3820d-71d2-4c4f-a192-848d482b75b5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0f9b4fe7ee53496db2381767acec2fed, user_id=6e6d487b67864d79b72a3291894c3015, role_id=e6fc1ee3fb974125ba9b4fe55b12b45e)` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 16:12:05.624838 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9b3820d-71d2-4c4f-a192-848d482b75b5 admin admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 16:12:05.705780 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 1124/2226] 10.222.0.15 () {60 vars in 1483 bytes} [Mon Aug 22 16:12:05 2022] PUT /identity/v3/projects/0f9b4fe7ee53496db2381767acec2fed/users/6e6d487b67864d79b72a3291894c3015/roles/e6fc1ee3fb974125ba9b4fe55b12b45e => generated 0 bytes in 202 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 22 16:12:05.720020 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c7a6ef-e2ac-4873-9872-2a4f2b33e414 None None] REQUEST_METHOD: `POST` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:12:05.720408 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c7a6ef-e2ac-4873-9872-2a4f2b33e414 None None] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:12:05.720812 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c7a6ef-e2ac-4873-9872-2a4f2b33e414 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:12:05.771611 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.auth.core [None req-12c7a6ef-e2ac-4873-9872-2a4f2b33e414 None None] MFA Rules not processed for user `6e6d487b67864d79b72a3291894c3015`. Rule list: `[]` (Enabled: `True`). {{(pid=81767) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 16:12:05.772905 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.fernet_utils [None req-12c7a6ef-e2ac-4873-9872-2a4f2b33e414 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81767) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 16:12:05.802765 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=100 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 16:12:05.915476 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 1103/2227] 10.222.0.15 () {58 vars in 958 bytes} [Mon Aug 22 16:12:05 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 197 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 16:12:05.919729 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9b3820d-71d2-4c4f-a192-848d482b75b5 admin admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 16:12:06.047351 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9b3820d-71d2-4c4f-a192-848d482b75b5 admin admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 16:12:06.061624 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG dbcounter [-] [81767] Writing DB stats keystone:INSERT=1,keystone:DELETE=3,keystone:SELECT=96 {{(pid=81767) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 16:12:06.065619 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dac10b61-5cb2-409b-b950-0fa109fb2317 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a98b1e4431940ab8d1fc95df2858a6c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 16:12:06.066541 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac10b61-5cb2-409b-b950-0fa109fb2317 admin admin] REQUEST_METHOD: `POST` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:12:06.066677 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac10b61-5cb2-409b-b950-0fa109fb2317 admin admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:12:06.066950 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac10b61-5cb2-409b-b950-0fa109fb2317 admin admin] PATH_INFO: `/v3/projects` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:12:06.069048 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dac10b61-5cb2-409b-b950-0fa109fb2317 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 16:12:06.072240 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dac10b61-5cb2-409b-b950-0fa109fb2317 admin admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 16:12:06.200545 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 1125/2228] 10.222.0.15 () {60 vars in 1153 bytes} [Mon Aug 22 16:12:05 2022] POST /identity/v3/projects => generated 383 bytes in 281 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 22 16:12:06.204953 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12c7a6ef-e2ac-4873-9872-2a4f2b33e414 None None] Authenticating user token {{(pid=81767) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 16:12:06.232310 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12c7a6ef-e2ac-4873-9872-2a4f2b33e414 None None] Validating token access rules against request {{(pid=81767) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 16:12:06.304340 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deff6fe1-f715-4fad-b123-bac98a37e93e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a98b1e4431940ab8d1fc95df2858a6c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81767) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 16:12:06.305073 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deff6fe1-f715-4fad-b123-bac98a37e93e admin admin] REQUEST_METHOD: `POST` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:12:06.305419 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deff6fe1-f715-4fad-b123-bac98a37e93e admin admin] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:12:06.305759 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deff6fe1-f715-4fad-b123-bac98a37e93e admin admin] PATH_INFO: `/v3/users` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:12:06.306465 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deff6fe1-f715-4fad-b123-bac98a37e93e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 16:12:06.308584 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deff6fe1-f715-4fad-b123-bac98a37e93e admin admin] RBAC: Authorization granted {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 16:12:06.404646 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 16:12:06.438497 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 1104/2229] 10.222.0.15 () {60 vars in 1144 bytes} [Mon Aug 22 16:12:06 2022] POST /identity/v3/users => generated 298 bytes in 234 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 22 16:12:06.443187 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dac10b61-5cb2-409b-b950-0fa109fb2317 admin admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 16:12:06.456277 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dac10b61-5cb2-409b-b950-0fa109fb2317 admin admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 16:12:06.468461 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6357db34-7cc8-4ae7-add0-d22973dbffa7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a98b1e4431940ab8d1fc95df2858a6c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 16:12:06.469193 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6357db34-7cc8-4ae7-add0-d22973dbffa7 admin admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:12:06.469616 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6357db34-7cc8-4ae7-add0-d22973dbffa7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:12:06.469961 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6357db34-7cc8-4ae7-add0-d22973dbffa7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:12:06.470803 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6357db34-7cc8-4ae7-add0-d22973dbffa7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 16:12:06.472743 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6357db34-7cc8-4ae7-add0-d22973dbffa7 admin admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 16:12:06.485009 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 1126/2230] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Aug 22 16:12:06 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 22 16:12:06.489990 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deff6fe1-f715-4fad-b123-bac98a37e93e admin admin] Authenticating user token {{(pid=81767) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 16:12:06.503400 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deff6fe1-f715-4fad-b123-bac98a37e93e admin admin] Validating token access rules against request {{(pid=81767) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 16:12:06.508833 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dadceab5-bc34-404f-9079-9569fa26f317 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a98b1e4431940ab8d1fc95df2858a6c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81767) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 16:12:06.509514 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dadceab5-bc34-404f-9079-9569fa26f317 admin admin] REQUEST_METHOD: `PUT` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:12:06.509796 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dadceab5-bc34-404f-9079-9569fa26f317 admin admin] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:12:06.510072 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dadceab5-bc34-404f-9079-9569fa26f317 admin admin] PATH_INFO: `/v3/projects/a610ea9550694405817a23dc4fb53351/users/eac3491f5c77473db709eba84163575c/roles/9d9848cfd22e4b089db443c350e56670` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:12:06.526488 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dadceab5-bc34-404f-9079-9569fa26f317 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a610ea9550694405817a23dc4fb53351, user_id=eac3491f5c77473db709eba84163575c, role_id=9d9848cfd22e4b089db443c350e56670)` {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 16:12:06.529417 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dadceab5-bc34-404f-9079-9569fa26f317 admin admin] RBAC: Authorization granted {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 16:12:06.603311 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6357db34-7cc8-4ae7-add0-d22973dbffa7 admin admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 16:12:06.605089 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-6357db34-7cc8-4ae7-add0-d22973dbffa7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 16:12:06.620258 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6357db34-7cc8-4ae7-add0-d22973dbffa7 admin admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 16:12:06.625090 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-154753ed-789b-4b0d-8785-961db07b47ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a98b1e4431940ab8d1fc95df2858a6c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 16:12:06.628848 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-154753ed-789b-4b0d-8785-961db07b47ae admin admin] REQUEST_METHOD: `DELETE` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:12:06.628848 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-154753ed-789b-4b0d-8785-961db07b47ae admin admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:12:06.628848 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-154753ed-789b-4b0d-8785-961db07b47ae admin admin] PATH_INFO: `/v3/users/352157e88c864b3dbc3f3baa8350fea9` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:12:06.628848 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-154753ed-789b-4b0d-8785-961db07b47ae admin admin] RBAC: Authorizing `identity:delete_user(user_id=352157e88c864b3dbc3f3baa8350fea9)` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 16:12:06.630486 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-154753ed-789b-4b0d-8785-961db07b47ae admin admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 16:12:06.634897 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 1105/2231] 10.222.0.15 () {60 vars in 1483 bytes} [Mon Aug 22 16:12:06 2022] PUT /identity/v3/projects/a610ea9550694405817a23dc4fb53351/users/eac3491f5c77473db709eba84163575c/roles/9d9848cfd22e4b089db443c350e56670 => generated 0 bytes in 145 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 22 16:12:06.638946 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dadceab5-bc34-404f-9079-9569fa26f317 admin admin] Authenticating user token {{(pid=81767) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 16:12:06.724022 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dadceab5-bc34-404f-9079-9569fa26f317 admin admin] Validating token access rules against request {{(pid=81767) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 16:12:06.729184 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeb2d804-3a10-4ffb-abee-637ac531bb03 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a98b1e4431940ab8d1fc95df2858a6c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81767) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 16:12:06.729788 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb2d804-3a10-4ffb-abee-637ac531bb03 admin admin] REQUEST_METHOD: `GET` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:12:06.730065 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb2d804-3a10-4ffb-abee-637ac531bb03 admin admin] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:12:06.730332 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb2d804-3a10-4ffb-abee-637ac531bb03 admin admin] PATH_INFO: `/v3/roles` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:12:06.731039 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeb2d804-3a10-4ffb-abee-637ac531bb03 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 16:12:06.740106 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeb2d804-3a10-4ffb-abee-637ac531bb03 admin admin] RBAC: Authorization granted {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 16:12:06.745176 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 1106/2232] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Aug 22 16:12:06 2022] GET /identity/v3/roles => generated 1420 bytes in 107 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 22 16:12:06.749402 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeb2d804-3a10-4ffb-abee-637ac531bb03 admin admin] Authenticating user token {{(pid=81767) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 16:12:06.759776 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeb2d804-3a10-4ffb-abee-637ac531bb03 admin admin] Validating token access rules against request {{(pid=81767) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 16:12:06.764501 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad2a765e-2fda-49d3-829a-4fa104b3e426 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a98b1e4431940ab8d1fc95df2858a6c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81767) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 16:12:06.765190 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad2a765e-2fda-49d3-829a-4fa104b3e426 admin admin] REQUEST_METHOD: `PUT` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:12:06.765459 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad2a765e-2fda-49d3-829a-4fa104b3e426 admin admin] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:12:06.765731 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad2a765e-2fda-49d3-829a-4fa104b3e426 admin admin] PATH_INFO: `/v3/projects/a610ea9550694405817a23dc4fb53351/users/eac3491f5c77473db709eba84163575c/roles/9d9848cfd22e4b089db443c350e56670` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:12:06.767644 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad2a765e-2fda-49d3-829a-4fa104b3e426 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a610ea9550694405817a23dc4fb53351, user_id=eac3491f5c77473db709eba84163575c, role_id=9d9848cfd22e4b089db443c350e56670)` {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 16:12:06.770266 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad2a765e-2fda-49d3-829a-4fa104b3e426 admin admin] RBAC: Authorization granted {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 16:12:06.782627 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 1107/2233] 10.222.0.15 () {60 vars in 1483 bytes} [Mon Aug 22 16:12:06 2022] PUT /identity/v3/projects/a610ea9550694405817a23dc4fb53351/users/eac3491f5c77473db709eba84163575c/roles/9d9848cfd22e4b089db443c350e56670 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 22 16:12:06.793595 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc14735-f311-43b4-99da-0acaff8ab0e3 None None] REQUEST_METHOD: `POST` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:12:06.793878 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc14735-f311-43b4-99da-0acaff8ab0e3 None None] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:12:06.794328 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc14735-f311-43b4-99da-0acaff8ab0e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:12:06.823919 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.auth.core [None req-7bc14735-f311-43b4-99da-0acaff8ab0e3 None None] MFA Rules not processed for user `eac3491f5c77473db709eba84163575c`. Rule list: `[]` (Enabled: `True`). {{(pid=81767) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 16:12:06.825132 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.fernet_utils [None req-7bc14735-f311-43b4-99da-0acaff8ab0e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81767) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 16:12:06.899370 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 1108/2234] 10.222.0.15 () {58 vars in 958 bytes} [Mon Aug 22 16:12:06 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 22 16:12:06.910160 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bc14735-f311-43b4-99da-0acaff8ab0e3 None None] Authenticating user token {{(pid=81767) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 16:12:06.968511 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-154753ed-789b-4b0d-8785-961db07b47ae admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '352157e88c864b3dbc3f3baa8350fea9'} {{(pid=81768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 16:12:06.969419 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-154753ed-789b-4b0d-8785-961db07b47ae admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '352157e88c864b3dbc3f3baa8350fea9'} {{(pid=81768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 16:12:06.970044 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-154753ed-789b-4b0d-8785-961db07b47ae admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '352157e88c864b3dbc3f3baa8350fea9'} {{(pid=81768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 16:12:06.981495 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-154753ed-789b-4b0d-8785-961db07b47ae admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '352157e88c864b3dbc3f3baa8350fea9'} {{(pid=81768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 16:12:06.994579 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bc14735-f311-43b4-99da-0acaff8ab0e3 None None] Validating token access rules against request {{(pid=81767) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 16:12:07.000578 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-920b2849-623b-4fe6-814e-28ec22b7c716 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79706d480f864998b9f8594be407ceab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4ee0413f9ef4d4fb1f95a660babe090', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81767) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 16:12:07.001217 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-920b2849-623b-4fe6-814e-28ec22b7c716 service cinder] REQUEST_METHOD: `GET` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:12:07.002203 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-920b2849-623b-4fe6-814e-28ec22b7c716 service cinder] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:12:07.002631 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-920b2849-623b-4fe6-814e-28ec22b7c716 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:12:07.004571 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.fernet_utils [None req-920b2849-623b-4fe6-814e-28ec22b7c716 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81767) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 16:12:07.017895 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-920b2849-623b-4fe6-814e-28ec22b7c716 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 16:12:07.020050 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-920b2849-623b-4fe6-814e-28ec22b7c716 service cinder] RBAC: Authorization granted {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 16:12:07.030444 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 1109/2235] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Aug 22 16:12:06 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 121 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 16:12:07.055917 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-154753ed-789b-4b0d-8785-961db07b47ae admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '352157e88c864b3dbc3f3baa8350fea9'} {{(pid=81768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 16:12:07.109891 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 1127/2236] 10.222.0.15 () {58 vars in 1224 bytes} [Mon Aug 22 16:12:06 2022] DELETE /identity/v3/users/352157e88c864b3dbc3f3baa8350fea9 => generated 0 bytes in 506 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 22 16:12:07.118685 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97494cb7-9d7e-4864-8c81-3a839cc4e3a1 None None] REQUEST_METHOD: `POST` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:12:07.118964 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97494cb7-9d7e-4864-8c81-3a839cc4e3a1 None None] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:12:07.119221 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97494cb7-9d7e-4864-8c81-3a839cc4e3a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:12:07.168160 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.auth.core [None req-97494cb7-9d7e-4864-8c81-3a839cc4e3a1 None None] MFA Rules not processed for user `6b2199ae0fb0480aa917ada391f5ec54`. Rule list: `[]` (Enabled: `True`). {{(pid=81767) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 16:12:07.169488 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.fernet_utils [None req-97494cb7-9d7e-4864-8c81-3a839cc4e3a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81767) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 16:12:07.269005 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 1110/2237] 10.222.0.15 () {58 vars in 958 bytes} [Mon Aug 22 16:12:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 16:12:07.280717 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-154753ed-789b-4b0d-8785-961db07b47ae admin admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 16:12:07.284472 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-154753ed-789b-4b0d-8785-961db07b47ae admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 16:12:07.289433 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97494cb7-9d7e-4864-8c81-3a839cc4e3a1 None None] Authenticating user token {{(pid=81767) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 16:12:07.295915 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.fernet_utils [None req-97494cb7-9d7e-4864-8c81-3a839cc4e3a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81767) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 16:12:07.395239 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-154753ed-789b-4b0d-8785-961db07b47ae admin admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 16:12:07.399330 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0257f99-dd42-4e53-b5cd-e008908cee1f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79706d480f864998b9f8594be407ceab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4ee0413f9ef4d4fb1f95a660babe090', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 16:12:07.400024 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0257f99-dd42-4e53-b5cd-e008908cee1f service cinder] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:12:07.400367 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0257f99-dd42-4e53-b5cd-e008908cee1f service cinder] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:12:07.400725 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0257f99-dd42-4e53-b5cd-e008908cee1f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:12:07.402822 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-c0257f99-dd42-4e53-b5cd-e008908cee1f service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 16:12:07.417945 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97494cb7-9d7e-4864-8c81-3a839cc4e3a1 None None] Validating token access rules against request {{(pid=81767) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 16:12:07.427017 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-313b282c-430b-445f-ab57-6d9515a7490e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '234687adad594f1aa45775a565ffbdc7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4ee0413f9ef4d4fb1f95a660babe090', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81767) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 16:12:07.427799 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-313b282c-430b-445f-ab57-6d9515a7490e service neutron] REQUEST_METHOD: `GET` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:12:07.428248 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-313b282c-430b-445f-ab57-6d9515a7490e service neutron] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:12:07.428677 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-313b282c-430b-445f-ab57-6d9515a7490e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:12:07.439435 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-313b282c-430b-445f-ab57-6d9515a7490e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 16:12:07.441228 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-313b282c-430b-445f-ab57-6d9515a7490e service neutron] RBAC: Authorization granted {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 16:12:07.453646 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 1111/2238] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Aug 22 16:12:07 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 166 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 16:12:07.474563 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0257f99-dd42-4e53-b5cd-e008908cee1f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 16:12:07.476197 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0257f99-dd42-4e53-b5cd-e008908cee1f service cinder] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 16:12:07.487783 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 1128/2239] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Aug 22 16:12:07 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 208 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 16:12:07.762859 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG dbcounter [-] [81767] Writing DB stats keystone:SELECT=88,keystone:INSERT=4,keystone:DELETE=8 {{(pid=81767) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 16:12:08.171071 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-313b282c-430b-445f-ab57-6d9515a7490e service neutron] Authenticating user token {{(pid=81767) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 16:12:08.184262 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.fernet_utils [None req-313b282c-430b-445f-ab57-6d9515a7490e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81767) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 16:12:08.206612 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-313b282c-430b-445f-ab57-6d9515a7490e service neutron] Validating token access rules against request {{(pid=81767) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 16:12:08.212279 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3534a99-8bf8-4441-ba32-420ea12197b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a98b1e4431940ab8d1fc95df2858a6c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81767) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 16:12:08.212916 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3534a99-8bf8-4441-ba32-420ea12197b6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:12:08.213234 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3534a99-8bf8-4441-ba32-420ea12197b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:12:08.213481 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3534a99-8bf8-4441-ba32-420ea12197b6 admin admin] PATH_INFO: `/v3/projects/15b380b8d48c41ba87da5f9a329b25fe` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:12:08.214761 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3534a99-8bf8-4441-ba32-420ea12197b6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=15b380b8d48c41ba87da5f9a329b25fe)` {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 16:12:08.216908 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3534a99-8bf8-4441-ba32-420ea12197b6 admin admin] RBAC: Authorization granted {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 16:12:08.255992 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 16:12:08.383946 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.notifications [None req-d3534a99-8bf8-4441-ba32-420ea12197b6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '15b380b8d48c41ba87da5f9a329b25fe', 'request_id': 'req-d3534a99-8bf8-4441-ba32-420ea12197b6'} {{(pid=81767) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 16:12:08.390792 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.notifications [None req-d3534a99-8bf8-4441-ba32-420ea12197b6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '15b380b8d48c41ba87da5f9a329b25fe', 'request_id': 'req-d3534a99-8bf8-4441-ba32-420ea12197b6'} {{(pid=81767) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 16:12:08.516596 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.notifications [None req-d3534a99-8bf8-4441-ba32-420ea12197b6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '15b380b8d48c41ba87da5f9a329b25fe', 'request_id': 'req-d3534a99-8bf8-4441-ba32-420ea12197b6'} {{(pid=81767) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 16:12:08.535896 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.notifications [None req-d3534a99-8bf8-4441-ba32-420ea12197b6 admin admin] The token cache is being invalidate because project 15b380b8d48c41ba87da5f9a329b25fe was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81767) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 22 16:12:08.536203 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.notifications [None req-d3534a99-8bf8-4441-ba32-420ea12197b6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81767) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 16:12:08.536913 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.notifications [None req-d3534a99-8bf8-4441-ba32-420ea12197b6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81767) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 16:12:08.539104 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 1112/2240] 10.222.0.15 () {58 vars in 1233 bytes} [Mon Aug 22 16:12:08 2022] DELETE /identity/v3/projects/15b380b8d48c41ba87da5f9a329b25fe => generated 0 bytes in 370 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 16:12:08.543532 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0257f99-dd42-4e53-b5cd-e008908cee1f service cinder] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 16:12:08.555869 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0257f99-dd42-4e53-b5cd-e008908cee1f service cinder] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 16:12:08.663635 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91faa7a5-8d58-4743-96df-c4d978da1cad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a98b1e4431940ab8d1fc95df2858a6c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 16:12:08.669264 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91faa7a5-8d58-4743-96df-c4d978da1cad admin admin] REQUEST_METHOD: `DELETE` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:12:08.669610 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91faa7a5-8d58-4743-96df-c4d978da1cad admin admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:12:08.669889 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91faa7a5-8d58-4743-96df-c4d978da1cad admin admin] PATH_INFO: `/v3/users/81ec415f5f004eaa959c87f13933df51` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:12:08.672179 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91faa7a5-8d58-4743-96df-c4d978da1cad admin admin] RBAC: Authorizing `identity:delete_user(user_id=81ec415f5f004eaa959c87f13933df51)` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 16:12:08.677220 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91faa7a5-8d58-4743-96df-c4d978da1cad admin admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 16:12:09.074926 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-91faa7a5-8d58-4743-96df-c4d978da1cad admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '81ec415f5f004eaa959c87f13933df51'} {{(pid=81768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 16:12:09.082410 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-91faa7a5-8d58-4743-96df-c4d978da1cad admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '81ec415f5f004eaa959c87f13933df51'} {{(pid=81768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 16:12:09.082410 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-91faa7a5-8d58-4743-96df-c4d978da1cad admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '81ec415f5f004eaa959c87f13933df51'} {{(pid=81768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 16:12:09.087026 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-91faa7a5-8d58-4743-96df-c4d978da1cad admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '81ec415f5f004eaa959c87f13933df51'} {{(pid=81768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 16:12:09.331034 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-91faa7a5-8d58-4743-96df-c4d978da1cad admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '81ec415f5f004eaa959c87f13933df51'} {{(pid=81768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 16:12:09.359900 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 1129/2241] 10.222.0.15 () {58 vars in 1224 bytes} [Mon Aug 22 16:12:08 2022] DELETE /identity/v3/users/81ec415f5f004eaa959c87f13933df51 => generated 0 bytes in 817 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 16:12:09.872425 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3534a99-8bf8-4441-ba32-420ea12197b6 admin admin] Authenticating user token {{(pid=81767) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 16:12:09.874012 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.fernet_utils [None req-d3534a99-8bf8-4441-ba32-420ea12197b6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81767) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 16:12:09.894961 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3534a99-8bf8-4441-ba32-420ea12197b6 admin admin] Validating token access rules against request {{(pid=81767) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 16:12:09.986232 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4d44f23-845b-4acf-8998-6f7c3fbc3845 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a98b1e4431940ab8d1fc95df2858a6c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81767) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 16:12:09.987790 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d44f23-845b-4acf-8998-6f7c3fbc3845 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:12:09.992377 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d44f23-845b-4acf-8998-6f7c3fbc3845 admin admin] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:12:09.992703 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d44f23-845b-4acf-8998-6f7c3fbc3845 admin admin] PATH_INFO: `/v3/projects/3af9d400743147ed957a276423b43f39` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:12:09.997891 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4d44f23-845b-4acf-8998-6f7c3fbc3845 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3af9d400743147ed957a276423b43f39)` {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 16:12:09.998225 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4d44f23-845b-4acf-8998-6f7c3fbc3845 admin admin] RBAC: Authorization granted {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 16:12:10.137275 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.notifications [None req-e4d44f23-845b-4acf-8998-6f7c3fbc3845 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3af9d400743147ed957a276423b43f39', 'request_id': 'req-e4d44f23-845b-4acf-8998-6f7c3fbc3845'} {{(pid=81767) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 16:12:10.143443 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.notifications [None req-e4d44f23-845b-4acf-8998-6f7c3fbc3845 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3af9d400743147ed957a276423b43f39', 'request_id': 'req-e4d44f23-845b-4acf-8998-6f7c3fbc3845'} {{(pid=81767) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 16:12:10.242149 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.notifications [None req-e4d44f23-845b-4acf-8998-6f7c3fbc3845 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3af9d400743147ed957a276423b43f39', 'request_id': 'req-e4d44f23-845b-4acf-8998-6f7c3fbc3845'} {{(pid=81767) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 16:12:10.255971 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.notifications [None req-e4d44f23-845b-4acf-8998-6f7c3fbc3845 admin admin] The token cache is being invalidate because project 3af9d400743147ed957a276423b43f39 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81767) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 22 16:12:10.256268 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.notifications [None req-e4d44f23-845b-4acf-8998-6f7c3fbc3845 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81767) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 16:12:10.261244 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.notifications [None req-e4d44f23-845b-4acf-8998-6f7c3fbc3845 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81767) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 16:12:10.262871 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 1113/2242] 10.222.0.15 () {58 vars in 1233 bytes} [Mon Aug 22 16:12:09 2022] DELETE /identity/v3/projects/3af9d400743147ed957a276423b43f39 => generated 0 bytes in 391 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 16:12:10.267699 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2eaa56d-c91e-46e3-b335-3fb3a6b986d8 None None] REQUEST_METHOD: `POST` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:12:10.268214 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2eaa56d-c91e-46e3-b335-3fb3a6b986d8 None None] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:12:10.268500 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2eaa56d-c91e-46e3-b335-3fb3a6b986d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:12:10.290447 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.auth.core [None req-b2eaa56d-c91e-46e3-b335-3fb3a6b986d8 None None] MFA Rules not processed for user `6b2199ae0fb0480aa917ada391f5ec54`. Rule list: `[]` (Enabled: `True`). {{(pid=81768) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 16:12:10.292071 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-b2eaa56d-c91e-46e3-b335-3fb3a6b986d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 16:12:10.374183 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 1130/2243] 10.222.0.15 () {58 vars in 958 bytes} [Mon Aug 22 16:12:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 16:12:10.382364 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a572953-a042-4b20-9d47-1f3e948ab77c None None] REQUEST_METHOD: `POST` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:12:10.382669 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a572953-a042-4b20-9d47-1f3e948ab77c None None] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:12:10.383058 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a572953-a042-4b20-9d47-1f3e948ab77c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:12:10.403767 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.auth.core [None req-8a572953-a042-4b20-9d47-1f3e948ab77c None None] MFA Rules not processed for user `6b2199ae0fb0480aa917ada391f5ec54`. Rule list: `[]` (Enabled: `True`). {{(pid=81767) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 16:12:10.404902 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.fernet_utils [None req-8a572953-a042-4b20-9d47-1f3e948ab77c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81767) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 16:12:10.413607 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 1114/2244] 10.222.0.15 () {58 vars in 958 bytes} [Mon Aug 22 16:12:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 16:12:10.417621 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2eaa56d-c91e-46e3-b335-3fb3a6b986d8 None None] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 16:12:10.429890 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2eaa56d-c91e-46e3-b335-3fb3a6b986d8 None None] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 16:12:10.434892 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92951eaa-ed1a-40a2-b296-b3e037c3017b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a98b1e4431940ab8d1fc95df2858a6c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 16:12:10.435722 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92951eaa-ed1a-40a2-b296-b3e037c3017b admin admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:12:10.436007 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92951eaa-ed1a-40a2-b296-b3e037c3017b admin admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:12:10.436332 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92951eaa-ed1a-40a2-b296-b3e037c3017b admin admin] PATH_INFO: `/v3/domains` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:12:10.436960 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92951eaa-ed1a-40a2-b296-b3e037c3017b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81768) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 16:12:10.437260 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92951eaa-ed1a-40a2-b296-b3e037c3017b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 16:12:10.438890 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92951eaa-ed1a-40a2-b296-b3e037c3017b admin admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 16:12:10.452167 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 1131/2245] 10.222.0.15 () {58 vars in 1153 bytes} [Mon Aug 22 16:12:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 16:12:10.456957 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a572953-a042-4b20-9d47-1f3e948ab77c None None] Authenticating user token {{(pid=81767) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 16:12:10.475146 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a572953-a042-4b20-9d47-1f3e948ab77c None None] Validating token access rules against request {{(pid=81767) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 16:12:10.480444 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d6e1761-77cf-41ea-88df-becbffd98a78 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a98b1e4431940ab8d1fc95df2858a6c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81767) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 16:12:10.481070 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d6e1761-77cf-41ea-88df-becbffd98a78 admin admin] REQUEST_METHOD: `POST` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:12:10.483109 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d6e1761-77cf-41ea-88df-becbffd98a78 admin admin] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:12:10.483109 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d6e1761-77cf-41ea-88df-becbffd98a78 admin admin] PATH_INFO: `/v3/projects` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:12:10.483109 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d6e1761-77cf-41ea-88df-becbffd98a78 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 16:12:10.484459 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d6e1761-77cf-41ea-88df-becbffd98a78 admin admin] RBAC: Authorization granted {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 16:12:10.597888 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 1115/2246] 10.222.0.15 () {60 vars in 1153 bytes} [Mon Aug 22 16:12:10 2022] POST /identity/v3/projects => generated 385 bytes in 141 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 22 16:12:10.602305 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92951eaa-ed1a-40a2-b296-b3e037c3017b admin admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 16:12:10.618553 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92951eaa-ed1a-40a2-b296-b3e037c3017b admin admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 16:12:10.700884 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bbe47d3-952c-47d4-b88d-3ad04fb24241 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a98b1e4431940ab8d1fc95df2858a6c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 16:12:10.701510 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bbe47d3-952c-47d4-b88d-3ad04fb24241 admin admin] REQUEST_METHOD: `POST` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:12:10.701791 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bbe47d3-952c-47d4-b88d-3ad04fb24241 admin admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:12:10.702064 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bbe47d3-952c-47d4-b88d-3ad04fb24241 admin admin] PATH_INFO: `/v3/users` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:12:10.702739 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bbe47d3-952c-47d4-b88d-3ad04fb24241 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 16:12:10.704815 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bbe47d3-952c-47d4-b88d-3ad04fb24241 admin admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 16:12:10.974696 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d6e1761-77cf-41ea-88df-becbffd98a78 admin admin] Authenticating user token {{(pid=81767) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 16:12:10.983804 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 1132/2247] 10.222.0.15 () {60 vars in 1144 bytes} [Mon Aug 22 16:12:10 2022] POST /identity/v3/users => generated 293 bytes in 382 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 22 16:12:10.989105 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d6e1761-77cf-41ea-88df-becbffd98a78 admin admin] Validating token access rules against request {{(pid=81767) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 16:12:11.010024 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2db4532-15d7-40ae-8f37-c31ce2989fb7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a98b1e4431940ab8d1fc95df2858a6c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81767) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 16:12:11.010914 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2db4532-15d7-40ae-8f37-c31ce2989fb7 admin admin] REQUEST_METHOD: `GET` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:12:11.011474 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2db4532-15d7-40ae-8f37-c31ce2989fb7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:12:11.011744 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2db4532-15d7-40ae-8f37-c31ce2989fb7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:12:11.012465 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2db4532-15d7-40ae-8f37-c31ce2989fb7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 16:12:11.018459 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2db4532-15d7-40ae-8f37-c31ce2989fb7 admin admin] RBAC: Authorization granted {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 16:12:11.037648 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 1116/2248] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Aug 22 16:12:10 2022] GET /identity/v3/roles => generated 1420 bytes in 63 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 22 16:12:11.038596 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bbe47d3-952c-47d4-b88d-3ad04fb24241 admin admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 16:12:11.070659 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bbe47d3-952c-47d4-b88d-3ad04fb24241 admin admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 16:12:11.073021 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG dbcounter [-] [81767] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=81767) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 16:12:11.080771 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32697665-94e5-4896-93b4-d2aa2ad54fd0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a98b1e4431940ab8d1fc95df2858a6c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 16:12:11.080771 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32697665-94e5-4896-93b4-d2aa2ad54fd0 admin admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:12:11.080771 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32697665-94e5-4896-93b4-d2aa2ad54fd0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:12:11.081233 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32697665-94e5-4896-93b4-d2aa2ad54fd0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:12:11.081825 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32697665-94e5-4896-93b4-d2aa2ad54fd0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 16:12:11.083694 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32697665-94e5-4896-93b4-d2aa2ad54fd0 admin admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 16:12:11.099298 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 1133/2249] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Aug 22 16:12:11 2022] GET /identity/v3/roles => generated 1420 bytes in 60 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 22 16:12:11.110007 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2db4532-15d7-40ae-8f37-c31ce2989fb7 admin admin] Authenticating user token {{(pid=81767) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 16:12:11.110007 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.fernet_utils [None req-f2db4532-15d7-40ae-8f37-c31ce2989fb7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81767) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 16:12:11.111165 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32697665-94e5-4896-93b4-d2aa2ad54fd0 admin admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 16:12:11.127795 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2db4532-15d7-40ae-8f37-c31ce2989fb7 admin admin] Validating token access rules against request {{(pid=81767) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 16:12:11.142216 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32697665-94e5-4896-93b4-d2aa2ad54fd0 admin admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 16:12:11.148756 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d78b826-934f-4976-8f02-423c2cd04aa2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a98b1e4431940ab8d1fc95df2858a6c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81767) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 16:12:11.151837 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d78b826-934f-4976-8f02-423c2cd04aa2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:12:11.152193 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d78b826-934f-4976-8f02-423c2cd04aa2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:12:11.152614 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d78b826-934f-4976-8f02-423c2cd04aa2 admin admin] PATH_INFO: `/v3/users/7258e36c0b2f42198d87fff78f1978bb` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:12:11.156538 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a84c2da0-5694-4449-80ee-c95979176956 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a98b1e4431940ab8d1fc95df2858a6c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 16:12:11.157152 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a84c2da0-5694-4449-80ee-c95979176956 admin admin] REQUEST_METHOD: `PUT` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:12:11.157443 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a84c2da0-5694-4449-80ee-c95979176956 admin admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:12:11.157696 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a84c2da0-5694-4449-80ee-c95979176956 admin admin] PATH_INFO: `/v3/projects/6902a92f7a29499cb5315eb1ef9dee79/users/ddb50764dc754749b67c4bdb6d156efc/roles/e6fc1ee3fb974125ba9b4fe55b12b45e` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:12:11.159182 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d78b826-934f-4976-8f02-423c2cd04aa2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7258e36c0b2f42198d87fff78f1978bb)` {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 16:12:11.163728 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d78b826-934f-4976-8f02-423c2cd04aa2 admin admin] RBAC: Authorization granted {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 16:12:11.225239 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a84c2da0-5694-4449-80ee-c95979176956 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6902a92f7a29499cb5315eb1ef9dee79, user_id=ddb50764dc754749b67c4bdb6d156efc, role_id=e6fc1ee3fb974125ba9b4fe55b12b45e)` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 16:12:11.228672 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a84c2da0-5694-4449-80ee-c95979176956 admin admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 16:12:11.333458 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 1134/2250] 10.222.0.15 () {60 vars in 1483 bytes} [Mon Aug 22 16:12:11 2022] PUT /identity/v3/projects/6902a92f7a29499cb5315eb1ef9dee79/users/ddb50764dc754749b67c4bdb6d156efc/roles/e6fc1ee3fb974125ba9b4fe55b12b45e => generated 0 bytes in 223 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 22 16:12:11.346968 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec7ac821-a64e-4878-b265-883ed0f11657 None None] REQUEST_METHOD: `POST` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:12:11.347344 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec7ac821-a64e-4878-b265-883ed0f11657 None None] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:12:11.347642 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec7ac821-a64e-4878-b265-883ed0f11657 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:12:11.359195 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=83,keystone:DELETE=15,keystone:INSERT=2 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 16:12:11.393556 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.auth.core [None req-ec7ac821-a64e-4878-b265-883ed0f11657 None None] MFA Rules not processed for user `ddb50764dc754749b67c4bdb6d156efc`. Rule list: `[]` (Enabled: `True`). {{(pid=81768) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 16:12:11.394834 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-ec7ac821-a64e-4878-b265-883ed0f11657 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 16:12:11.442800 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.notifications [None req-4d78b826-934f-4976-8f02-423c2cd04aa2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7258e36c0b2f42198d87fff78f1978bb'} {{(pid=81767) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 16:12:11.443558 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.notifications [None req-4d78b826-934f-4976-8f02-423c2cd04aa2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7258e36c0b2f42198d87fff78f1978bb'} {{(pid=81767) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 16:12:11.513544 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.notifications [None req-4d78b826-934f-4976-8f02-423c2cd04aa2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7258e36c0b2f42198d87fff78f1978bb'} {{(pid=81767) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 16:12:11.524422 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.notifications [None req-4d78b826-934f-4976-8f02-423c2cd04aa2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7258e36c0b2f42198d87fff78f1978bb'} {{(pid=81767) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 16:12:11.527728 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 1135/2251] 10.222.0.15 () {58 vars in 958 bytes} [Mon Aug 22 16:12:11 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 22 16:12:11.537274 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec7ac821-a64e-4878-b265-883ed0f11657 None None] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 16:12:11.538726 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-ec7ac821-a64e-4878-b265-883ed0f11657 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 16:12:11.552694 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.notifications [None req-4d78b826-934f-4976-8f02-423c2cd04aa2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7258e36c0b2f42198d87fff78f1978bb'} {{(pid=81767) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 16:12:11.561680 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 1117/2252] 10.222.0.15 () {58 vars in 1224 bytes} [Mon Aug 22 16:12:11 2022] DELETE /identity/v3/users/7258e36c0b2f42198d87fff78f1978bb => generated 0 bytes in 468 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 16:12:11.566387 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90834eb7-652a-4987-8951-4fa2b0d2d0c7 None None] REQUEST_METHOD: `POST` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:12:11.566781 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90834eb7-652a-4987-8951-4fa2b0d2d0c7 None None] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:12:11.567075 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90834eb7-652a-4987-8951-4fa2b0d2d0c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:12:11.614291 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.auth.core [None req-90834eb7-652a-4987-8951-4fa2b0d2d0c7 None None] MFA Rules not processed for user `6b2199ae0fb0480aa917ada391f5ec54`. Rule list: `[]` (Enabled: `True`). {{(pid=81767) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 16:12:11.615906 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.fernet_utils [None req-90834eb7-652a-4987-8951-4fa2b0d2d0c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81767) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 16:12:11.651756 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec7ac821-a64e-4878-b265-883ed0f11657 None None] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 16:12:11.731111 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 1118/2253] 10.222.0.15 () {58 vars in 958 bytes} [Mon Aug 22 16:12:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 16:12:11.745307 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90834eb7-652a-4987-8951-4fa2b0d2d0c7 None None] Authenticating user token {{(pid=81767) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 16:12:11.745696 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.fernet_utils [None req-90834eb7-652a-4987-8951-4fa2b0d2d0c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81767) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 16:12:11.790456 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb002872-98af-4bfb-95e8-640abe6a52cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a98b1e4431940ab8d1fc95df2858a6c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 16:12:11.797303 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb002872-98af-4bfb-95e8-640abe6a52cb admin admin] REQUEST_METHOD: `POST` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:12:11.797303 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb002872-98af-4bfb-95e8-640abe6a52cb admin admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:12:11.797303 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb002872-98af-4bfb-95e8-640abe6a52cb admin admin] PATH_INFO: `/v3/projects` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:12:11.797303 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb002872-98af-4bfb-95e8-640abe6a52cb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 16:12:11.797303 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb002872-98af-4bfb-95e8-640abe6a52cb admin admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 16:12:11.844276 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90834eb7-652a-4987-8951-4fa2b0d2d0c7 None None] Validating token access rules against request {{(pid=81767) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 16:12:11.855015 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-358246b3-fbc2-4e43-8dfd-bdfe33f79193 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '234687adad594f1aa45775a565ffbdc7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4ee0413f9ef4d4fb1f95a660babe090', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81767) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 16:12:11.855681 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-358246b3-fbc2-4e43-8dfd-bdfe33f79193 service neutron] REQUEST_METHOD: `GET` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:12:11.855962 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-358246b3-fbc2-4e43-8dfd-bdfe33f79193 service neutron] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:12:11.856240 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-358246b3-fbc2-4e43-8dfd-bdfe33f79193 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:12:11.875870 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-358246b3-fbc2-4e43-8dfd-bdfe33f79193 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 16:12:11.878185 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-358246b3-fbc2-4e43-8dfd-bdfe33f79193 service neutron] RBAC: Authorization granted {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 16:12:11.891701 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 1119/2254] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Aug 22 16:12:11 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 149 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 22 16:12:11.915930 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 1136/2255] 10.222.0.15 () {60 vars in 1153 bytes} [Mon Aug 22 16:12:11 2022] POST /identity/v3/projects => generated 385 bytes in 379 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 22 16:12:11.920215 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-358246b3-fbc2-4e43-8dfd-bdfe33f79193 service neutron] Authenticating user token {{(pid=81767) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 16:12:11.965462 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-358246b3-fbc2-4e43-8dfd-bdfe33f79193 service neutron] Validating token access rules against request {{(pid=81767) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 16:12:12.101687 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a999cb50-65c5-4fbb-a87f-2a9f291e1de1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a98b1e4431940ab8d1fc95df2858a6c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81767) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 16:12:12.102388 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a999cb50-65c5-4fbb-a87f-2a9f291e1de1 admin admin] REQUEST_METHOD: `POST` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:12:12.102588 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a999cb50-65c5-4fbb-a87f-2a9f291e1de1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:12:12.102846 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a999cb50-65c5-4fbb-a87f-2a9f291e1de1 admin admin] PATH_INFO: `/v3/users` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:12:12.103814 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a999cb50-65c5-4fbb-a87f-2a9f291e1de1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 16:12:12.107349 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a999cb50-65c5-4fbb-a87f-2a9f291e1de1 admin admin] RBAC: Authorization granted {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 16:12:12.289677 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 1120/2256] 10.222.0.15 () {60 vars in 1144 bytes} [Mon Aug 22 16:12:11 2022] POST /identity/v3/users => generated 299 bytes in 367 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 22 16:12:12.294986 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb002872-98af-4bfb-95e8-640abe6a52cb admin admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 16:12:12.315341 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb002872-98af-4bfb-95e8-640abe6a52cb admin admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 16:12:12.325662 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2431e9b2-5156-43de-9a88-8d5f056ada4f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a98b1e4431940ab8d1fc95df2858a6c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 16:12:12.327901 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2431e9b2-5156-43de-9a88-8d5f056ada4f admin admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:12:12.328620 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2431e9b2-5156-43de-9a88-8d5f056ada4f admin admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:12:12.329097 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2431e9b2-5156-43de-9a88-8d5f056ada4f admin admin] PATH_INFO: `/v3/roles` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:12:12.329781 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2431e9b2-5156-43de-9a88-8d5f056ada4f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 16:12:12.336141 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2431e9b2-5156-43de-9a88-8d5f056ada4f admin admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 16:12:12.387410 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 1137/2257] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Aug 22 16:12:12 2022] GET /identity/v3/roles => generated 1420 bytes in 96 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 22 16:12:12.392132 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a999cb50-65c5-4fbb-a87f-2a9f291e1de1 admin admin] Authenticating user token {{(pid=81767) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 16:12:12.405644 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a999cb50-65c5-4fbb-a87f-2a9f291e1de1 admin admin] Validating token access rules against request {{(pid=81767) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 16:12:12.417798 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d18a46b3-8c01-4bc0-a0cd-ff4c54f1e71a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a98b1e4431940ab8d1fc95df2858a6c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81767) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 16:12:12.423649 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d18a46b3-8c01-4bc0-a0cd-ff4c54f1e71a admin admin] REQUEST_METHOD: `PUT` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:12:12.423649 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d18a46b3-8c01-4bc0-a0cd-ff4c54f1e71a admin admin] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:12:12.423649 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d18a46b3-8c01-4bc0-a0cd-ff4c54f1e71a admin admin] PATH_INFO: `/v3/projects/27fe16fa1d8845718830fa26a9dcff7e/users/8efc4f0227b6447696ce159f946d0428/roles/9d9848cfd22e4b089db443c350e56670` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:12:12.453010 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d18a46b3-8c01-4bc0-a0cd-ff4c54f1e71a admin admin] RBAC: Authorizing `identity:create_grant(project_id=27fe16fa1d8845718830fa26a9dcff7e, user_id=8efc4f0227b6447696ce159f946d0428, role_id=9d9848cfd22e4b089db443c350e56670)` {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 16:12:12.457524 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d18a46b3-8c01-4bc0-a0cd-ff4c54f1e71a admin admin] RBAC: Authorization granted {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 16:12:12.636353 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 1121/2258] 10.222.0.15 () {60 vars in 1483 bytes} [Mon Aug 22 16:12:12 2022] PUT /identity/v3/projects/27fe16fa1d8845718830fa26a9dcff7e/users/8efc4f0227b6447696ce159f946d0428/roles/9d9848cfd22e4b089db443c350e56670 => generated 0 bytes in 245 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 22 16:12:12.647380 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2431e9b2-5156-43de-9a88-8d5f056ada4f admin admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 16:12:12.661959 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2431e9b2-5156-43de-9a88-8d5f056ada4f admin admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 16:12:12.693978 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG dbcounter [-] [81767] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81767) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 16:12:12.702180 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d18a46b3-8c01-4bc0-a0cd-ff4c54f1e71a admin admin] Authenticating user token {{(pid=81767) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 16:12:12.703444 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.fernet_utils [None req-d18a46b3-8c01-4bc0-a0cd-ff4c54f1e71a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81767) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 16:12:12.763820 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17af55a8-9a4e-4cfc-9fe1-205683c9a5df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a98b1e4431940ab8d1fc95df2858a6c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 16:12:12.763820 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17af55a8-9a4e-4cfc-9fe1-205683c9a5df admin admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:12:12.763820 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17af55a8-9a4e-4cfc-9fe1-205683c9a5df admin admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:12:12.763820 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17af55a8-9a4e-4cfc-9fe1-205683c9a5df admin admin] PATH_INFO: `/v3/roles` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:12:12.764954 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17af55a8-9a4e-4cfc-9fe1-205683c9a5df admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 16:12:12.767012 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17af55a8-9a4e-4cfc-9fe1-205683c9a5df admin admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 16:12:12.785137 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 1138/2259] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Aug 22 16:12:12 2022] GET /identity/v3/roles => generated 1420 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 22 16:12:12.786390 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17af55a8-9a4e-4cfc-9fe1-205683c9a5df admin admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 16:12:12.808625 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17af55a8-9a4e-4cfc-9fe1-205683c9a5df admin admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 16:12:12.823990 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12a127c6-29cb-4146-a9b9-7cce0620f118 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a98b1e4431940ab8d1fc95df2858a6c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 16:12:12.825076 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12a127c6-29cb-4146-a9b9-7cce0620f118 admin admin] REQUEST_METHOD: `PUT` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:12:12.831209 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12a127c6-29cb-4146-a9b9-7cce0620f118 admin admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:12:12.831209 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12a127c6-29cb-4146-a9b9-7cce0620f118 admin admin] PATH_INFO: `/v3/projects/27fe16fa1d8845718830fa26a9dcff7e/users/8efc4f0227b6447696ce159f946d0428/roles/9d9848cfd22e4b089db443c350e56670` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:12:12.831209 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12a127c6-29cb-4146-a9b9-7cce0620f118 admin admin] RBAC: Authorizing `identity:create_grant(project_id=27fe16fa1d8845718830fa26a9dcff7e, user_id=8efc4f0227b6447696ce159f946d0428, role_id=9d9848cfd22e4b089db443c350e56670)` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 16:12:12.836924 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12a127c6-29cb-4146-a9b9-7cce0620f118 admin admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 16:12:12.848608 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d18a46b3-8c01-4bc0-a0cd-ff4c54f1e71a admin admin] Validating token access rules against request {{(pid=81767) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 16:12:12.867564 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 1139/2260] 10.222.0.15 () {60 vars in 1483 bytes} [Mon Aug 22 16:12:12 2022] PUT /identity/v3/projects/27fe16fa1d8845718830fa26a9dcff7e/users/8efc4f0227b6447696ce159f946d0428/roles/9d9848cfd22e4b089db443c350e56670 => generated 0 bytes in 82 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 16:12:12.868732 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12a127c6-29cb-4146-a9b9-7cce0620f118 admin admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 16:12:12.871103 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-12a127c6-29cb-4146-a9b9-7cce0620f118 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 16:12:12.943762 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8207022a-951f-45e4-8ef4-5c5dd25a3dcc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a98b1e4431940ab8d1fc95df2858a6c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81767) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 16:12:12.944401 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8207022a-951f-45e4-8ef4-5c5dd25a3dcc admin admin] REQUEST_METHOD: `DELETE` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:12:12.944680 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8207022a-951f-45e4-8ef4-5c5dd25a3dcc admin admin] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:12:12.944952 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8207022a-951f-45e4-8ef4-5c5dd25a3dcc admin admin] PATH_INFO: `/v3/projects/d07fcbf4f0e34ffdb541d32ee95b9a5c` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:12:12.946195 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8207022a-951f-45e4-8ef4-5c5dd25a3dcc admin admin] RBAC: Authorizing `identity:delete_project(project_id=d07fcbf4f0e34ffdb541d32ee95b9a5c)` {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 16:12:12.948307 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8207022a-951f-45e4-8ef4-5c5dd25a3dcc admin admin] RBAC: Authorization granted {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 16:12:12.993789 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12a127c6-29cb-4146-a9b9-7cce0620f118 admin admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 16:12:12.996843 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99c04b72-08f3-4c62-a15a-8bfe8ffef325 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a98b1e4431940ab8d1fc95df2858a6c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 16:12:12.997536 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99c04b72-08f3-4c62-a15a-8bfe8ffef325 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:12:12.998683 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99c04b72-08f3-4c62-a15a-8bfe8ffef325 admin admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:12:12.998683 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99c04b72-08f3-4c62-a15a-8bfe8ffef325 admin admin] PATH_INFO: `/v3/users/6e6d487b67864d79b72a3291894c3015` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:12:12.999710 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99c04b72-08f3-4c62-a15a-8bfe8ffef325 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6e6d487b67864d79b72a3291894c3015)` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 16:12:13.001795 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99c04b72-08f3-4c62-a15a-8bfe8ffef325 admin admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 16:12:13.056246 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:DELETE=9,keystone:SELECT=88,keystone:INSERT=3 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 16:12:13.057410 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.notifications [None req-8207022a-951f-45e4-8ef4-5c5dd25a3dcc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd07fcbf4f0e34ffdb541d32ee95b9a5c', 'request_id': 'req-8207022a-951f-45e4-8ef4-5c5dd25a3dcc'} {{(pid=81767) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 16:12:13.063535 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.notifications [None req-8207022a-951f-45e4-8ef4-5c5dd25a3dcc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd07fcbf4f0e34ffdb541d32ee95b9a5c', 'request_id': 'req-8207022a-951f-45e4-8ef4-5c5dd25a3dcc'} {{(pid=81767) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 16:12:13.109768 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.notifications [None req-8207022a-951f-45e4-8ef4-5c5dd25a3dcc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd07fcbf4f0e34ffdb541d32ee95b9a5c', 'request_id': 'req-8207022a-951f-45e4-8ef4-5c5dd25a3dcc'} {{(pid=81767) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 16:12:13.126541 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.notifications [None req-8207022a-951f-45e4-8ef4-5c5dd25a3dcc admin admin] The token cache is being invalidate because project d07fcbf4f0e34ffdb541d32ee95b9a5c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81767) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 22 16:12:13.126831 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.notifications [None req-8207022a-951f-45e4-8ef4-5c5dd25a3dcc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81767) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 16:12:13.128359 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.notifications [None req-8207022a-951f-45e4-8ef4-5c5dd25a3dcc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81767) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 16:12:13.130057 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 1122/2261] 10.222.0.15 () {58 vars in 1233 bytes} [Mon Aug 22 16:12:12 2022] DELETE /identity/v3/projects/d07fcbf4f0e34ffdb541d32ee95b9a5c => generated 0 bytes in 429 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 22 16:12:13.133389 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2afdc044-3631-47f0-93e7-b9135adcff26 None None] REQUEST_METHOD: `POST` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:12:13.133389 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2afdc044-3631-47f0-93e7-b9135adcff26 None None] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:12:13.133389 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2afdc044-3631-47f0-93e7-b9135adcff26 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:12:13.166609 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.auth.core [None req-2afdc044-3631-47f0-93e7-b9135adcff26 None None] MFA Rules not processed for user `8efc4f0227b6447696ce159f946d0428`. Rule list: `[]` (Enabled: `True`). {{(pid=81767) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 16:12:13.167938 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.fernet_utils [None req-2afdc044-3631-47f0-93e7-b9135adcff26 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81767) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 16:12:13.190092 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-99c04b72-08f3-4c62-a15a-8bfe8ffef325 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6e6d487b67864d79b72a3291894c3015'} {{(pid=81768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 16:12:13.190812 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-99c04b72-08f3-4c62-a15a-8bfe8ffef325 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6e6d487b67864d79b72a3291894c3015'} {{(pid=81768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 16:12:13.191482 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-99c04b72-08f3-4c62-a15a-8bfe8ffef325 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6e6d487b67864d79b72a3291894c3015'} {{(pid=81768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 16:12:13.199364 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-99c04b72-08f3-4c62-a15a-8bfe8ffef325 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6e6d487b67864d79b72a3291894c3015'} {{(pid=81768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 16:12:13.245923 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 1123/2262] 10.222.0.15 () {58 vars in 958 bytes} [Mon Aug 22 16:12:13 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 16:12:13.247198 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2afdc044-3631-47f0-93e7-b9135adcff26 None None] Authenticating user token {{(pid=81767) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 16:12:13.248688 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.fernet_utils [None req-2afdc044-3631-47f0-93e7-b9135adcff26 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81767) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 16:12:13.306421 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-99c04b72-08f3-4c62-a15a-8bfe8ffef325 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6e6d487b67864d79b72a3291894c3015'} {{(pid=81768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 16:12:13.319688 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2afdc044-3631-47f0-93e7-b9135adcff26 None None] Validating token access rules against request {{(pid=81767) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 16:12:13.324373 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00d43087-68bb-4b0e-a2da-cc5229e9a761 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a98b1e4431940ab8d1fc95df2858a6c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81767) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 16:12:13.325006 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d43087-68bb-4b0e-a2da-cc5229e9a761 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:12:13.325282 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d43087-68bb-4b0e-a2da-cc5229e9a761 admin admin] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:12:13.325553 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d43087-68bb-4b0e-a2da-cc5229e9a761 admin admin] PATH_INFO: `/v3/users/0e52755517194970ac5a9652bf24827a` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:12:13.325658 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 1140/2263] 10.222.0.15 () {58 vars in 1224 bytes} [Mon Aug 22 16:12:12 2022] DELETE /identity/v3/users/6e6d487b67864d79b72a3291894c3015 => generated 0 bytes in 457 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 16:12:13.326647 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99c04b72-08f3-4c62-a15a-8bfe8ffef325 admin admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 16:12:13.326772 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00d43087-68bb-4b0e-a2da-cc5229e9a761 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0e52755517194970ac5a9652bf24827a)` {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 16:12:13.327974 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-99c04b72-08f3-4c62-a15a-8bfe8ffef325 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 16:12:13.328685 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00d43087-68bb-4b0e-a2da-cc5229e9a761 admin admin] RBAC: Authorization granted {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 16:12:13.385590 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99c04b72-08f3-4c62-a15a-8bfe8ffef325 admin admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 16:12:13.389350 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f68c0fe-9425-4d4c-81ea-bd62fe17087c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79706d480f864998b9f8594be407ceab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4ee0413f9ef4d4fb1f95a660babe090', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 16:12:13.389966 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f68c0fe-9425-4d4c-81ea-bd62fe17087c service cinder] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:12:13.390258 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f68c0fe-9425-4d4c-81ea-bd62fe17087c service cinder] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:12:13.390511 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f68c0fe-9425-4d4c-81ea-bd62fe17087c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:12:13.392377 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-6f68c0fe-9425-4d4c-81ea-bd62fe17087c service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 16:12:13.460598 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f68c0fe-9425-4d4c-81ea-bd62fe17087c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 16:12:13.462097 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f68c0fe-9425-4d4c-81ea-bd62fe17087c service cinder] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 16:12:13.472160 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 1141/2264] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Aug 22 16:12:13 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 146 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 16:12:13.473620 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-345a060c-7387-4c13-8db0-4470dc3e200e None None] REQUEST_METHOD: `POST` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:12:13.473964 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-345a060c-7387-4c13-8db0-4470dc3e200e None None] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:12:13.474247 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-345a060c-7387-4c13-8db0-4470dc3e200e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:12:13.494080 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.auth.core [None req-345a060c-7387-4c13-8db0-4470dc3e200e None None] MFA Rules not processed for user `6b2199ae0fb0480aa917ada391f5ec54`. Rule list: `[]` (Enabled: `True`). {{(pid=81768) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 16:12:13.495357 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-345a060c-7387-4c13-8db0-4470dc3e200e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 16:12:13.525901 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.notifications [None req-00d43087-68bb-4b0e-a2da-cc5229e9a761 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0e52755517194970ac5a9652bf24827a'} {{(pid=81767) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 16:12:13.525901 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.notifications [None req-00d43087-68bb-4b0e-a2da-cc5229e9a761 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0e52755517194970ac5a9652bf24827a'} {{(pid=81767) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 16:12:13.559586 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 1142/2265] 10.222.0.15 () {58 vars in 958 bytes} [Mon Aug 22 16:12:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 16:12:13.572883 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-345a060c-7387-4c13-8db0-4470dc3e200e None None] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 16:12:13.574429 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-345a060c-7387-4c13-8db0-4470dc3e200e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 16:12:13.615650 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.notifications [None req-00d43087-68bb-4b0e-a2da-cc5229e9a761 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0e52755517194970ac5a9652bf24827a'} {{(pid=81767) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 16:12:13.639403 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-345a060c-7387-4c13-8db0-4470dc3e200e None None] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 16:12:13.641239 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.notifications [None req-00d43087-68bb-4b0e-a2da-cc5229e9a761 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0e52755517194970ac5a9652bf24827a'} {{(pid=81767) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 16:12:13.643811 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33a6f455-8ef0-415e-be75-6c6fe3fb5920 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '234687adad594f1aa45775a565ffbdc7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4ee0413f9ef4d4fb1f95a660babe090', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 16:12:13.644424 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33a6f455-8ef0-415e-be75-6c6fe3fb5920 service neutron] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:12:13.644703 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33a6f455-8ef0-415e-be75-6c6fe3fb5920 service neutron] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:12:13.644977 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33a6f455-8ef0-415e-be75-6c6fe3fb5920 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:12:13.656469 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33a6f455-8ef0-415e-be75-6c6fe3fb5920 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 16:12:13.659214 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33a6f455-8ef0-415e-be75-6c6fe3fb5920 service neutron] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 16:12:13.664492 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.notifications [None req-00d43087-68bb-4b0e-a2da-cc5229e9a761 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0e52755517194970ac5a9652bf24827a'} {{(pid=81767) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 16:12:13.673924 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 1143/2267] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Aug 22 16:12:13 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 22 16:12:13.674459 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 1124/2267] 10.222.0.15 () {58 vars in 1224 bytes} [Mon Aug 22 16:12:13 2022] DELETE /identity/v3/users/0e52755517194970ac5a9652bf24827a => generated 0 bytes in 425 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 16:12:13.775823 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG dbcounter [-] [81767] Writing DB stats keystone:SELECT=100 {{(pid=81767) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 16:12:14.147113 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG dbcounter [-] [81767] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81767) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 16:12:14.264522 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33a6f455-8ef0-415e-be75-6c6fe3fb5920 service neutron] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 16:12:14.264522 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-33a6f455-8ef0-415e-be75-6c6fe3fb5920 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 16:12:14.280053 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33a6f455-8ef0-415e-be75-6c6fe3fb5920 service neutron] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 16:12:14.342584 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39cea49f-4769-4e43-be69-b7fc790c9f0b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a98b1e4431940ab8d1fc95df2858a6c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 16:12:14.343224 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39cea49f-4769-4e43-be69-b7fc790c9f0b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:12:14.343545 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39cea49f-4769-4e43-be69-b7fc790c9f0b admin admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:12:14.343812 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39cea49f-4769-4e43-be69-b7fc790c9f0b admin admin] PATH_INFO: `/v3/projects/0fccbf3a45f24f2eaf8d24d6513c148a` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:12:14.345084 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39cea49f-4769-4e43-be69-b7fc790c9f0b admin admin] RBAC: Authorizing `identity:delete_project(project_id=0fccbf3a45f24f2eaf8d24d6513c148a)` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 16:12:14.347077 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39cea49f-4769-4e43-be69-b7fc790c9f0b admin admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 16:12:14.389640 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00d43087-68bb-4b0e-a2da-cc5229e9a761 admin admin] Authenticating user token {{(pid=81767) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 16:12:14.391525 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.fernet_utils [None req-00d43087-68bb-4b0e-a2da-cc5229e9a761 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81767) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 16:12:14.414577 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00d43087-68bb-4b0e-a2da-cc5229e9a761 admin admin] Validating token access rules against request {{(pid=81767) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 16:12:14.420516 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0881a0a0-f802-46c5-a5f2-bddd22822e11 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a98b1e4431940ab8d1fc95df2858a6c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81767) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 16:12:14.421419 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0881a0a0-f802-46c5-a5f2-bddd22822e11 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:12:14.421900 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0881a0a0-f802-46c5-a5f2-bddd22822e11 admin admin] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:12:14.422242 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0881a0a0-f802-46c5-a5f2-bddd22822e11 admin admin] PATH_INFO: `/v3/projects/0f9b4fe7ee53496db2381767acec2fed` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:12:14.423559 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0881a0a0-f802-46c5-a5f2-bddd22822e11 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0f9b4fe7ee53496db2381767acec2fed)` {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 16:12:14.425605 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0881a0a0-f802-46c5-a5f2-bddd22822e11 admin admin] RBAC: Authorization granted {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 16:12:14.502054 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-39cea49f-4769-4e43-be69-b7fc790c9f0b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0fccbf3a45f24f2eaf8d24d6513c148a', 'request_id': 'req-39cea49f-4769-4e43-be69-b7fc790c9f0b'} {{(pid=81768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 16:12:14.508547 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-39cea49f-4769-4e43-be69-b7fc790c9f0b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0fccbf3a45f24f2eaf8d24d6513c148a', 'request_id': 'req-39cea49f-4769-4e43-be69-b7fc790c9f0b'} {{(pid=81768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 16:12:14.518158 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 16:12:14.559724 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.notifications [None req-0881a0a0-f802-46c5-a5f2-bddd22822e11 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0f9b4fe7ee53496db2381767acec2fed', 'request_id': 'req-0881a0a0-f802-46c5-a5f2-bddd22822e11'} {{(pid=81767) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 16:12:14.569348 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.notifications [None req-0881a0a0-f802-46c5-a5f2-bddd22822e11 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0f9b4fe7ee53496db2381767acec2fed', 'request_id': 'req-0881a0a0-f802-46c5-a5f2-bddd22822e11'} {{(pid=81767) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 16:12:14.586575 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-39cea49f-4769-4e43-be69-b7fc790c9f0b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0fccbf3a45f24f2eaf8d24d6513c148a', 'request_id': 'req-39cea49f-4769-4e43-be69-b7fc790c9f0b'} {{(pid=81768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 16:12:14.599247 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-39cea49f-4769-4e43-be69-b7fc790c9f0b admin admin] The token cache is being invalidate because project 0fccbf3a45f24f2eaf8d24d6513c148a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81768) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 22 16:12:14.599564 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-39cea49f-4769-4e43-be69-b7fc790c9f0b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 16:12:14.600223 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-39cea49f-4769-4e43-be69-b7fc790c9f0b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 16:12:14.601774 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 1144/2268] 10.222.0.15 () {58 vars in 1233 bytes} [Mon Aug 22 16:12:14 2022] DELETE /identity/v3/projects/0fccbf3a45f24f2eaf8d24d6513c148a => generated 0 bytes in 340 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 16:12:14.607001 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c71247ae-cd3e-4ff5-8902-f8ccb0efa5ad None None] REQUEST_METHOD: `POST` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:12:14.607332 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c71247ae-cd3e-4ff5-8902-f8ccb0efa5ad None None] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:12:14.607655 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c71247ae-cd3e-4ff5-8902-f8ccb0efa5ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:12:14.628479 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.auth.core [None req-c71247ae-cd3e-4ff5-8902-f8ccb0efa5ad None None] MFA Rules not processed for user `6b2199ae0fb0480aa917ada391f5ec54`. Rule list: `[]` (Enabled: `True`). {{(pid=81768) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 16:12:14.629683 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-c71247ae-cd3e-4ff5-8902-f8ccb0efa5ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 16:12:14.678039 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.notifications [None req-0881a0a0-f802-46c5-a5f2-bddd22822e11 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0f9b4fe7ee53496db2381767acec2fed', 'request_id': 'req-0881a0a0-f802-46c5-a5f2-bddd22822e11'} {{(pid=81767) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 16:12:14.692624 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 1145/2269] 10.222.0.15 () {58 vars in 958 bytes} [Mon Aug 22 16:12:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 16:12:14.700446 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.notifications [None req-0881a0a0-f802-46c5-a5f2-bddd22822e11 admin admin] The token cache is being invalidate because project 0f9b4fe7ee53496db2381767acec2fed was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81767) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 22 16:12:14.700446 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.notifications [None req-0881a0a0-f802-46c5-a5f2-bddd22822e11 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81767) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 16:12:14.701156 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.notifications [None req-0881a0a0-f802-46c5-a5f2-bddd22822e11 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81767) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 16:12:14.702603 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8217fca2-ac4e-47cd-847f-c5ebea4077f4 None None] REQUEST_METHOD: `POST` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:12:14.702868 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 1125/2270] 10.222.0.15 () {58 vars in 1233 bytes} [Mon Aug 22 16:12:14 2022] DELETE /identity/v3/projects/0f9b4fe7ee53496db2381767acec2fed => generated 0 bytes in 314 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 22 16:12:14.703095 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8217fca2-ac4e-47cd-847f-c5ebea4077f4 None None] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:12:14.703557 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8217fca2-ac4e-47cd-847f-c5ebea4077f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:12:14.707589 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0881a0a0-f802-46c5-a5f2-bddd22822e11 admin admin] Authenticating user token {{(pid=81767) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 16:12:14.709403 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.fernet_utils [None req-0881a0a0-f802-46c5-a5f2-bddd22822e11 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81767) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 16:12:14.723437 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0881a0a0-f802-46c5-a5f2-bddd22822e11 admin admin] Validating token access rules against request {{(pid=81767) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 16:12:14.726470 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.auth.core [None req-8217fca2-ac4e-47cd-847f-c5ebea4077f4 None None] MFA Rules not processed for user `6b2199ae0fb0480aa917ada391f5ec54`. Rule list: `[]` (Enabled: `True`). {{(pid=81768) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 16:12:14.727748 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-8217fca2-ac4e-47cd-847f-c5ebea4077f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 16:12:14.728721 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fa7daf7-0334-4825-9ddf-921e90460500 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a98b1e4431940ab8d1fc95df2858a6c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81767) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 16:12:14.729754 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fa7daf7-0334-4825-9ddf-921e90460500 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:12:14.729754 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fa7daf7-0334-4825-9ddf-921e90460500 admin admin] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:12:14.729977 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fa7daf7-0334-4825-9ddf-921e90460500 admin admin] PATH_INFO: `/v3/users/eac3491f5c77473db709eba84163575c` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:12:14.731033 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fa7daf7-0334-4825-9ddf-921e90460500 admin admin] RBAC: Authorizing `identity:delete_user(user_id=eac3491f5c77473db709eba84163575c)` {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 16:12:14.732980 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fa7daf7-0334-4825-9ddf-921e90460500 admin admin] RBAC: Authorization granted {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 16:12:14.736314 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 1146/2271] 10.222.0.15 () {58 vars in 958 bytes} [Mon Aug 22 16:12:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 22 16:12:14.741204 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8217fca2-ac4e-47cd-847f-c5ebea4077f4 None None] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 16:12:14.751159 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8217fca2-ac4e-47cd-847f-c5ebea4077f4 None None] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 16:12:14.756714 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2568587-3117-405a-86b2-0f010ae0810c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a98b1e4431940ab8d1fc95df2858a6c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 16:12:14.757340 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2568587-3117-405a-86b2-0f010ae0810c admin admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:12:14.757651 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2568587-3117-405a-86b2-0f010ae0810c admin admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:12:14.757924 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2568587-3117-405a-86b2-0f010ae0810c admin admin] PATH_INFO: `/v3/domains` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:12:14.758612 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2568587-3117-405a-86b2-0f010ae0810c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81768) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 16:12:14.758911 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2568587-3117-405a-86b2-0f010ae0810c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 16:12:14.760738 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2568587-3117-405a-86b2-0f010ae0810c admin admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 16:12:14.773385 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 1147/2272] 10.222.0.15 () {58 vars in 1153 bytes} [Mon Aug 22 16:12:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 22 16:12:14.796774 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2568587-3117-405a-86b2-0f010ae0810c admin admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 16:12:14.796774 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2568587-3117-405a-86b2-0f010ae0810c admin admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 16:12:14.796774 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5815abf9-9b96-4bd6-ae0b-316cd9bb36aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a98b1e4431940ab8d1fc95df2858a6c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 16:12:14.796774 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5815abf9-9b96-4bd6-ae0b-316cd9bb36aa admin admin] REQUEST_METHOD: `POST` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:12:14.797218 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5815abf9-9b96-4bd6-ae0b-316cd9bb36aa admin admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:12:14.797218 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5815abf9-9b96-4bd6-ae0b-316cd9bb36aa admin admin] PATH_INFO: `/v3/projects` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:12:14.797218 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5815abf9-9b96-4bd6-ae0b-316cd9bb36aa admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 16:12:14.797218 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5815abf9-9b96-4bd6-ae0b-316cd9bb36aa admin admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 16:12:14.916951 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 1148/2273] 10.222.0.15 () {60 vars in 1153 bytes} [Mon Aug 22 16:12:14 2022] POST /identity/v3/projects => generated 387 bytes in 132 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 22 16:12:14.919659 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5815abf9-9b96-4bd6-ae0b-316cd9bb36aa admin admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 16:12:14.981168 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.notifications [None req-7fa7daf7-0334-4825-9ddf-921e90460500 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'eac3491f5c77473db709eba84163575c'} {{(pid=81767) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 16:12:14.981871 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.notifications [None req-7fa7daf7-0334-4825-9ddf-921e90460500 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'eac3491f5c77473db709eba84163575c'} {{(pid=81767) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 16:12:15.001342 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5815abf9-9b96-4bd6-ae0b-316cd9bb36aa admin admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 16:12:15.006935 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-739c15dc-c50e-4890-b3f3-ff7dac904034 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a98b1e4431940ab8d1fc95df2858a6c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 16:12:15.007575 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-739c15dc-c50e-4890-b3f3-ff7dac904034 admin admin] REQUEST_METHOD: `POST` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:12:15.007850 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-739c15dc-c50e-4890-b3f3-ff7dac904034 admin admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:12:15.008118 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-739c15dc-c50e-4890-b3f3-ff7dac904034 admin admin] PATH_INFO: `/v3/users` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:12:15.008736 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-739c15dc-c50e-4890-b3f3-ff7dac904034 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 16:12:15.010690 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-739c15dc-c50e-4890-b3f3-ff7dac904034 admin admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 16:12:15.041687 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.notifications [None req-7fa7daf7-0334-4825-9ddf-921e90460500 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'eac3491f5c77473db709eba84163575c'} {{(pid=81767) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 16:12:15.050911 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.notifications [None req-7fa7daf7-0334-4825-9ddf-921e90460500 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'eac3491f5c77473db709eba84163575c'} {{(pid=81767) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 16:12:15.063487 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.notifications [None req-7fa7daf7-0334-4825-9ddf-921e90460500 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'eac3491f5c77473db709eba84163575c'} {{(pid=81767) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 16:12:15.068223 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG dbcounter [-] [81767] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81767) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 16:12:15.075216 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 1126/2274] 10.222.0.15 () {58 vars in 1224 bytes} [Mon Aug 22 16:12:14 2022] DELETE /identity/v3/users/eac3491f5c77473db709eba84163575c => generated 0 bytes in 367 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 16:12:15.108739 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 1149/2275] 10.222.0.15 () {60 vars in 1144 bytes} [Mon Aug 22 16:12:14 2022] POST /identity/v3/users => generated 294 bytes in 186 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 22 16:12:15.114433 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fa7daf7-0334-4825-9ddf-921e90460500 admin admin] Authenticating user token {{(pid=81767) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 16:12:15.114433 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.fernet_utils [None req-7fa7daf7-0334-4825-9ddf-921e90460500 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81767) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 16:12:15.184625 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fa7daf7-0334-4825-9ddf-921e90460500 admin admin] Validating token access rules against request {{(pid=81767) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 16:12:15.189474 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c12ea1d-201a-4ef0-8dda-11cbe5c2374d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a98b1e4431940ab8d1fc95df2858a6c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81767) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 16:12:15.190109 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c12ea1d-201a-4ef0-8dda-11cbe5c2374d admin admin] REQUEST_METHOD: `GET` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:12:15.190390 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c12ea1d-201a-4ef0-8dda-11cbe5c2374d admin admin] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:12:15.190665 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c12ea1d-201a-4ef0-8dda-11cbe5c2374d admin admin] PATH_INFO: `/v3/roles` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:12:15.191334 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c12ea1d-201a-4ef0-8dda-11cbe5c2374d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 16:12:15.192985 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c12ea1d-201a-4ef0-8dda-11cbe5c2374d admin admin] RBAC: Authorization granted {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 16:12:15.203569 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 1127/2276] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Aug 22 16:12:15 2022] GET /identity/v3/roles => generated 1420 bytes in 94 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 22 16:12:15.208340 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-739c15dc-c50e-4890-b3f3-ff7dac904034 admin admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 16:12:15.218964 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-739c15dc-c50e-4890-b3f3-ff7dac904034 admin admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 16:12:15.224424 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5f2d471-a46f-4dc9-b4f7-597aa244f41b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a98b1e4431940ab8d1fc95df2858a6c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 16:12:15.225025 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5f2d471-a46f-4dc9-b4f7-597aa244f41b admin admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:12:15.225303 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5f2d471-a46f-4dc9-b4f7-597aa244f41b admin admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:12:15.225572 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5f2d471-a46f-4dc9-b4f7-597aa244f41b admin admin] PATH_INFO: `/v3/roles` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:12:15.226210 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5f2d471-a46f-4dc9-b4f7-597aa244f41b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 16:12:15.228235 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5f2d471-a46f-4dc9-b4f7-597aa244f41b admin admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 16:12:15.238653 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 1150/2277] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Aug 22 16:12:15 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 22 16:12:15.243092 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c12ea1d-201a-4ef0-8dda-11cbe5c2374d admin admin] Authenticating user token {{(pid=81767) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 16:12:15.253658 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c12ea1d-201a-4ef0-8dda-11cbe5c2374d admin admin] Validating token access rules against request {{(pid=81767) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 16:12:15.258405 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0db363d4-b362-4f92-9b97-4111b0dca762 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a98b1e4431940ab8d1fc95df2858a6c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81767) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 16:12:15.259081 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db363d4-b362-4f92-9b97-4111b0dca762 admin admin] REQUEST_METHOD: `PUT` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:12:15.259393 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db363d4-b362-4f92-9b97-4111b0dca762 admin admin] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:12:15.259672 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db363d4-b362-4f92-9b97-4111b0dca762 admin admin] PATH_INFO: `/v3/projects/b3eb08b95c9c46e9b95ba6feb59019ae/users/ff43dbeb065442c1945fdf4a72425db3/roles/e6fc1ee3fb974125ba9b4fe55b12b45e` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:12:15.276629 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0db363d4-b362-4f92-9b97-4111b0dca762 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b3eb08b95c9c46e9b95ba6feb59019ae, user_id=ff43dbeb065442c1945fdf4a72425db3, role_id=e6fc1ee3fb974125ba9b4fe55b12b45e)` {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 16:12:15.276751 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 16:12:15.281857 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0db363d4-b362-4f92-9b97-4111b0dca762 admin admin] RBAC: Authorization granted {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 16:12:15.369636 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 1128/2278] 10.222.0.15 () {60 vars in 1483 bytes} [Mon Aug 22 16:12:15 2022] PUT /identity/v3/projects/b3eb08b95c9c46e9b95ba6feb59019ae/users/ff43dbeb065442c1945fdf4a72425db3/roles/e6fc1ee3fb974125ba9b4fe55b12b45e => generated 0 bytes in 127 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 16:12:15.381658 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c25aa7e6-ec29-40e5-a554-f32127ece0a1 None None] REQUEST_METHOD: `POST` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:12:15.381943 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c25aa7e6-ec29-40e5-a554-f32127ece0a1 None None] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:12:15.382412 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c25aa7e6-ec29-40e5-a554-f32127ece0a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:12:15.412752 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.auth.core [None req-c25aa7e6-ec29-40e5-a554-f32127ece0a1 None None] MFA Rules not processed for user `ff43dbeb065442c1945fdf4a72425db3`. Rule list: `[]` (Enabled: `True`). {{(pid=81768) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 16:12:15.414071 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-c25aa7e6-ec29-40e5-a554-f32127ece0a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 16:12:15.461044 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0db363d4-b362-4f92-9b97-4111b0dca762 admin admin] Authenticating user token {{(pid=81767) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 16:12:15.463716 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.fernet_utils [None req-0db363d4-b362-4f92-9b97-4111b0dca762 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81767) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 16:12:15.495510 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 1151/2279] 10.222.0.15 () {58 vars in 958 bytes} [Mon Aug 22 16:12:15 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 16:12:15.495510 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c25aa7e6-ec29-40e5-a554-f32127ece0a1 None None] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 16:12:15.539002 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0db363d4-b362-4f92-9b97-4111b0dca762 admin admin] Validating token access rules against request {{(pid=81767) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 16:12:15.544288 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-732520fa-8c37-4471-bf62-925e97af1275 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a98b1e4431940ab8d1fc95df2858a6c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81767) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 16:12:15.545173 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-732520fa-8c37-4471-bf62-925e97af1275 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:12:15.545589 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-732520fa-8c37-4471-bf62-925e97af1275 admin admin] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:12:15.545877 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-732520fa-8c37-4471-bf62-925e97af1275 admin admin] PATH_INFO: `/v3/users/6199a95ab49f400ca53c846e292a4712` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:12:15.547468 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-732520fa-8c37-4471-bf62-925e97af1275 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6199a95ab49f400ca53c846e292a4712)` {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 16:12:15.553699 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-732520fa-8c37-4471-bf62-925e97af1275 admin admin] RBAC: Authorization granted {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 16:12:15.569589 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c25aa7e6-ec29-40e5-a554-f32127ece0a1 None None] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 16:12:15.575348 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4608d4d9-e7d2-4a29-bc6e-576957f7bf8c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a98b1e4431940ab8d1fc95df2858a6c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 16:12:15.576062 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4608d4d9-e7d2-4a29-bc6e-576957f7bf8c admin admin] REQUEST_METHOD: `POST` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:12:15.576398 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4608d4d9-e7d2-4a29-bc6e-576957f7bf8c admin admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:12:15.576732 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4608d4d9-e7d2-4a29-bc6e-576957f7bf8c admin admin] PATH_INFO: `/v3/projects` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:12:15.577412 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4608d4d9-e7d2-4a29-bc6e-576957f7bf8c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 16:12:15.579413 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4608d4d9-e7d2-4a29-bc6e-576957f7bf8c admin admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 16:12:15.701383 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 1152/2280] 10.222.0.15 () {60 vars in 1153 bytes} [Mon Aug 22 16:12:15 2022] POST /identity/v3/projects => generated 387 bytes in 206 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 22 16:12:15.701383 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4608d4d9-e7d2-4a29-bc6e-576957f7bf8c admin admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 16:12:15.702087 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG dbcounter [-] [81767] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=81767) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 16:12:15.702188 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-4608d4d9-e7d2-4a29-bc6e-576957f7bf8c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 16:12:15.766176 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4608d4d9-e7d2-4a29-bc6e-576957f7bf8c admin admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 16:12:15.770779 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcb3360f-3a39-4a8b-bc03-0b8d2ce9e66d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a98b1e4431940ab8d1fc95df2858a6c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 16:12:15.771425 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcb3360f-3a39-4a8b-bc03-0b8d2ce9e66d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:12:15.771700 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcb3360f-3a39-4a8b-bc03-0b8d2ce9e66d admin admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:12:15.771965 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcb3360f-3a39-4a8b-bc03-0b8d2ce9e66d admin admin] PATH_INFO: `/v3/projects/a610ea9550694405817a23dc4fb53351` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:12:15.773055 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcb3360f-3a39-4a8b-bc03-0b8d2ce9e66d admin admin] RBAC: Authorizing `identity:delete_project(project_id=a610ea9550694405817a23dc4fb53351)` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 16:12:15.774949 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcb3360f-3a39-4a8b-bc03-0b8d2ce9e66d admin admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 16:12:15.834962 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.notifications [None req-732520fa-8c37-4471-bf62-925e97af1275 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6199a95ab49f400ca53c846e292a4712'} {{(pid=81767) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 16:12:15.835594 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.notifications [None req-732520fa-8c37-4471-bf62-925e97af1275 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6199a95ab49f400ca53c846e292a4712'} {{(pid=81767) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 16:12:15.895029 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-fcb3360f-3a39-4a8b-bc03-0b8d2ce9e66d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a610ea9550694405817a23dc4fb53351', 'request_id': 'req-fcb3360f-3a39-4a8b-bc03-0b8d2ce9e66d'} {{(pid=81768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 16:12:15.900783 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.notifications [None req-732520fa-8c37-4471-bf62-925e97af1275 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6199a95ab49f400ca53c846e292a4712'} {{(pid=81767) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 16:12:15.901130 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-fcb3360f-3a39-4a8b-bc03-0b8d2ce9e66d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a610ea9550694405817a23dc4fb53351', 'request_id': 'req-fcb3360f-3a39-4a8b-bc03-0b8d2ce9e66d'} {{(pid=81768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 16:12:15.908771 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.notifications [None req-732520fa-8c37-4471-bf62-925e97af1275 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6199a95ab49f400ca53c846e292a4712'} {{(pid=81767) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 16:12:15.920975 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.notifications [None req-732520fa-8c37-4471-bf62-925e97af1275 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6199a95ab49f400ca53c846e292a4712'} {{(pid=81767) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 16:12:15.927759 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 1129/2281] 10.222.0.15 () {58 vars in 1224 bytes} [Mon Aug 22 16:12:15 2022] DELETE /identity/v3/users/6199a95ab49f400ca53c846e292a4712 => generated 0 bytes in 467 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 22 16:12:15.928540 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-732520fa-8c37-4471-bf62-925e97af1275 admin admin] Authenticating user token {{(pid=81767) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 16:12:15.929915 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.fernet_utils [None req-732520fa-8c37-4471-bf62-925e97af1275 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81767) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 16:12:15.980678 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-fcb3360f-3a39-4a8b-bc03-0b8d2ce9e66d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a610ea9550694405817a23dc4fb53351', 'request_id': 'req-fcb3360f-3a39-4a8b-bc03-0b8d2ce9e66d'} {{(pid=81768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 16:12:15.993213 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-fcb3360f-3a39-4a8b-bc03-0b8d2ce9e66d admin admin] The token cache is being invalidate because project a610ea9550694405817a23dc4fb53351 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81768) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 22 16:12:15.993593 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-fcb3360f-3a39-4a8b-bc03-0b8d2ce9e66d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 16:12:15.994318 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-fcb3360f-3a39-4a8b-bc03-0b8d2ce9e66d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 16:12:15.996069 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 1153/2282] 10.222.0.15 () {58 vars in 1233 bytes} [Mon Aug 22 16:12:15 2022] DELETE /identity/v3/projects/a610ea9550694405817a23dc4fb53351 => generated 0 bytes in 297 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 16:12:15.999226 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-732520fa-8c37-4471-bf62-925e97af1275 admin admin] Validating token access rules against request {{(pid=81767) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 16:12:16.001201 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f331318e-d74c-4540-9430-3717c748451c None None] REQUEST_METHOD: `POST` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:12:16.001553 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f331318e-d74c-4540-9430-3717c748451c None None] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:12:16.002118 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f331318e-d74c-4540-9430-3717c748451c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:12:16.004590 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cba25b4f-8f71-4c87-a3de-f8fbee6b6706 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a98b1e4431940ab8d1fc95df2858a6c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81767) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 16:12:16.005240 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cba25b4f-8f71-4c87-a3de-f8fbee6b6706 admin admin] REQUEST_METHOD: `POST` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:12:16.005523 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cba25b4f-8f71-4c87-a3de-f8fbee6b6706 admin admin] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:12:16.005792 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cba25b4f-8f71-4c87-a3de-f8fbee6b6706 admin admin] PATH_INFO: `/v3/users` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:12:16.006412 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cba25b4f-8f71-4c87-a3de-f8fbee6b6706 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 16:12:16.008359 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cba25b4f-8f71-4c87-a3de-f8fbee6b6706 admin admin] RBAC: Authorization granted {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 16:12:16.028018 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.auth.core [None req-f331318e-d74c-4540-9430-3717c748451c None None] MFA Rules not processed for user `6b2199ae0fb0480aa917ada391f5ec54`. Rule list: `[]` (Enabled: `True`). {{(pid=81768) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 16:12:16.028018 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-f331318e-d74c-4540-9430-3717c748451c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 16:12:16.034198 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 1154/2283] 10.222.0.15 () {58 vars in 958 bytes} [Mon Aug 22 16:12:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 16:12:16.035433 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e5915f6-908c-43a3-9667-b14407925ac7 None None] REQUEST_METHOD: `POST` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:12:16.042710 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e5915f6-908c-43a3-9667-b14407925ac7 None None] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:12:16.042710 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e5915f6-908c-43a3-9667-b14407925ac7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:12:16.056245 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.auth.core [None req-6e5915f6-908c-43a3-9667-b14407925ac7 None None] MFA Rules not processed for user `6b2199ae0fb0480aa917ada391f5ec54`. Rule list: `[]` (Enabled: `True`). {{(pid=81768) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 16:12:16.057450 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-6e5915f6-908c-43a3-9667-b14407925ac7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 16:12:16.065299 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 1155/2284] 10.222.0.15 () {58 vars in 958 bytes} [Mon Aug 22 16:12:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 16:12:16.066106 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e5915f6-908c-43a3-9667-b14407925ac7 None None] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 16:12:16.067337 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-6e5915f6-908c-43a3-9667-b14407925ac7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 16:12:16.115347 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 1130/2285] 10.222.0.15 () {60 vars in 1144 bytes} [Mon Aug 22 16:12:15 2022] POST /identity/v3/users => generated 300 bytes in 184 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 22 16:12:16.115347 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd0c1f94-e22d-4665-91dd-25bd27e034e4 None None] REQUEST_METHOD: `POST` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:12:16.116308 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd0c1f94-e22d-4665-91dd-25bd27e034e4 None None] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:12:16.116589 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd0c1f94-e22d-4665-91dd-25bd27e034e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:12:16.125489 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e5915f6-908c-43a3-9667-b14407925ac7 None None] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 16:12:16.129308 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba370884-5f21-40b4-bb3e-005e48447a8c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '234687adad594f1aa45775a565ffbdc7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4ee0413f9ef4d4fb1f95a660babe090', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 16:12:16.129917 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba370884-5f21-40b4-bb3e-005e48447a8c service neutron] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:12:16.130189 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba370884-5f21-40b4-bb3e-005e48447a8c service neutron] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:12:16.130457 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba370884-5f21-40b4-bb3e-005e48447a8c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:12:16.137138 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.auth.core [None req-cd0c1f94-e22d-4665-91dd-25bd27e034e4 None None] MFA Rules not processed for user `6b2199ae0fb0480aa917ada391f5ec54`. Rule list: `[]` (Enabled: `True`). {{(pid=81767) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 16:12:16.138466 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.fernet_utils [None req-cd0c1f94-e22d-4665-91dd-25bd27e034e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81767) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 16:12:16.140537 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba370884-5f21-40b4-bb3e-005e48447a8c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 16:12:16.144143 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba370884-5f21-40b4-bb3e-005e48447a8c service neutron] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 16:12:16.152702 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 1131/2286] 10.222.0.15 () {58 vars in 958 bytes} [Mon Aug 22 16:12:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 16:12:16.152702 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd0c1f94-e22d-4665-91dd-25bd27e034e4 None None] Authenticating user token {{(pid=81767) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 16:12:16.152982 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 1156/2287] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Aug 22 16:12:16 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 16:12:16.153757 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba370884-5f21-40b4-bb3e-005e48447a8c service neutron] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 16:12:16.154203 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.fernet_utils [None req-cd0c1f94-e22d-4665-91dd-25bd27e034e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81767) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 16:12:16.165547 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba370884-5f21-40b4-bb3e-005e48447a8c service neutron] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 16:12:16.171106 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd0c1f94-e22d-4665-91dd-25bd27e034e4 None None] Validating token access rules against request {{(pid=81767) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 16:12:16.171257 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72fb6eaf-4f42-4280-b0e7-db8e2b149bf9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a98b1e4431940ab8d1fc95df2858a6c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 16:12:16.171784 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72fb6eaf-4f42-4280-b0e7-db8e2b149bf9 admin admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:12:16.172070 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72fb6eaf-4f42-4280-b0e7-db8e2b149bf9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:12:16.172474 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72fb6eaf-4f42-4280-b0e7-db8e2b149bf9 admin admin] PATH_INFO: `/v3/domains` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:12:16.173133 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72fb6eaf-4f42-4280-b0e7-db8e2b149bf9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81768) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 16:12:16.173434 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72fb6eaf-4f42-4280-b0e7-db8e2b149bf9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 16:12:16.173941 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60e49df8-34ad-4150-b25e-8c5c477fe920 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a98b1e4431940ab8d1fc95df2858a6c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81767) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 16:12:16.174889 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60e49df8-34ad-4150-b25e-8c5c477fe920 admin admin] REQUEST_METHOD: `GET` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:12:16.175183 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60e49df8-34ad-4150-b25e-8c5c477fe920 admin admin] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:12:16.175326 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72fb6eaf-4f42-4280-b0e7-db8e2b149bf9 admin admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 16:12:16.175801 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60e49df8-34ad-4150-b25e-8c5c477fe920 admin admin] PATH_INFO: `/v3/roles` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:12:16.176551 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60e49df8-34ad-4150-b25e-8c5c477fe920 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 16:12:16.178235 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60e49df8-34ad-4150-b25e-8c5c477fe920 admin admin] RBAC: Authorization granted {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 16:12:16.191684 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 1157/2288] 10.222.0.15 () {58 vars in 1153 bytes} [Mon Aug 22 16:12:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 16:12:16.191886 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 1132/2289] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Aug 22 16:12:16 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 22 16:12:16.192449 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72fb6eaf-4f42-4280-b0e7-db8e2b149bf9 admin admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 16:12:16.193492 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60e49df8-34ad-4150-b25e-8c5c477fe920 admin admin] Authenticating user token {{(pid=81767) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 16:12:16.203981 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60e49df8-34ad-4150-b25e-8c5c477fe920 admin admin] Validating token access rules against request {{(pid=81767) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 16:12:16.204590 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72fb6eaf-4f42-4280-b0e7-db8e2b149bf9 admin admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 16:12:16.209953 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a02abe43-d1e4-4876-8f9a-61e08820897e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a98b1e4431940ab8d1fc95df2858a6c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81767) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 16:12:16.210570 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3877515e-7c93-40b8-976f-8699e9fd87cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a98b1e4431940ab8d1fc95df2858a6c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 16:12:16.210700 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a02abe43-d1e4-4876-8f9a-61e08820897e admin admin] REQUEST_METHOD: `PUT` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:12:16.210885 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a02abe43-d1e4-4876-8f9a-61e08820897e admin admin] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:12:16.211157 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a02abe43-d1e4-4876-8f9a-61e08820897e admin admin] PATH_INFO: `/v3/projects/5bd8c32ccb2e4056bd363b8d695c2962/users/ebe6ac072762420184313995825f90c9/roles/9d9848cfd22e4b089db443c350e56670` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:12:16.211571 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3877515e-7c93-40b8-976f-8699e9fd87cb admin admin] REQUEST_METHOD: `POST` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:12:16.211972 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3877515e-7c93-40b8-976f-8699e9fd87cb admin admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:12:16.212301 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3877515e-7c93-40b8-976f-8699e9fd87cb admin admin] PATH_INFO: `/v3/projects` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:12:16.213022 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3877515e-7c93-40b8-976f-8699e9fd87cb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 16:12:16.215193 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3877515e-7c93-40b8-976f-8699e9fd87cb admin admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 16:12:16.228494 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a02abe43-d1e4-4876-8f9a-61e08820897e admin admin] RBAC: Authorizing `identity:create_grant(project_id=5bd8c32ccb2e4056bd363b8d695c2962, user_id=ebe6ac072762420184313995825f90c9, role_id=9d9848cfd22e4b089db443c350e56670)` {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 16:12:16.230993 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=82,keystone:DELETE=16,keystone:INSERT=2 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 16:12:16.231239 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a02abe43-d1e4-4876-8f9a-61e08820897e admin admin] RBAC: Authorization granted {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 16:12:16.299250 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 1133/2290] 10.222.0.15 () {60 vars in 1483 bytes} [Mon Aug 22 16:12:16 2022] PUT /identity/v3/projects/5bd8c32ccb2e4056bd363b8d695c2962/users/ebe6ac072762420184313995825f90c9/roles/9d9848cfd22e4b089db443c350e56670 => generated 0 bytes in 106 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 16:12:16.303240 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a02abe43-d1e4-4876-8f9a-61e08820897e admin admin] Authenticating user token {{(pid=81767) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 16:12:16.304246 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 1158/2291] 10.222.0.15 () {60 vars in 1153 bytes} [Mon Aug 22 16:12:16 2022] POST /identity/v3/projects => generated 373 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 22 16:12:16.310931 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3877515e-7c93-40b8-976f-8699e9fd87cb admin admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 16:12:16.372935 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a02abe43-d1e4-4876-8f9a-61e08820897e admin admin] Validating token access rules against request {{(pid=81767) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 16:12:16.386325 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3877515e-7c93-40b8-976f-8699e9fd87cb admin admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 16:12:16.391722 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-807faa33-5ead-4dac-b354-292939fc51ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a98b1e4431940ab8d1fc95df2858a6c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 16:12:16.392345 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-807faa33-5ead-4dac-b354-292939fc51ad admin admin] REQUEST_METHOD: `POST` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:12:16.392669 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-807faa33-5ead-4dac-b354-292939fc51ad admin admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:12:16.392941 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-807faa33-5ead-4dac-b354-292939fc51ad admin admin] PATH_INFO: `/v3/users` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:12:16.394836 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-807faa33-5ead-4dac-b354-292939fc51ad admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 16:12:16.395476 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-807faa33-5ead-4dac-b354-292939fc51ad admin admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 16:12:16.435990 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7c35ddb-17e5-4dcf-b504-f82c7ba4b369 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a98b1e4431940ab8d1fc95df2858a6c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81767) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 16:12:16.436639 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7c35ddb-17e5-4dcf-b504-f82c7ba4b369 admin admin] REQUEST_METHOD: `GET` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:12:16.436931 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7c35ddb-17e5-4dcf-b504-f82c7ba4b369 admin admin] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:12:16.437203 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7c35ddb-17e5-4dcf-b504-f82c7ba4b369 admin admin] PATH_INFO: `/v3/roles` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:12:16.437852 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7c35ddb-17e5-4dcf-b504-f82c7ba4b369 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 16:12:16.440069 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7c35ddb-17e5-4dcf-b504-f82c7ba4b369 admin admin] RBAC: Authorization granted {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 16:12:16.451619 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 1134/2292] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Aug 22 16:12:16 2022] GET /identity/v3/roles => generated 1420 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 22 16:12:16.461058 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7c35ddb-17e5-4dcf-b504-f82c7ba4b369 admin admin] Authenticating user token {{(pid=81767) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 16:12:16.471420 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7c35ddb-17e5-4dcf-b504-f82c7ba4b369 admin admin] Validating token access rules against request {{(pid=81767) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 16:12:16.476201 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-790b2428-7f35-4ed3-a929-ca1634b4782e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a98b1e4431940ab8d1fc95df2858a6c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81767) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 16:12:16.476866 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-790b2428-7f35-4ed3-a929-ca1634b4782e admin admin] REQUEST_METHOD: `PUT` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:12:16.477152 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-790b2428-7f35-4ed3-a929-ca1634b4782e admin admin] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:12:16.477424 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-790b2428-7f35-4ed3-a929-ca1634b4782e admin admin] PATH_INFO: `/v3/projects/5bd8c32ccb2e4056bd363b8d695c2962/users/ebe6ac072762420184313995825f90c9/roles/9d9848cfd22e4b089db443c350e56670` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:12:16.479405 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-790b2428-7f35-4ed3-a929-ca1634b4782e admin admin] RBAC: Authorizing `identity:create_grant(project_id=5bd8c32ccb2e4056bd363b8d695c2962, user_id=ebe6ac072762420184313995825f90c9, role_id=9d9848cfd22e4b089db443c350e56670)` {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 16:12:16.483697 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-790b2428-7f35-4ed3-a929-ca1634b4782e admin admin] RBAC: Authorization granted {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 16:12:16.488680 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 1159/2293] 10.222.0.15 () {60 vars in 1144 bytes} [Mon Aug 22 16:12:16 2022] POST /identity/v3/users => generated 287 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 22 16:12:16.492638 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-807faa33-5ead-4dac-b354-292939fc51ad admin admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 16:12:16.495787 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 1135/2294] 10.222.0.15 () {60 vars in 1483 bytes} [Mon Aug 22 16:12:16 2022] PUT /identity/v3/projects/5bd8c32ccb2e4056bd363b8d695c2962/users/ebe6ac072762420184313995825f90c9/roles/9d9848cfd22e4b089db443c350e56670 => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 22 16:12:16.505015 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-807faa33-5ead-4dac-b354-292939fc51ad admin admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 16:12:16.509145 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acb5a7fe-cd28-46a9-91ac-5aae648dc1a6 None None] REQUEST_METHOD: `POST` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:12:16.509535 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acb5a7fe-cd28-46a9-91ac-5aae648dc1a6 None None] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:12:16.509911 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acb5a7fe-cd28-46a9-91ac-5aae648dc1a6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:12:16.545066 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.auth.core [None req-acb5a7fe-cd28-46a9-91ac-5aae648dc1a6 None None] MFA Rules not processed for user `ebe6ac072762420184313995825f90c9`. Rule list: `[]` (Enabled: `True`). {{(pid=81767) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 16:12:16.546451 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.fernet_utils [None req-acb5a7fe-cd28-46a9-91ac-5aae648dc1a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81767) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 16:12:16.607221 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfe2b52c-bf42-4497-b0bf-f237c2582506 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a98b1e4431940ab8d1fc95df2858a6c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 16:12:16.608041 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfe2b52c-bf42-4497-b0bf-f237c2582506 admin admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:12:16.608317 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfe2b52c-bf42-4497-b0bf-f237c2582506 admin admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:12:16.608589 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfe2b52c-bf42-4497-b0bf-f237c2582506 admin admin] PATH_INFO: `/v3/roles` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:12:16.609244 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfe2b52c-bf42-4497-b0bf-f237c2582506 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 16:12:16.610902 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfe2b52c-bf42-4497-b0bf-f237c2582506 admin admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 16:12:16.624432 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 1160/2295] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Aug 22 16:12:16 2022] GET /identity/v3/roles => generated 1420 bytes in 132 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 22 16:12:16.625478 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfe2b52c-bf42-4497-b0bf-f237c2582506 admin admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 16:12:16.635588 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfe2b52c-bf42-4497-b0bf-f237c2582506 admin admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 16:12:16.636695 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 1136/2296] 10.222.0.15 () {58 vars in 958 bytes} [Mon Aug 22 16:12:16 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 16:12:16.640665 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8f1c603-ea47-4ed5-83b6-0f3f4bbb3806 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a98b1e4431940ab8d1fc95df2858a6c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 16:12:16.643426 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8f1c603-ea47-4ed5-83b6-0f3f4bbb3806 admin admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:12:16.643426 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8f1c603-ea47-4ed5-83b6-0f3f4bbb3806 admin admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:12:16.643426 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8f1c603-ea47-4ed5-83b6-0f3f4bbb3806 admin admin] PATH_INFO: `/v3/roles` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:12:16.643426 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8f1c603-ea47-4ed5-83b6-0f3f4bbb3806 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 16:12:16.644250 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8f1c603-ea47-4ed5-83b6-0f3f4bbb3806 admin admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 16:12:16.651576 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acb5a7fe-cd28-46a9-91ac-5aae648dc1a6 None None] Authenticating user token {{(pid=81767) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 16:12:16.653163 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.fernet_utils [None req-acb5a7fe-cd28-46a9-91ac-5aae648dc1a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81767) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 16:12:16.655119 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 1161/2297] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Aug 22 16:12:16 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 22 16:12:16.659792 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8f1c603-ea47-4ed5-83b6-0f3f4bbb3806 admin admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 16:12:16.670268 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8f1c603-ea47-4ed5-83b6-0f3f4bbb3806 admin admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 16:12:16.676092 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e46fdc23-94da-4d36-b298-d0f10c043692 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a98b1e4431940ab8d1fc95df2858a6c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 16:12:16.676750 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e46fdc23-94da-4d36-b298-d0f10c043692 admin admin] REQUEST_METHOD: `PUT` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:12:16.687445 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e46fdc23-94da-4d36-b298-d0f10c043692 admin admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:12:16.687445 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e46fdc23-94da-4d36-b298-d0f10c043692 admin admin] PATH_INFO: `/v3/projects/d221e15c61bb49189f9380b318fb554e/users/3922ae7a8fb44e11bc3d83a57feb5ea0/roles/e6fc1ee3fb974125ba9b4fe55b12b45e` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:12:16.692483 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e46fdc23-94da-4d36-b298-d0f10c043692 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d221e15c61bb49189f9380b318fb554e, user_id=3922ae7a8fb44e11bc3d83a57feb5ea0, role_id=e6fc1ee3fb974125ba9b4fe55b12b45e)` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 16:12:16.695163 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e46fdc23-94da-4d36-b298-d0f10c043692 admin admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 16:12:16.714962 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acb5a7fe-cd28-46a9-91ac-5aae648dc1a6 None None] Validating token access rules against request {{(pid=81767) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 16:12:16.718888 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-407948d2-09db-459a-8ac0-e3de7fc0c946 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79706d480f864998b9f8594be407ceab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4ee0413f9ef4d4fb1f95a660babe090', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81767) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 16:12:16.719460 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-407948d2-09db-459a-8ac0-e3de7fc0c946 service cinder] REQUEST_METHOD: `GET` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:12:16.722155 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-407948d2-09db-459a-8ac0-e3de7fc0c946 service cinder] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:12:16.722155 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-407948d2-09db-459a-8ac0-e3de7fc0c946 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:12:16.730072 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-407948d2-09db-459a-8ac0-e3de7fc0c946 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 16:12:16.731530 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-407948d2-09db-459a-8ac0-e3de7fc0c946 service cinder] RBAC: Authorization granted {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 16:12:16.741381 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 1137/2298] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Aug 22 16:12:16 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 22 16:12:16.756498 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-407948d2-09db-459a-8ac0-e3de7fc0c946 service cinder] Authenticating user token {{(pid=81767) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 16:12:16.756498 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.fernet_utils [None req-407948d2-09db-459a-8ac0-e3de7fc0c946 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81767) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 16:12:16.764988 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-407948d2-09db-459a-8ac0-e3de7fc0c946 service cinder] Validating token access rules against request {{(pid=81767) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 16:12:16.770497 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaf3dfaf-815e-4ccd-8c9d-cbfb13927b77 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a98b1e4431940ab8d1fc95df2858a6c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81767) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 16:12:16.771179 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaf3dfaf-815e-4ccd-8c9d-cbfb13927b77 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:12:16.771491 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaf3dfaf-815e-4ccd-8c9d-cbfb13927b77 admin admin] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:12:16.771764 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaf3dfaf-815e-4ccd-8c9d-cbfb13927b77 admin admin] PATH_INFO: `/v3/projects/5ac04c67c44b48a19e86f367a1f3acc0` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:12:16.772921 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaf3dfaf-815e-4ccd-8c9d-cbfb13927b77 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5ac04c67c44b48a19e86f367a1f3acc0)` {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 16:12:16.775122 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaf3dfaf-815e-4ccd-8c9d-cbfb13927b77 admin admin] RBAC: Authorization granted {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 16:12:16.786206 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG dbcounter [-] [81767] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=81767) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 16:12:16.787237 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 1162/2299] 10.222.0.15 () {60 vars in 1483 bytes} [Mon Aug 22 16:12:16 2022] PUT /identity/v3/projects/d221e15c61bb49189f9380b318fb554e/users/3922ae7a8fb44e11bc3d83a57feb5ea0/roles/e6fc1ee3fb974125ba9b4fe55b12b45e => generated 0 bytes in 128 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 16:12:16.798858 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef5eb3c1-26b7-43b9-924e-85d7c7d2992e None None] REQUEST_METHOD: `POST` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:12:16.799141 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef5eb3c1-26b7-43b9-924e-85d7c7d2992e None None] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:12:16.799452 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef5eb3c1-26b7-43b9-924e-85d7c7d2992e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:12:16.828233 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.auth.core [None req-ef5eb3c1-26b7-43b9-924e-85d7c7d2992e None None] MFA Rules not processed for user `3922ae7a8fb44e11bc3d83a57feb5ea0`. Rule list: `[]` (Enabled: `True`). {{(pid=81768) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 16:12:16.829436 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-ef5eb3c1-26b7-43b9-924e-85d7c7d2992e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 16:12:16.899424 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 1163/2300] 10.222.0.15 () {58 vars in 958 bytes} [Mon Aug 22 16:12:16 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 22 16:12:16.910807 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.notifications [None req-aaf3dfaf-815e-4ccd-8c9d-cbfb13927b77 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5ac04c67c44b48a19e86f367a1f3acc0', 'request_id': 'req-aaf3dfaf-815e-4ccd-8c9d-cbfb13927b77'} {{(pid=81767) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 16:12:16.911428 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef5eb3c1-26b7-43b9-924e-85d7c7d2992e None None] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 16:12:16.919990 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.notifications [None req-aaf3dfaf-815e-4ccd-8c9d-cbfb13927b77 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5ac04c67c44b48a19e86f367a1f3acc0', 'request_id': 'req-aaf3dfaf-815e-4ccd-8c9d-cbfb13927b77'} {{(pid=81767) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 16:12:16.972990 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef5eb3c1-26b7-43b9-924e-85d7c7d2992e None None] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 16:12:16.986691 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.notifications [None req-aaf3dfaf-815e-4ccd-8c9d-cbfb13927b77 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5ac04c67c44b48a19e86f367a1f3acc0', 'request_id': 'req-aaf3dfaf-815e-4ccd-8c9d-cbfb13927b77'} {{(pid=81767) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 16:12:16.998308 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.notifications [None req-aaf3dfaf-815e-4ccd-8c9d-cbfb13927b77 admin admin] The token cache is being invalidate because project 5ac04c67c44b48a19e86f367a1f3acc0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81767) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 22 16:12:16.998609 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.notifications [None req-aaf3dfaf-815e-4ccd-8c9d-cbfb13927b77 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81767) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 16:12:16.999259 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.notifications [None req-aaf3dfaf-815e-4ccd-8c9d-cbfb13927b77 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81767) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 16:12:17.001106 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 1138/2301] 10.222.0.15 () {58 vars in 1233 bytes} [Mon Aug 22 16:12:16 2022] DELETE /identity/v3/projects/5ac04c67c44b48a19e86f367a1f3acc0 => generated 0 bytes in 253 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 22 16:12:17.004824 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaf3dfaf-815e-4ccd-8c9d-cbfb13927b77 admin admin] Authenticating user token {{(pid=81767) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 16:12:17.006215 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.fernet_utils [None req-aaf3dfaf-815e-4ccd-8c9d-cbfb13927b77 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81767) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 16:12:17.023786 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ce52035-700a-4a6b-9066-b5ef4ccf4047 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79706d480f864998b9f8594be407ceab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4ee0413f9ef4d4fb1f95a660babe090', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 16:12:17.024386 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ce52035-700a-4a6b-9066-b5ef4ccf4047 service cinder] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:12:17.024672 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ce52035-700a-4a6b-9066-b5ef4ccf4047 service cinder] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:12:17.024943 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ce52035-700a-4a6b-9066-b5ef4ccf4047 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:12:17.026724 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-7ce52035-700a-4a6b-9066-b5ef4ccf4047 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 16:12:17.076596 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaf3dfaf-815e-4ccd-8c9d-cbfb13927b77 admin admin] Validating token access rules against request {{(pid=81767) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 16:12:17.082010 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-203f198c-d62a-4004-9a4a-269282dc2152 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a98b1e4431940ab8d1fc95df2858a6c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81767) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 16:12:17.082656 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-203f198c-d62a-4004-9a4a-269282dc2152 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:12:17.085273 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-203f198c-d62a-4004-9a4a-269282dc2152 admin admin] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:12:17.085273 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-203f198c-d62a-4004-9a4a-269282dc2152 admin admin] PATH_INFO: `/v3/users/e5f4c1d845314a7dadd032b792cbdf7d` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:12:17.085273 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-203f198c-d62a-4004-9a4a-269282dc2152 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e5f4c1d845314a7dadd032b792cbdf7d)` {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 16:12:17.086764 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-203f198c-d62a-4004-9a4a-269282dc2152 admin admin] RBAC: Authorization granted {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 16:12:17.107573 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ce52035-700a-4a6b-9066-b5ef4ccf4047 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 16:12:17.107573 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ce52035-700a-4a6b-9066-b5ef4ccf4047 service cinder] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 16:12:17.110792 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 1164/2302] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Aug 22 16:12:16 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 200 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 22 16:12:17.371765 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.notifications [None req-203f198c-d62a-4004-9a4a-269282dc2152 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e5f4c1d845314a7dadd032b792cbdf7d'} {{(pid=81767) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 16:12:17.372647 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.notifications [None req-203f198c-d62a-4004-9a4a-269282dc2152 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e5f4c1d845314a7dadd032b792cbdf7d'} {{(pid=81767) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 16:12:17.461202 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.notifications [None req-203f198c-d62a-4004-9a4a-269282dc2152 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e5f4c1d845314a7dadd032b792cbdf7d'} {{(pid=81767) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 16:12:17.470369 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 16:12:17.470596 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.notifications [None req-203f198c-d62a-4004-9a4a-269282dc2152 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e5f4c1d845314a7dadd032b792cbdf7d'} {{(pid=81767) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 16:12:17.487351 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.notifications [None req-203f198c-d62a-4004-9a4a-269282dc2152 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e5f4c1d845314a7dadd032b792cbdf7d'} {{(pid=81767) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 16:12:17.503152 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 1139/2303] 10.222.0.15 () {58 vars in 1224 bytes} [Mon Aug 22 16:12:17 2022] DELETE /identity/v3/users/e5f4c1d845314a7dadd032b792cbdf7d => generated 0 bytes in 498 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 22 16:12:17.886411 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ce52035-700a-4a6b-9066-b5ef4ccf4047 service cinder] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 16:12:17.909390 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ce52035-700a-4a6b-9066-b5ef4ccf4047 service cinder] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 16:12:17.967755 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a77bfcc2-fce5-4223-ac63-75b6eb1bea70 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a98b1e4431940ab8d1fc95df2858a6c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 16:12:17.968320 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a77bfcc2-fce5-4223-ac63-75b6eb1bea70 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:12:17.968827 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a77bfcc2-fce5-4223-ac63-75b6eb1bea70 admin admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:12:17.969281 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a77bfcc2-fce5-4223-ac63-75b6eb1bea70 admin admin] PATH_INFO: `/v3/projects/0be493010bc64873bd83fc32b2d0ca5f` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:12:17.970769 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a77bfcc2-fce5-4223-ac63-75b6eb1bea70 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0be493010bc64873bd83fc32b2d0ca5f)` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 16:12:17.972846 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a77bfcc2-fce5-4223-ac63-75b6eb1bea70 admin admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 16:12:18.082573 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-a77bfcc2-fce5-4223-ac63-75b6eb1bea70 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0be493010bc64873bd83fc32b2d0ca5f', 'request_id': 'req-a77bfcc2-fce5-4223-ac63-75b6eb1bea70'} {{(pid=81768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 16:12:18.088758 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-a77bfcc2-fce5-4223-ac63-75b6eb1bea70 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0be493010bc64873bd83fc32b2d0ca5f', 'request_id': 'req-a77bfcc2-fce5-4223-ac63-75b6eb1bea70'} {{(pid=81768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 16:12:18.170202 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-203f198c-d62a-4004-9a4a-269282dc2152 admin admin] Authenticating user token {{(pid=81767) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 16:12:18.171664 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.fernet_utils [None req-203f198c-d62a-4004-9a4a-269282dc2152 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81767) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 16:12:18.185064 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-203f198c-d62a-4004-9a4a-269282dc2152 admin admin] Validating token access rules against request {{(pid=81767) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 16:12:18.208697 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-a77bfcc2-fce5-4223-ac63-75b6eb1bea70 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0be493010bc64873bd83fc32b2d0ca5f', 'request_id': 'req-a77bfcc2-fce5-4223-ac63-75b6eb1bea70'} {{(pid=81768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 16:12:18.222164 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-a77bfcc2-fce5-4223-ac63-75b6eb1bea70 admin admin] The token cache is being invalidate because project 0be493010bc64873bd83fc32b2d0ca5f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81768) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 22 16:12:18.222377 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-a77bfcc2-fce5-4223-ac63-75b6eb1bea70 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 16:12:18.223018 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-a77bfcc2-fce5-4223-ac63-75b6eb1bea70 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 16:12:18.224728 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 1165/2304] 10.222.0.15 () {58 vars in 1233 bytes} [Mon Aug 22 16:12:17 2022] DELETE /identity/v3/projects/0be493010bc64873bd83fc32b2d0ca5f => generated 0 bytes in 339 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 22 16:12:18.231551 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7423d582-e47d-428b-9f39-326517b8a47f None None] REQUEST_METHOD: `POST` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:12:18.231825 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7423d582-e47d-428b-9f39-326517b8a47f None None] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:12:18.232111 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7423d582-e47d-428b-9f39-326517b8a47f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:12:18.244597 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aaffaca-7518-4ea3-b164-3f3aea976a43 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a98b1e4431940ab8d1fc95df2858a6c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81767) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 16:12:18.245522 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aaffaca-7518-4ea3-b164-3f3aea976a43 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:12:18.245876 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aaffaca-7518-4ea3-b164-3f3aea976a43 admin admin] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:12:18.246226 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aaffaca-7518-4ea3-b164-3f3aea976a43 admin admin] PATH_INFO: `/v3/users/ff43dbeb065442c1945fdf4a72425db3` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:12:18.247509 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aaffaca-7518-4ea3-b164-3f3aea976a43 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ff43dbeb065442c1945fdf4a72425db3)` {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 16:12:18.249484 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aaffaca-7518-4ea3-b164-3f3aea976a43 admin admin] RBAC: Authorization granted {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 16:12:18.253035 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.auth.core [None req-7423d582-e47d-428b-9f39-326517b8a47f None None] MFA Rules not processed for user `6b2199ae0fb0480aa917ada391f5ec54`. Rule list: `[]` (Enabled: `True`). {{(pid=81768) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 16:12:18.254248 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-7423d582-e47d-428b-9f39-326517b8a47f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 16:12:18.263038 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 1166/2305] 10.222.0.15 () {58 vars in 958 bytes} [Mon Aug 22 16:12:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 22 16:12:18.272252 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d8f2df6-250f-4d3f-a84f-78f9bc93e82a None None] REQUEST_METHOD: `POST` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:12:18.274162 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d8f2df6-250f-4d3f-a84f-78f9bc93e82a None None] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:12:18.274162 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d8f2df6-250f-4d3f-a84f-78f9bc93e82a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:12:18.293191 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG dbcounter [-] [81767] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81767) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 16:12:18.296997 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.auth.core [None req-1d8f2df6-250f-4d3f-a84f-78f9bc93e82a None None] MFA Rules not processed for user `6b2199ae0fb0480aa917ada391f5ec54`. Rule list: `[]` (Enabled: `True`). {{(pid=81768) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 16:12:18.296997 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-1d8f2df6-250f-4d3f-a84f-78f9bc93e82a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 16:12:18.304321 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 1167/2306] 10.222.0.15 () {58 vars in 958 bytes} [Mon Aug 22 16:12:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 22 16:12:18.308736 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d8f2df6-250f-4d3f-a84f-78f9bc93e82a None None] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 16:12:18.317948 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d8f2df6-250f-4d3f-a84f-78f9bc93e82a None None] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 16:12:18.322830 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-222935af-d820-402c-a6ef-4fbbfd481707 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a98b1e4431940ab8d1fc95df2858a6c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 16:12:18.323451 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-222935af-d820-402c-a6ef-4fbbfd481707 admin admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:12:18.323727 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-222935af-d820-402c-a6ef-4fbbfd481707 admin admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:12:18.323996 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-222935af-d820-402c-a6ef-4fbbfd481707 admin admin] PATH_INFO: `/v3/domains` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:12:18.324634 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-222935af-d820-402c-a6ef-4fbbfd481707 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81768) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 16:12:18.324936 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-222935af-d820-402c-a6ef-4fbbfd481707 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 16:12:18.326660 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-222935af-d820-402c-a6ef-4fbbfd481707 admin admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 16:12:18.342690 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 1168/2307] 10.222.0.15 () {58 vars in 1153 bytes} [Mon Aug 22 16:12:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 22 16:12:18.351870 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-222935af-d820-402c-a6ef-4fbbfd481707 admin admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 16:12:18.354459 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-222935af-d820-402c-a6ef-4fbbfd481707 admin admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 16:12:18.359322 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5406b5db-a4e2-40a6-8eb6-b379fd6ceeae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a98b1e4431940ab8d1fc95df2858a6c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 16:12:18.359937 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5406b5db-a4e2-40a6-8eb6-b379fd6ceeae admin admin] REQUEST_METHOD: `POST` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:12:18.360219 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5406b5db-a4e2-40a6-8eb6-b379fd6ceeae admin admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:12:18.360488 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5406b5db-a4e2-40a6-8eb6-b379fd6ceeae admin admin] PATH_INFO: `/v3/projects` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:12:18.361109 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5406b5db-a4e2-40a6-8eb6-b379fd6ceeae admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 16:12:18.363249 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5406b5db-a4e2-40a6-8eb6-b379fd6ceeae admin admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 16:12:18.434869 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.notifications [None req-5aaffaca-7518-4ea3-b164-3f3aea976a43 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ff43dbeb065442c1945fdf4a72425db3'} {{(pid=81767) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 16:12:18.435586 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.notifications [None req-5aaffaca-7518-4ea3-b164-3f3aea976a43 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ff43dbeb065442c1945fdf4a72425db3'} {{(pid=81767) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 16:12:18.450796 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 1169/2308] 10.222.0.15 () {60 vars in 1153 bytes} [Mon Aug 22 16:12:18 2022] POST /identity/v3/projects => generated 401 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 22 16:12:18.455719 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5406b5db-a4e2-40a6-8eb6-b379fd6ceeae admin admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 16:12:18.457399 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-5406b5db-a4e2-40a6-8eb6-b379fd6ceeae admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 16:12:18.495480 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.notifications [None req-5aaffaca-7518-4ea3-b164-3f3aea976a43 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ff43dbeb065442c1945fdf4a72425db3'} {{(pid=81767) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 16:12:18.503740 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.notifications [None req-5aaffaca-7518-4ea3-b164-3f3aea976a43 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ff43dbeb065442c1945fdf4a72425db3'} {{(pid=81767) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 16:12:18.516609 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.notifications [None req-5aaffaca-7518-4ea3-b164-3f3aea976a43 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ff43dbeb065442c1945fdf4a72425db3'} {{(pid=81767) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 16:12:18.521460 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5406b5db-a4e2-40a6-8eb6-b379fd6ceeae admin admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 16:12:18.529352 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f8fff16-8d6e-4303-9f32-ea9be8b3bd5e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a98b1e4431940ab8d1fc95df2858a6c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 16:12:18.529352 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f8fff16-8d6e-4303-9f32-ea9be8b3bd5e admin admin] REQUEST_METHOD: `POST` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:12:18.529904 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f8fff16-8d6e-4303-9f32-ea9be8b3bd5e admin admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:12:18.530018 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 1140/2309] 10.222.0.15 () {58 vars in 1224 bytes} [Mon Aug 22 16:12:18 2022] DELETE /identity/v3/users/ff43dbeb065442c1945fdf4a72425db3 => generated 0 bytes in 357 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 16:12:18.530098 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f8fff16-8d6e-4303-9f32-ea9be8b3bd5e admin admin] PATH_INFO: `/v3/users` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:12:18.530724 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f8fff16-8d6e-4303-9f32-ea9be8b3bd5e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 16:12:18.531255 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f952109e-e363-4836-8d6b-e6bbcbe705bf None None] REQUEST_METHOD: `POST` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:12:18.531579 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f952109e-e363-4836-8d6b-e6bbcbe705bf None None] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:12:18.531851 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f952109e-e363-4836-8d6b-e6bbcbe705bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:12:18.532719 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f8fff16-8d6e-4303-9f32-ea9be8b3bd5e admin admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 16:12:18.552530 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.auth.core [None req-f952109e-e363-4836-8d6b-e6bbcbe705bf None None] MFA Rules not processed for user `6b2199ae0fb0480aa917ada391f5ec54`. Rule list: `[]` (Enabled: `True`). {{(pid=81767) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 16:12:18.553791 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.fernet_utils [None req-f952109e-e363-4836-8d6b-e6bbcbe705bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81767) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 16:12:18.616808 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 1141/2310] 10.222.0.15 () {58 vars in 958 bytes} [Mon Aug 22 16:12:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 16:12:18.621043 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 1170/2311] 10.222.0.15 () {60 vars in 1144 bytes} [Mon Aug 22 16:12:18 2022] POST /identity/v3/users => generated 301 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 22 16:12:18.625381 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f952109e-e363-4836-8d6b-e6bbcbe705bf None None] Authenticating user token {{(pid=81767) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 16:12:18.631041 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f8fff16-8d6e-4303-9f32-ea9be8b3bd5e admin admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 16:12:18.632965 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-5f8fff16-8d6e-4303-9f32-ea9be8b3bd5e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 16:12:18.635811 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f952109e-e363-4836-8d6b-e6bbcbe705bf None None] Validating token access rules against request {{(pid=81767) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 16:12:18.640926 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e79849b0-8423-4227-b7a6-18455335b1f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a98b1e4431940ab8d1fc95df2858a6c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81767) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 16:12:18.641606 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e79849b0-8423-4227-b7a6-18455335b1f3 admin admin] REQUEST_METHOD: `GET` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:12:18.646163 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e79849b0-8423-4227-b7a6-18455335b1f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:12:18.646163 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e79849b0-8423-4227-b7a6-18455335b1f3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:12:18.646163 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e79849b0-8423-4227-b7a6-18455335b1f3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 16:12:18.647316 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e79849b0-8423-4227-b7a6-18455335b1f3 admin admin] RBAC: Authorization granted {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 16:12:18.659219 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 1142/2312] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Aug 22 16:12:18 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 22 16:12:18.663463 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e79849b0-8423-4227-b7a6-18455335b1f3 admin admin] Authenticating user token {{(pid=81767) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 16:12:18.675013 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e79849b0-8423-4227-b7a6-18455335b1f3 admin admin] Validating token access rules against request {{(pid=81767) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 16:12:18.680668 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15f6dae2-667e-41f9-ab32-9b244cca5f55 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a98b1e4431940ab8d1fc95df2858a6c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81767) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 16:12:18.681365 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f6dae2-667e-41f9-ab32-9b244cca5f55 admin admin] REQUEST_METHOD: `GET` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:12:18.681718 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f6dae2-667e-41f9-ab32-9b244cca5f55 admin admin] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:12:18.682245 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f6dae2-667e-41f9-ab32-9b244cca5f55 admin admin] PATH_INFO: `/v3/roles` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:12:18.683076 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15f6dae2-667e-41f9-ab32-9b244cca5f55 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 16:12:18.688922 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15f6dae2-667e-41f9-ab32-9b244cca5f55 admin admin] RBAC: Authorization granted {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 16:12:18.699445 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 1143/2313] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Aug 22 16:12:18 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 22 16:12:18.701415 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15f6dae2-667e-41f9-ab32-9b244cca5f55 admin admin] Authenticating user token {{(pid=81767) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 16:12:18.720281 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15f6dae2-667e-41f9-ab32-9b244cca5f55 admin admin] Validating token access rules against request {{(pid=81767) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 16:12:18.723158 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7c3a261-bc83-4235-b9dc-3d25a5bc4f03 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a98b1e4431940ab8d1fc95df2858a6c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81767) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 16:12:18.723702 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f8fff16-8d6e-4303-9f32-ea9be8b3bd5e admin admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 16:12:18.723887 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7c3a261-bc83-4235-b9dc-3d25a5bc4f03 admin admin] REQUEST_METHOD: `PUT` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:12:18.724168 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7c3a261-bc83-4235-b9dc-3d25a5bc4f03 admin admin] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:12:18.724444 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7c3a261-bc83-4235-b9dc-3d25a5bc4f03 admin admin] PATH_INFO: `/v3/projects/0e5c1343cecc4c6d9d17bd83d4d95116/users/e9d4e77022fe483999eefda45f27e410/roles/e6fc1ee3fb974125ba9b4fe55b12b45e` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:12:18.749230 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7c3a261-bc83-4235-b9dc-3d25a5bc4f03 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0e5c1343cecc4c6d9d17bd83d4d95116, user_id=e9d4e77022fe483999eefda45f27e410, role_id=e6fc1ee3fb974125ba9b4fe55b12b45e)` {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 16:12:18.749230 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7c3a261-bc83-4235-b9dc-3d25a5bc4f03 admin admin] RBAC: Authorization granted {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 16:12:18.752849 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 16:12:18.777023 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cd6f65f-b4b2-4c15-820a-edfebb9a15f0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '234687adad594f1aa45775a565ffbdc7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4ee0413f9ef4d4fb1f95a660babe090', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 16:12:18.777640 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd6f65f-b4b2-4c15-820a-edfebb9a15f0 service neutron] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:12:18.777922 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd6f65f-b4b2-4c15-820a-edfebb9a15f0 service neutron] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:12:18.778227 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd6f65f-b4b2-4c15-820a-edfebb9a15f0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:12:18.787871 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cd6f65f-b4b2-4c15-820a-edfebb9a15f0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 16:12:18.789324 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cd6f65f-b4b2-4c15-820a-edfebb9a15f0 service neutron] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 16:12:18.799613 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 1171/2314] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Aug 22 16:12:18 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 169 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 22 16:12:18.866275 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 1144/2315] 10.222.0.15 () {60 vars in 1483 bytes} [Mon Aug 22 16:12:18 2022] PUT /identity/v3/projects/0e5c1343cecc4c6d9d17bd83d4d95116/users/e9d4e77022fe483999eefda45f27e410/roles/e6fc1ee3fb974125ba9b4fe55b12b45e => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 16:12:18.878639 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-201a57f0-e6f6-48da-bb25-bab7a691d0e5 None None] REQUEST_METHOD: `POST` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:12:18.878941 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-201a57f0-e6f6-48da-bb25-bab7a691d0e5 None None] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:12:18.879197 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-201a57f0-e6f6-48da-bb25-bab7a691d0e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:12:18.909845 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.auth.core [None req-201a57f0-e6f6-48da-bb25-bab7a691d0e5 None None] MFA Rules not processed for user `e9d4e77022fe483999eefda45f27e410`. Rule list: `[]` (Enabled: `True`). {{(pid=81768) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 16:12:18.911049 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-201a57f0-e6f6-48da-bb25-bab7a691d0e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 16:12:18.938553 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG dbcounter [-] [81767] Writing DB stats keystone:SELECT=78,keystone:INSERT=5,keystone:DELETE=17 {{(pid=81767) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 16:12:18.978893 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 1172/2316] 10.222.0.15 () {58 vars in 958 bytes} [Mon Aug 22 16:12:18 2022] POST /identity/v3/auth/tokens => generated 3278 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 16:12:18.983678 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7c3a261-bc83-4235-b9dc-3d25a5bc4f03 admin admin] Authenticating user token {{(pid=81767) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 16:12:19.052582 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7c3a261-bc83-4235-b9dc-3d25a5bc4f03 admin admin] Validating token access rules against request {{(pid=81767) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 16:12:19.057775 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ef7d777-83f5-4531-827f-7c69fdba9594 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a98b1e4431940ab8d1fc95df2858a6c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81767) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 16:12:19.058569 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ef7d777-83f5-4531-827f-7c69fdba9594 admin admin] REQUEST_METHOD: `POST` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:12:19.058848 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ef7d777-83f5-4531-827f-7c69fdba9594 admin admin] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:12:19.059111 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ef7d777-83f5-4531-827f-7c69fdba9594 admin admin] PATH_INFO: `/v3/projects` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:12:19.059767 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ef7d777-83f5-4531-827f-7c69fdba9594 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 16:12:19.061677 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ef7d777-83f5-4531-827f-7c69fdba9594 admin admin] RBAC: Authorization granted {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 16:12:19.178087 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 1145/2317] 10.222.0.15 () {60 vars in 1153 bytes} [Mon Aug 22 16:12:18 2022] POST /identity/v3/projects => generated 401 bytes in 192 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 22 16:12:19.179205 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-201a57f0-e6f6-48da-bb25-bab7a691d0e5 None None] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 16:12:19.190736 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-201a57f0-e6f6-48da-bb25-bab7a691d0e5 None None] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 16:12:19.254252 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-933dadd6-eb28-41b0-a08e-54914cba0340 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a98b1e4431940ab8d1fc95df2858a6c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 16:12:19.255101 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-933dadd6-eb28-41b0-a08e-54914cba0340 admin admin] REQUEST_METHOD: `POST` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:12:19.255101 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-933dadd6-eb28-41b0-a08e-54914cba0340 admin admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:12:19.258265 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-933dadd6-eb28-41b0-a08e-54914cba0340 admin admin] PATH_INFO: `/v3/users` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:12:19.258265 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-933dadd6-eb28-41b0-a08e-54914cba0340 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 16:12:19.258265 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-933dadd6-eb28-41b0-a08e-54914cba0340 admin admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 16:12:19.261556 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ef7d777-83f5-4531-827f-7c69fdba9594 admin admin] Authenticating user token {{(pid=81767) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 16:12:19.263645 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.fernet_utils [None req-2ef7d777-83f5-4531-827f-7c69fdba9594 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81767) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 16:12:19.279734 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ef7d777-83f5-4531-827f-7c69fdba9594 admin admin] Validating token access rules against request {{(pid=81767) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 16:12:19.287591 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b10b1147-98de-4321-a189-ff30f10b9bff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a98b1e4431940ab8d1fc95df2858a6c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81767) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 16:12:19.288501 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b10b1147-98de-4321-a189-ff30f10b9bff admin admin] REQUEST_METHOD: `DELETE` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:12:19.288782 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b10b1147-98de-4321-a189-ff30f10b9bff admin admin] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:12:19.289049 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b10b1147-98de-4321-a189-ff30f10b9bff admin admin] PATH_INFO: `/v3/projects/b3eb08b95c9c46e9b95ba6feb59019ae` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:12:19.290210 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b10b1147-98de-4321-a189-ff30f10b9bff admin admin] RBAC: Authorizing `identity:delete_project(project_id=b3eb08b95c9c46e9b95ba6feb59019ae)` {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 16:12:19.292302 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b10b1147-98de-4321-a189-ff30f10b9bff admin admin] RBAC: Authorization granted {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 16:12:19.339765 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 1173/2318] 10.222.0.15 () {60 vars in 1144 bytes} [Mon Aug 22 16:12:19 2022] POST /identity/v3/users => generated 307 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 22 16:12:19.343053 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-933dadd6-eb28-41b0-a08e-54914cba0340 admin admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 16:12:19.353407 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-933dadd6-eb28-41b0-a08e-54914cba0340 admin admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 16:12:19.358699 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74237244-fb8f-4ec4-8c48-c8ea476f3995 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a98b1e4431940ab8d1fc95df2858a6c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 16:12:19.359307 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74237244-fb8f-4ec4-8c48-c8ea476f3995 admin admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:12:19.359634 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74237244-fb8f-4ec4-8c48-c8ea476f3995 admin admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:12:19.360011 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74237244-fb8f-4ec4-8c48-c8ea476f3995 admin admin] PATH_INFO: `/v3/roles` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:12:19.360577 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74237244-fb8f-4ec4-8c48-c8ea476f3995 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 16:12:19.362217 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74237244-fb8f-4ec4-8c48-c8ea476f3995 admin admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 16:12:19.372464 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 1174/2319] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Aug 22 16:12:19 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 22 16:12:19.376491 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74237244-fb8f-4ec4-8c48-c8ea476f3995 admin admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 16:12:19.386864 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74237244-fb8f-4ec4-8c48-c8ea476f3995 admin admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 16:12:19.391563 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-becd77da-e076-4ea2-b2d9-bc12df32d67d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a98b1e4431940ab8d1fc95df2858a6c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 16:12:19.392191 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-becd77da-e076-4ea2-b2d9-bc12df32d67d admin admin] REQUEST_METHOD: `PUT` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:12:19.392471 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-becd77da-e076-4ea2-b2d9-bc12df32d67d admin admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:12:19.392827 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-becd77da-e076-4ea2-b2d9-bc12df32d67d admin admin] PATH_INFO: `/v3/projects/86b4b257251d4bf0ad86d874b55c8515/users/fcdeaaa4687340deafb946b7d3f7fac1/roles/9d9848cfd22e4b089db443c350e56670` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:12:19.409311 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-becd77da-e076-4ea2-b2d9-bc12df32d67d admin admin] RBAC: Authorizing `identity:create_grant(project_id=86b4b257251d4bf0ad86d874b55c8515, user_id=fcdeaaa4687340deafb946b7d3f7fac1, role_id=9d9848cfd22e4b089db443c350e56670)` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 16:12:19.411994 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-becd77da-e076-4ea2-b2d9-bc12df32d67d admin admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 16:12:19.422321 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.notifications [None req-b10b1147-98de-4321-a189-ff30f10b9bff admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b3eb08b95c9c46e9b95ba6feb59019ae', 'request_id': 'req-b10b1147-98de-4321-a189-ff30f10b9bff'} {{(pid=81767) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 16:12:19.429160 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.notifications [None req-b10b1147-98de-4321-a189-ff30f10b9bff admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b3eb08b95c9c46e9b95ba6feb59019ae', 'request_id': 'req-b10b1147-98de-4321-a189-ff30f10b9bff'} {{(pid=81767) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 16:12:19.510534 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 1175/2320] 10.222.0.15 () {60 vars in 1483 bytes} [Mon Aug 22 16:12:19 2022] PUT /identity/v3/projects/86b4b257251d4bf0ad86d874b55c8515/users/fcdeaaa4687340deafb946b7d3f7fac1/roles/9d9848cfd22e4b089db443c350e56670 => generated 0 bytes in 134 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 22 16:12:19.514536 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-becd77da-e076-4ea2-b2d9-bc12df32d67d admin admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 16:12:19.533860 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.notifications [None req-b10b1147-98de-4321-a189-ff30f10b9bff admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b3eb08b95c9c46e9b95ba6feb59019ae', 'request_id': 'req-b10b1147-98de-4321-a189-ff30f10b9bff'} {{(pid=81767) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 16:12:19.534424 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 16:12:19.546437 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.notifications [None req-b10b1147-98de-4321-a189-ff30f10b9bff admin admin] The token cache is being invalidate because project b3eb08b95c9c46e9b95ba6feb59019ae was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81767) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 22 16:12:19.546733 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.notifications [None req-b10b1147-98de-4321-a189-ff30f10b9bff admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81767) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 16:12:19.547465 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.notifications [None req-b10b1147-98de-4321-a189-ff30f10b9bff admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81767) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 16:12:19.549783 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 1146/2321] 10.222.0.15 () {58 vars in 1233 bytes} [Mon Aug 22 16:12:19 2022] DELETE /identity/v3/projects/b3eb08b95c9c46e9b95ba6feb59019ae => generated 0 bytes in 288 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 22 16:12:19.553468 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b10b1147-98de-4321-a189-ff30f10b9bff admin admin] Authenticating user token {{(pid=81767) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 16:12:19.554868 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.fernet_utils [None req-b10b1147-98de-4321-a189-ff30f10b9bff admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81767) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 16:12:19.580939 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-becd77da-e076-4ea2-b2d9-bc12df32d67d admin admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 16:12:19.586332 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3aa2682-9ff9-4873-9657-17a3e6228d7a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a98b1e4431940ab8d1fc95df2858a6c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 16:12:19.586948 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3aa2682-9ff9-4873-9657-17a3e6228d7a admin admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:12:19.587298 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3aa2682-9ff9-4873-9657-17a3e6228d7a admin admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:12:19.587579 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3aa2682-9ff9-4873-9657-17a3e6228d7a admin admin] PATH_INFO: `/v3/roles` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:12:19.588357 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3aa2682-9ff9-4873-9657-17a3e6228d7a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 16:12:19.589988 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3aa2682-9ff9-4873-9657-17a3e6228d7a admin admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 16:12:19.600845 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 1176/2322] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Aug 22 16:12:19 2022] GET /identity/v3/roles => generated 1420 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 22 16:12:19.605127 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3aa2682-9ff9-4873-9657-17a3e6228d7a admin admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 16:12:19.606911 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-b3aa2682-9ff9-4873-9657-17a3e6228d7a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 16:12:19.620246 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3aa2682-9ff9-4873-9657-17a3e6228d7a admin admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 16:12:19.621280 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b10b1147-98de-4321-a189-ff30f10b9bff admin admin] Validating token access rules against request {{(pid=81767) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 16:12:19.624945 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56fff602-d948-43af-bccb-0099ab478171 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a98b1e4431940ab8d1fc95df2858a6c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 16:12:19.627784 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56fff602-d948-43af-bccb-0099ab478171 admin admin] REQUEST_METHOD: `PUT` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:12:19.627784 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56fff602-d948-43af-bccb-0099ab478171 admin admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:12:19.627784 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56fff602-d948-43af-bccb-0099ab478171 admin admin] PATH_INFO: `/v3/projects/86b4b257251d4bf0ad86d874b55c8515/users/fcdeaaa4687340deafb946b7d3f7fac1/roles/9d9848cfd22e4b089db443c350e56670` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:12:19.628062 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a442b32-a59c-4dd4-89c1-34dc92773c37 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a98b1e4431940ab8d1fc95df2858a6c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81767) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 16:12:19.628062 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a442b32-a59c-4dd4-89c1-34dc92773c37 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:12:19.628062 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a442b32-a59c-4dd4-89c1-34dc92773c37 admin admin] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:12:19.628318 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56fff602-d948-43af-bccb-0099ab478171 admin admin] RBAC: Authorizing `identity:create_grant(project_id=86b4b257251d4bf0ad86d874b55c8515, user_id=fcdeaaa4687340deafb946b7d3f7fac1, role_id=9d9848cfd22e4b089db443c350e56670)` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 16:12:19.629366 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG dbcounter [-] [81767] Writing DB stats keystone:SELECT=93,keystone:INSERT=4,keystone:DELETE=3 {{(pid=81767) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 16:12:19.629903 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a442b32-a59c-4dd4-89c1-34dc92773c37 admin admin] PATH_INFO: `/v3/users/ebe6ac072762420184313995825f90c9` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:12:19.630784 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56fff602-d948-43af-bccb-0099ab478171 admin admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 16:12:19.631106 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a442b32-a59c-4dd4-89c1-34dc92773c37 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ebe6ac072762420184313995825f90c9)` {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 16:12:19.633061 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a442b32-a59c-4dd4-89c1-34dc92773c37 admin admin] RBAC: Authorization granted {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 16:12:19.641601 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 1177/2323] 10.222.0.15 () {60 vars in 1483 bytes} [Mon Aug 22 16:12:19 2022] PUT /identity/v3/projects/86b4b257251d4bf0ad86d874b55c8515/users/fcdeaaa4687340deafb946b7d3f7fac1/roles/9d9848cfd22e4b089db443c350e56670 => generated 0 bytes in 37 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 22 16:12:19.652354 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01e58fea-6aee-4044-bae5-ef6fef09b3ad None None] REQUEST_METHOD: `POST` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:12:19.652679 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01e58fea-6aee-4044-bae5-ef6fef09b3ad None None] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:12:19.653511 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01e58fea-6aee-4044-bae5-ef6fef09b3ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:12:19.683550 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.auth.core [None req-01e58fea-6aee-4044-bae5-ef6fef09b3ad None None] MFA Rules not processed for user `fcdeaaa4687340deafb946b7d3f7fac1`. Rule list: `[]` (Enabled: `True`). {{(pid=81768) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 16:12:19.684763 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-01e58fea-6aee-4044-bae5-ef6fef09b3ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 16:12:19.765009 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 1178/2324] 10.222.0.15 () {58 vars in 958 bytes} [Mon Aug 22 16:12:19 2022] POST /identity/v3/auth/tokens => generated 3345 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 16:12:19.777432 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01e58fea-6aee-4044-bae5-ef6fef09b3ad None None] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 16:12:19.778873 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-01e58fea-6aee-4044-bae5-ef6fef09b3ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 16:12:19.843255 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01e58fea-6aee-4044-bae5-ef6fef09b3ad None None] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 16:12:19.847815 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d171774-9aa4-44d6-a9e7-283300c7294e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79706d480f864998b9f8594be407ceab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4ee0413f9ef4d4fb1f95a660babe090', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 16:12:19.849902 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=90,keystone:INSERT=2,keystone:DELETE=8 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 16:12:19.850622 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d171774-9aa4-44d6-a9e7-283300c7294e service cinder] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:12:19.850753 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.notifications [None req-1a442b32-a59c-4dd4-89c1-34dc92773c37 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ebe6ac072762420184313995825f90c9'} {{(pid=81767) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 16:12:19.851227 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d171774-9aa4-44d6-a9e7-283300c7294e service cinder] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:12:19.851450 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.notifications [None req-1a442b32-a59c-4dd4-89c1-34dc92773c37 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ebe6ac072762420184313995825f90c9'} {{(pid=81767) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 16:12:19.851555 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d171774-9aa4-44d6-a9e7-283300c7294e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:12:19.853657 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-5d171774-9aa4-44d6-a9e7-283300c7294e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 16:12:19.867077 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d171774-9aa4-44d6-a9e7-283300c7294e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 16:12:19.868597 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d171774-9aa4-44d6-a9e7-283300c7294e service cinder] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 16:12:19.879216 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 1179/2325] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Aug 22 16:12:19 2022] GET /identity/v3/auth/tokens => generated 3345 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 22 16:12:19.955662 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.notifications [None req-1a442b32-a59c-4dd4-89c1-34dc92773c37 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ebe6ac072762420184313995825f90c9'} {{(pid=81767) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 16:12:19.966019 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.notifications [None req-1a442b32-a59c-4dd4-89c1-34dc92773c37 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ebe6ac072762420184313995825f90c9'} {{(pid=81767) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 16:12:19.977689 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.notifications [None req-1a442b32-a59c-4dd4-89c1-34dc92773c37 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ebe6ac072762420184313995825f90c9'} {{(pid=81767) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 16:12:19.983632 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 1147/2326] 10.222.0.15 () {58 vars in 1224 bytes} [Mon Aug 22 16:12:19 2022] DELETE /identity/v3/users/ebe6ac072762420184313995825f90c9 => generated 0 bytes in 431 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 22 16:12:20.061468 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG dbcounter [-] [81767] Writing DB stats keystone:SELECT=89,keystone:DELETE=9,keystone:INSERT=2 {{(pid=81767) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 16:12:20.143875 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d171774-9aa4-44d6-a9e7-283300c7294e service cinder] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 16:12:20.145346 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-5d171774-9aa4-44d6-a9e7-283300c7294e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 16:12:20.155896 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d171774-9aa4-44d6-a9e7-283300c7294e service cinder] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 16:12:20.208729 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba5050cd-e2a9-4080-b184-743862efedf2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79706d480f864998b9f8594be407ceab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4ee0413f9ef4d4fb1f95a660babe090', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 16:12:20.209661 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba5050cd-e2a9-4080-b184-743862efedf2 service cinder] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:12:20.209937 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba5050cd-e2a9-4080-b184-743862efedf2 service cinder] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:12:20.210208 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba5050cd-e2a9-4080-b184-743862efedf2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:12:20.211983 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-ba5050cd-e2a9-4080-b184-743862efedf2 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 16:12:20.268540 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=89,keystone:DELETE=9,keystone:INSERT=2 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 16:12:20.291539 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba5050cd-e2a9-4080-b184-743862efedf2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 16:12:20.292936 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba5050cd-e2a9-4080-b184-743862efedf2 service cinder] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 16:12:20.305656 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 1180/2327] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Aug 22 16:12:20 2022] GET /identity/v3/auth/tokens => generated 3278 bytes in 162 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 16:12:20.457297 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a442b32-a59c-4dd4-89c1-34dc92773c37 admin admin] Authenticating user token {{(pid=81767) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 16:12:20.458802 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.fernet_utils [None req-1a442b32-a59c-4dd4-89c1-34dc92773c37 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81767) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 16:12:20.528310 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a442b32-a59c-4dd4-89c1-34dc92773c37 admin admin] Validating token access rules against request {{(pid=81767) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 16:12:20.533970 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04789693-a19a-4817-9a14-24f90ee2466b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a98b1e4431940ab8d1fc95df2858a6c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81767) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 16:12:20.534686 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04789693-a19a-4817-9a14-24f90ee2466b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:12:20.535035 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04789693-a19a-4817-9a14-24f90ee2466b admin admin] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:12:20.535413 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04789693-a19a-4817-9a14-24f90ee2466b admin admin] PATH_INFO: `/v3/projects/5bd8c32ccb2e4056bd363b8d695c2962` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:12:20.536975 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04789693-a19a-4817-9a14-24f90ee2466b admin admin] RBAC: Authorizing `identity:delete_project(project_id=5bd8c32ccb2e4056bd363b8d695c2962)` {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 16:12:20.538979 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04789693-a19a-4817-9a14-24f90ee2466b admin admin] RBAC: Authorization granted {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 16:12:20.609597 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG dbcounter [-] [81767] Writing DB stats keystone:SELECT=93,keystone:DELETE=5,keystone:INSERT=2 {{(pid=81767) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 16:12:20.657543 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.notifications [None req-04789693-a19a-4817-9a14-24f90ee2466b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5bd8c32ccb2e4056bd363b8d695c2962', 'request_id': 'req-04789693-a19a-4817-9a14-24f90ee2466b'} {{(pid=81767) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 16:12:20.664154 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.notifications [None req-04789693-a19a-4817-9a14-24f90ee2466b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5bd8c32ccb2e4056bd363b8d695c2962', 'request_id': 'req-04789693-a19a-4817-9a14-24f90ee2466b'} {{(pid=81767) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 16:12:20.717604 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.notifications [None req-04789693-a19a-4817-9a14-24f90ee2466b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5bd8c32ccb2e4056bd363b8d695c2962', 'request_id': 'req-04789693-a19a-4817-9a14-24f90ee2466b'} {{(pid=81767) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 16:12:20.730943 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.notifications [None req-04789693-a19a-4817-9a14-24f90ee2466b admin admin] The token cache is being invalidate because project 5bd8c32ccb2e4056bd363b8d695c2962 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81767) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 22 16:12:20.732828 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.notifications [None req-04789693-a19a-4817-9a14-24f90ee2466b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81767) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 16:12:20.732828 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.notifications [None req-04789693-a19a-4817-9a14-24f90ee2466b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81767) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 16:12:20.737438 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 1148/2328] 10.222.0.15 () {58 vars in 1233 bytes} [Mon Aug 22 16:12:20 2022] DELETE /identity/v3/projects/5bd8c32ccb2e4056bd363b8d695c2962 => generated 0 bytes in 278 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 16:12:20.739138 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0847a469-8798-4e0c-acae-e77e60565aff None None] REQUEST_METHOD: `POST` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:12:20.739473 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0847a469-8798-4e0c-acae-e77e60565aff None None] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:12:20.739732 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0847a469-8798-4e0c-acae-e77e60565aff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:12:20.761675 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.auth.core [None req-0847a469-8798-4e0c-acae-e77e60565aff None None] MFA Rules not processed for user `6b2199ae0fb0480aa917ada391f5ec54`. Rule list: `[]` (Enabled: `True`). {{(pid=81768) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 16:12:20.761675 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-0847a469-8798-4e0c-acae-e77e60565aff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 16:12:20.788002 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 16:12:20.825968 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 1181/2329] 10.222.0.15 () {58 vars in 958 bytes} [Mon Aug 22 16:12:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 22 16:12:20.834563 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dad7cdb2-8e28-48bc-bfab-a5b80348db84 None None] REQUEST_METHOD: `POST` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:12:20.834850 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dad7cdb2-8e28-48bc-bfab-a5b80348db84 None None] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:12:20.835122 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dad7cdb2-8e28-48bc-bfab-a5b80348db84 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:12:20.857546 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.auth.core [None req-dad7cdb2-8e28-48bc-bfab-a5b80348db84 None None] MFA Rules not processed for user `6b2199ae0fb0480aa917ada391f5ec54`. Rule list: `[]` (Enabled: `True`). {{(pid=81767) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 16:12:20.858760 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.fernet_utils [None req-dad7cdb2-8e28-48bc-bfab-a5b80348db84 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81767) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 16:12:20.867518 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 1149/2330] 10.222.0.15 () {58 vars in 958 bytes} [Mon Aug 22 16:12:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 16:12:20.872009 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0847a469-8798-4e0c-acae-e77e60565aff None None] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 16:12:20.882347 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0847a469-8798-4e0c-acae-e77e60565aff None None] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 16:12:20.886995 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-488f7360-95d9-415c-9241-6fef7b5c6e98 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a98b1e4431940ab8d1fc95df2858a6c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 16:12:20.887640 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-488f7360-95d9-415c-9241-6fef7b5c6e98 admin admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:12:20.887916 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-488f7360-95d9-415c-9241-6fef7b5c6e98 admin admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:12:20.888186 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-488f7360-95d9-415c-9241-6fef7b5c6e98 admin admin] PATH_INFO: `/v3/domains` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:12:20.888883 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-488f7360-95d9-415c-9241-6fef7b5c6e98 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81768) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 16:12:20.889130 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-488f7360-95d9-415c-9241-6fef7b5c6e98 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 16:12:20.890732 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-488f7360-95d9-415c-9241-6fef7b5c6e98 admin admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 16:12:20.908760 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 1182/2331] 10.222.0.15 () {58 vars in 1153 bytes} [Mon Aug 22 16:12:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 16:12:20.909737 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dad7cdb2-8e28-48bc-bfab-a5b80348db84 None None] Authenticating user token {{(pid=81767) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 16:12:20.919805 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dad7cdb2-8e28-48bc-bfab-a5b80348db84 None None] Validating token access rules against request {{(pid=81767) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 16:12:20.926723 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-481c738c-bac7-4719-a145-dd149ad16156 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a98b1e4431940ab8d1fc95df2858a6c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81767) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 16:12:20.927516 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-481c738c-bac7-4719-a145-dd149ad16156 admin admin] REQUEST_METHOD: `POST` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:12:20.927694 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-481c738c-bac7-4719-a145-dd149ad16156 admin admin] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:12:20.927989 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-481c738c-bac7-4719-a145-dd149ad16156 admin admin] PATH_INFO: `/v3/projects` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:12:20.928609 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-481c738c-bac7-4719-a145-dd149ad16156 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 16:12:20.930527 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-481c738c-bac7-4719-a145-dd149ad16156 admin admin] RBAC: Authorization granted {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 16:12:21.032976 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG dbcounter [-] [81767] Writing DB stats keystone:SELECT=88,keystone:INSERT=5,keystone:DELETE=7 {{(pid=81767) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 16:12:21.043923 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 1150/2332] 10.222.0.15 () {60 vars in 1153 bytes} [Mon Aug 22 16:12:20 2022] POST /identity/v3/projects => generated 373 bytes in 134 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 22 16:12:21.047804 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-488f7360-95d9-415c-9241-6fef7b5c6e98 admin admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 16:12:21.058157 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-488f7360-95d9-415c-9241-6fef7b5c6e98 admin admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 16:12:21.125993 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e95e56b-c972-498c-a72e-07d87e5d84d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a98b1e4431940ab8d1fc95df2858a6c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 16:12:21.125993 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e95e56b-c972-498c-a72e-07d87e5d84d4 admin admin] REQUEST_METHOD: `POST` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:12:21.125993 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e95e56b-c972-498c-a72e-07d87e5d84d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:12:21.125993 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e95e56b-c972-498c-a72e-07d87e5d84d4 admin admin] PATH_INFO: `/v3/users` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:12:21.126470 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e95e56b-c972-498c-a72e-07d87e5d84d4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 16:12:21.126470 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e95e56b-c972-498c-a72e-07d87e5d84d4 admin admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 16:12:21.295209 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 1183/2333] 10.222.0.15 () {60 vars in 1144 bytes} [Mon Aug 22 16:12:21 2022] POST /identity/v3/users => generated 287 bytes in 248 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 22 16:12:21.299918 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-481c738c-bac7-4719-a145-dd149ad16156 admin admin] Authenticating user token {{(pid=81767) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 16:12:21.310526 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-481c738c-bac7-4719-a145-dd149ad16156 admin admin] Validating token access rules against request {{(pid=81767) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 16:12:21.316169 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bb82ccb-8d36-4fcd-a67f-ef5d96f1b592 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a98b1e4431940ab8d1fc95df2858a6c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81767) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 16:12:21.316780 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bb82ccb-8d36-4fcd-a67f-ef5d96f1b592 admin admin] REQUEST_METHOD: `GET` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:12:21.317061 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bb82ccb-8d36-4fcd-a67f-ef5d96f1b592 admin admin] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:12:21.317338 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bb82ccb-8d36-4fcd-a67f-ef5d96f1b592 admin admin] PATH_INFO: `/v3/roles` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:12:21.318027 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bb82ccb-8d36-4fcd-a67f-ef5d96f1b592 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 16:12:21.320171 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bb82ccb-8d36-4fcd-a67f-ef5d96f1b592 admin admin] RBAC: Authorization granted {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 16:12:21.332468 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 1151/2334] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Aug 22 16:12:21 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 22 16:12:21.339400 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e95e56b-c972-498c-a72e-07d87e5d84d4 admin admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 16:12:21.351128 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e95e56b-c972-498c-a72e-07d87e5d84d4 admin admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 16:12:21.355961 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a33957b-fcc0-442a-87cb-20eb32ece9df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a98b1e4431940ab8d1fc95df2858a6c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 16:12:21.359639 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a33957b-fcc0-442a-87cb-20eb32ece9df admin admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:12:21.359639 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a33957b-fcc0-442a-87cb-20eb32ece9df admin admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:12:21.359639 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a33957b-fcc0-442a-87cb-20eb32ece9df admin admin] PATH_INFO: `/v3/roles` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:12:21.359639 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a33957b-fcc0-442a-87cb-20eb32ece9df admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 16:12:21.359639 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a33957b-fcc0-442a-87cb-20eb32ece9df admin admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 16:12:21.371745 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 1184/2335] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Aug 22 16:12:21 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 22 16:12:21.376072 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bb82ccb-8d36-4fcd-a67f-ef5d96f1b592 admin admin] Authenticating user token {{(pid=81767) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 16:12:21.387538 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bb82ccb-8d36-4fcd-a67f-ef5d96f1b592 admin admin] Validating token access rules against request {{(pid=81767) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 16:12:21.392330 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b97029d9-edac-45ea-b41f-e363e1c404b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a98b1e4431940ab8d1fc95df2858a6c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81767) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 16:12:21.393020 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b97029d9-edac-45ea-b41f-e363e1c404b8 admin admin] REQUEST_METHOD: `PUT` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:12:21.393297 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b97029d9-edac-45ea-b41f-e363e1c404b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:12:21.393569 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b97029d9-edac-45ea-b41f-e363e1c404b8 admin admin] PATH_INFO: `/v3/projects/6c8f21f91d4441a39dccb10831ccf011/users/01bb7d8843624abcbeba077d9ccdb6a6/roles/e6fc1ee3fb974125ba9b4fe55b12b45e` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:12:21.408364 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b97029d9-edac-45ea-b41f-e363e1c404b8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6c8f21f91d4441a39dccb10831ccf011, user_id=01bb7d8843624abcbeba077d9ccdb6a6, role_id=e6fc1ee3fb974125ba9b4fe55b12b45e)` {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 16:12:21.411033 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b97029d9-edac-45ea-b41f-e363e1c404b8 admin admin] RBAC: Authorization granted {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 16:12:21.429973 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 16:12:21.472865 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 1152/2336] 10.222.0.15 () {60 vars in 1483 bytes} [Mon Aug 22 16:12:21 2022] PUT /identity/v3/projects/6c8f21f91d4441a39dccb10831ccf011/users/01bb7d8843624abcbeba077d9ccdb6a6/roles/e6fc1ee3fb974125ba9b4fe55b12b45e => generated 0 bytes in 97 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 22 16:12:21.478214 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88bc25a6-5a37-4830-881f-6cf0cb745a9a None None] REQUEST_METHOD: `POST` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:12:21.478495 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88bc25a6-5a37-4830-881f-6cf0cb745a9a None None] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:12:21.478765 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88bc25a6-5a37-4830-881f-6cf0cb745a9a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:12:21.498960 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.auth.core [None req-88bc25a6-5a37-4830-881f-6cf0cb745a9a None None] MFA Rules not processed for user `6b2199ae0fb0480aa917ada391f5ec54`. Rule list: `[]` (Enabled: `True`). {{(pid=81768) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 16:12:21.503874 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-88bc25a6-5a37-4830-881f-6cf0cb745a9a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 16:12:21.562506 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 1185/2337] 10.222.0.15 () {58 vars in 958 bytes} [Mon Aug 22 16:12:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 16:12:21.572151 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b97029d9-edac-45ea-b41f-e363e1c404b8 admin admin] Authenticating user token {{(pid=81767) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 16:12:21.576513 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.fernet_utils [None req-b97029d9-edac-45ea-b41f-e363e1c404b8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81767) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 16:12:21.634825 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b97029d9-edac-45ea-b41f-e363e1c404b8 admin admin] Validating token access rules against request {{(pid=81767) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 16:12:21.638437 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32014b72-2713-42f4-a764-a4bff0472ab1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '234687adad594f1aa45775a565ffbdc7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4ee0413f9ef4d4fb1f95a660babe090', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81767) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 16:12:21.639049 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32014b72-2713-42f4-a764-a4bff0472ab1 service neutron] REQUEST_METHOD: `GET` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:12:21.639357 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32014b72-2713-42f4-a764-a4bff0472ab1 service neutron] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:12:21.639632 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32014b72-2713-42f4-a764-a4bff0472ab1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:12:21.649893 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32014b72-2713-42f4-a764-a4bff0472ab1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 16:12:21.651326 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32014b72-2713-42f4-a764-a4bff0472ab1 service neutron] RBAC: Authorization granted {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 16:12:21.661133 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 1153/2338] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Aug 22 16:12:21 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 16:12:21.689891 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG dbcounter [-] [81767] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81767) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 16:12:21.752813 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 16:12:22.030885 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG dbcounter [-] [81767] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81767) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 16:12:22.070284 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 16:12:22.140654 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ec15fe1-cc28-4704-ac49-9458945857ad None None] REQUEST_METHOD: `POST` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:12:22.141004 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ec15fe1-cc28-4704-ac49-9458945857ad None None] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:12:22.141502 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ec15fe1-cc28-4704-ac49-9458945857ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:12:22.188780 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.auth.core [None req-7ec15fe1-cc28-4704-ac49-9458945857ad None None] MFA Rules not processed for user `234687adad594f1aa45775a565ffbdc7`. Rule list: `[]` (Enabled: `True`). {{(pid=81768) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 16:12:22.190755 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-7ec15fe1-cc28-4704-ac49-9458945857ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 16:12:22.199454 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 1186/2339] 10.222.0.73 () {58 vars in 1015 bytes} [Mon Aug 22 16:12:22 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 60 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 16:12:22.213019 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32014b72-2713-42f4-a764-a4bff0472ab1 service neutron] Authenticating user token {{(pid=81767) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 16:12:22.214452 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.fernet_utils [None req-32014b72-2713-42f4-a764-a4bff0472ab1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81767) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 16:12:22.226677 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32014b72-2713-42f4-a764-a4bff0472ab1 service neutron] Validating token access rules against request {{(pid=81767) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 16:12:22.230480 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c605dd7-0ce6-4e4c-9764-a6774c238de7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '234687adad594f1aa45775a565ffbdc7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4ee0413f9ef4d4fb1f95a660babe090', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81767) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 16:12:22.231098 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c605dd7-0ce6-4e4c-9764-a6774c238de7 service neutron] REQUEST_METHOD: `GET` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:12:22.233647 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c605dd7-0ce6-4e4c-9764-a6774c238de7 service neutron] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:12:22.233647 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c605dd7-0ce6-4e4c-9764-a6774c238de7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:12:22.239753 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c605dd7-0ce6-4e4c-9764-a6774c238de7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 16:12:22.241168 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c605dd7-0ce6-4e4c-9764-a6774c238de7 service neutron] RBAC: Authorization granted {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 16:12:22.249431 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 1154/2340] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Aug 22 16:12:22 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 16:12:22.609743 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG dbcounter [-] [81767] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81767) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 16:12:22.691340 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 16:12:23.118468 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e57b56b-88ed-4e99-8b74-44dfc76280fb None None] REQUEST_METHOD: `POST` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:12:23.118802 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e57b56b-88ed-4e99-8b74-44dfc76280fb None None] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:12:23.119014 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e57b56b-88ed-4e99-8b74-44dfc76280fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:12:23.156109 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.auth.core [None req-3e57b56b-88ed-4e99-8b74-44dfc76280fb None None] MFA Rules not processed for user `01bb7d8843624abcbeba077d9ccdb6a6`. Rule list: `[]` (Enabled: `True`). {{(pid=81768) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 16:12:23.156345 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-3e57b56b-88ed-4e99-8b74-44dfc76280fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 16:12:23.233779 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 1187/2341] 10.222.0.15 () {58 vars in 958 bytes} [Mon Aug 22 16:12:23 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 16:12:23.238383 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c605dd7-0ce6-4e4c-9764-a6774c238de7 service neutron] Authenticating user token {{(pid=81767) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 16:12:23.253092 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c605dd7-0ce6-4e4c-9764-a6774c238de7 service neutron] Validating token access rules against request {{(pid=81767) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 16:12:23.257692 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ea1968d-f803-4c8e-a6a0-e92985dc2efb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a98b1e4431940ab8d1fc95df2858a6c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81767) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 16:12:23.258290 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ea1968d-f803-4c8e-a6a0-e92985dc2efb admin admin] REQUEST_METHOD: `POST` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:12:23.258567 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ea1968d-f803-4c8e-a6a0-e92985dc2efb admin admin] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:12:23.258852 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ea1968d-f803-4c8e-a6a0-e92985dc2efb admin admin] PATH_INFO: `/v3/projects` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:12:23.259504 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ea1968d-f803-4c8e-a6a0-e92985dc2efb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 16:12:23.261390 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ea1968d-f803-4c8e-a6a0-e92985dc2efb admin admin] RBAC: Authorization granted {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 16:12:23.337929 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c66cefc-e963-487d-ae5c-81feac4686ad None None] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:12:23.338516 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c66cefc-e963-487d-ae5c-81feac4686ad None None] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:12:23.338980 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c66cefc-e963-487d-ae5c-81feac4686ad None None] PATH_INFO: `/` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:12:23.343360 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 1188/2342] 10.222.0.15 () {54 vars in 921 bytes} [Mon Aug 22 16:12:23 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 16:12:23.363018 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cf65106-26bd-4922-928b-7e481b9541d1 None None] REQUEST_METHOD: `POST` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:12:23.363927 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cf65106-26bd-4922-928b-7e481b9541d1 None None] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:12:23.364212 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cf65106-26bd-4922-928b-7e481b9541d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:12:23.366181 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 1155/2343] 10.222.0.15 () {60 vars in 1153 bytes} [Mon Aug 22 16:12:23 2022] POST /identity/v3/projects => generated 371 bytes in 128 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 22 16:12:23.370260 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ea1968d-f803-4c8e-a6a0-e92985dc2efb admin admin] Authenticating user token {{(pid=81767) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 16:12:23.391689 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.auth.core [None req-9cf65106-26bd-4922-928b-7e481b9541d1 None None] MFA Rules not processed for user `e4ef569e41fb46d0b721bb1c7663616f`. Rule list: `[]` (Enabled: `True`). {{(pid=81768) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 16:12:23.394166 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-9cf65106-26bd-4922-928b-7e481b9541d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 16:12:23.482581 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ea1968d-f803-4c8e-a6a0-e92985dc2efb admin admin] Validating token access rules against request {{(pid=81767) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 16:12:23.503028 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcb07e2d-c8d3-44db-9b0a-97263b35b66a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a98b1e4431940ab8d1fc95df2858a6c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81767) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 16:12:23.504381 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcb07e2d-c8d3-44db-9b0a-97263b35b66a admin admin] REQUEST_METHOD: `POST` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:12:23.507423 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcb07e2d-c8d3-44db-9b0a-97263b35b66a admin admin] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:12:23.507955 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcb07e2d-c8d3-44db-9b0a-97263b35b66a admin admin] PATH_INFO: `/v3/users` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:12:23.508812 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcb07e2d-c8d3-44db-9b0a-97263b35b66a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 16:12:23.513796 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcb07e2d-c8d3-44db-9b0a-97263b35b66a admin admin] RBAC: Authorization granted {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 16:12:23.539551 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 1189/2344] 10.222.0.15 () {58 vars in 1018 bytes} [Mon Aug 22 16:12:23 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 16:12:23.558527 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cf65106-26bd-4922-928b-7e481b9541d1 None None] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 16:12:23.560341 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-9cf65106-26bd-4922-928b-7e481b9541d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 16:12:23.575731 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cf65106-26bd-4922-928b-7e481b9541d1 None None] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 16:12:23.582010 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66ab2c5d-9ab9-4a60-9730-2ce318e4165b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4ef569e41fb46d0b721bb1c7663616f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4ee0413f9ef4d4fb1f95a660babe090', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 16:12:23.582642 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66ab2c5d-9ab9-4a60-9730-2ce318e4165b service nova] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:12:23.582920 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66ab2c5d-9ab9-4a60-9730-2ce318e4165b service nova] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:12:23.583195 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66ab2c5d-9ab9-4a60-9730-2ce318e4165b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:12:23.594027 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66ab2c5d-9ab9-4a60-9730-2ce318e4165b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 16:12:23.596142 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66ab2c5d-9ab9-4a60-9730-2ce318e4165b service nova] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 16:12:23.615680 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 1190/2345] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Aug 22 16:12:23 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 57 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 22 16:12:23.623165 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 1156/2346] 10.222.0.15 () {60 vars in 1144 bytes} [Mon Aug 22 16:12:23 2022] POST /identity/v3/users => generated 292 bytes in 253 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 22 16:12:23.627077 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66ab2c5d-9ab9-4a60-9730-2ce318e4165b service nova] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 16:12:23.640116 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66ab2c5d-9ab9-4a60-9730-2ce318e4165b service nova] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 16:12:23.647245 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d27757b-a325-458b-8019-e77a38a442b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a98b1e4431940ab8d1fc95df2858a6c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 16:12:23.647877 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d27757b-a325-458b-8019-e77a38a442b1 admin admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:12:23.648156 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d27757b-a325-458b-8019-e77a38a442b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:12:23.648468 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d27757b-a325-458b-8019-e77a38a442b1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:12:23.649344 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d27757b-a325-458b-8019-e77a38a442b1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 16:12:23.650774 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d27757b-a325-458b-8019-e77a38a442b1 admin admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 16:12:23.662602 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 1191/2347] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Aug 22 16:12:23 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 22 16:12:23.668326 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcb07e2d-c8d3-44db-9b0a-97263b35b66a admin admin] Authenticating user token {{(pid=81767) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 16:12:23.690183 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcb07e2d-c8d3-44db-9b0a-97263b35b66a admin admin] Validating token access rules against request {{(pid=81767) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 16:12:23.700062 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-780268f1-597d-455b-9e32-bdbc3620855c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a98b1e4431940ab8d1fc95df2858a6c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81767) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 16:12:23.700745 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-780268f1-597d-455b-9e32-bdbc3620855c admin admin] REQUEST_METHOD: `PUT` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:12:23.701027 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-780268f1-597d-455b-9e32-bdbc3620855c admin admin] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:12:23.701300 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-780268f1-597d-455b-9e32-bdbc3620855c admin admin] PATH_INFO: `/v3/projects/e807ae4c0f2d4c67a3a7da815459c063/users/a165e8e56b4f4cdb8f6eb4831bf0feeb/roles/9d9848cfd22e4b089db443c350e56670` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:12:23.721202 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-780268f1-597d-455b-9e32-bdbc3620855c admin admin] RBAC: Authorizing `identity:create_grant(project_id=e807ae4c0f2d4c67a3a7da815459c063, user_id=a165e8e56b4f4cdb8f6eb4831bf0feeb, role_id=9d9848cfd22e4b089db443c350e56670)` {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 16:12:23.724466 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-780268f1-597d-455b-9e32-bdbc3620855c admin admin] RBAC: Authorization granted {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 16:12:23.787536 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 1157/2348] 10.222.0.15 () {60 vars in 1483 bytes} [Mon Aug 22 16:12:23 2022] PUT /identity/v3/projects/e807ae4c0f2d4c67a3a7da815459c063/users/a165e8e56b4f4cdb8f6eb4831bf0feeb/roles/9d9848cfd22e4b089db443c350e56670 => generated 0 bytes in 120 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 16:12:23.791725 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d27757b-a325-458b-8019-e77a38a442b1 admin admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 16:12:23.802225 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d27757b-a325-458b-8019-e77a38a442b1 admin admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 16:12:23.868132 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01b01ba7-f751-4eee-b945-53a99aeeee5d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a98b1e4431940ab8d1fc95df2858a6c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 16:12:23.868641 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01b01ba7-f751-4eee-b945-53a99aeeee5d admin admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:12:23.868905 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01b01ba7-f751-4eee-b945-53a99aeeee5d admin admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:12:23.869221 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01b01ba7-f751-4eee-b945-53a99aeeee5d admin admin] PATH_INFO: `/v3/roles` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:12:23.869849 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01b01ba7-f751-4eee-b945-53a99aeeee5d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 16:12:23.871595 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01b01ba7-f751-4eee-b945-53a99aeeee5d admin admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 16:12:23.883365 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 1192/2349] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Aug 22 16:12:23 2022] GET /identity/v3/roles => generated 1420 bytes in 92 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 22 16:12:23.887707 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG dbcounter [-] [81767] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81767) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 16:12:23.890090 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-780268f1-597d-455b-9e32-bdbc3620855c admin admin] Authenticating user token {{(pid=81767) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 16:12:23.901358 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-780268f1-597d-455b-9e32-bdbc3620855c admin admin] Validating token access rules against request {{(pid=81767) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 16:12:23.906671 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61ca1351-4678-41c2-b960-637dac5c993d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a98b1e4431940ab8d1fc95df2858a6c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81767) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 16:12:23.907444 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61ca1351-4678-41c2-b960-637dac5c993d admin admin] REQUEST_METHOD: `PUT` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:12:23.907826 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61ca1351-4678-41c2-b960-637dac5c993d admin admin] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:12:23.908291 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61ca1351-4678-41c2-b960-637dac5c993d admin admin] PATH_INFO: `/v3/projects/e807ae4c0f2d4c67a3a7da815459c063/users/a165e8e56b4f4cdb8f6eb4831bf0feeb/roles/9d9848cfd22e4b089db443c350e56670` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:12:23.910281 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61ca1351-4678-41c2-b960-637dac5c993d admin admin] RBAC: Authorizing `identity:create_grant(project_id=e807ae4c0f2d4c67a3a7da815459c063, user_id=a165e8e56b4f4cdb8f6eb4831bf0feeb, role_id=9d9848cfd22e4b089db443c350e56670)` {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 16:12:23.913034 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61ca1351-4678-41c2-b960-637dac5c993d admin admin] RBAC: Authorization granted {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 16:12:23.926593 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 1158/2350] 10.222.0.15 () {60 vars in 1483 bytes} [Mon Aug 22 16:12:23 2022] PUT /identity/v3/projects/e807ae4c0f2d4c67a3a7da815459c063/users/a165e8e56b4f4cdb8f6eb4831bf0feeb/roles/9d9848cfd22e4b089db443c350e56670 => generated 0 bytes in 37 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 16:12:23.941830 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 16:12:24.686918 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG dbcounter [-] [81767] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81767) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 16:12:24.864314 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 16:12:25.450963 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f0e2d9e-2714-4df7-803e-525ac4c03954 None None] REQUEST_METHOD: `POST` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:12:25.451254 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f0e2d9e-2714-4df7-803e-525ac4c03954 None None] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:12:25.451559 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f0e2d9e-2714-4df7-803e-525ac4c03954 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:12:25.482522 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.auth.core [None req-8f0e2d9e-2714-4df7-803e-525ac4c03954 None None] MFA Rules not processed for user `a165e8e56b4f4cdb8f6eb4831bf0feeb`. Rule list: `[]` (Enabled: `True`). {{(pid=81768) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 16:12:25.483771 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-8f0e2d9e-2714-4df7-803e-525ac4c03954 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 16:12:25.569855 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 1193/2351] 10.222.0.15 () {58 vars in 958 bytes} [Mon Aug 22 16:12:25 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 16:12:25.582540 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61ca1351-4678-41c2-b960-637dac5c993d admin admin] Authenticating user token {{(pid=81767) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 16:12:25.585066 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.fernet_utils [None req-61ca1351-4678-41c2-b960-637dac5c993d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81767) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 16:12:25.662637 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61ca1351-4678-41c2-b960-637dac5c993d admin admin] Validating token access rules against request {{(pid=81767) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 16:12:25.669722 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7d3c4ad-c83c-4aef-b4d9-b27a871d44d4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79706d480f864998b9f8594be407ceab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4ee0413f9ef4d4fb1f95a660babe090', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81767) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 16:12:25.670364 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7d3c4ad-c83c-4aef-b4d9-b27a871d44d4 service cinder] REQUEST_METHOD: `GET` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:12:25.670646 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7d3c4ad-c83c-4aef-b4d9-b27a871d44d4 service cinder] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:12:25.670916 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7d3c4ad-c83c-4aef-b4d9-b27a871d44d4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:12:25.735818 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7d3c4ad-c83c-4aef-b4d9-b27a871d44d4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 16:12:25.737190 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7d3c4ad-c83c-4aef-b4d9-b27a871d44d4 service cinder] RBAC: Authorization granted {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 16:12:25.746126 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 1159/2352] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Aug 22 16:12:25 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 164 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 16:12:25.898313 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG dbcounter [-] [81767] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81767) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 16:12:25.973273 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 16:12:26.647173 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG dbcounter [-] [81767] Writing DB stats keystone:SELECT=80,keystone:DELETE=17,keystone:INSERT=3 {{(pid=81767) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 16:12:26.676224 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 16:12:26.912596 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG dbcounter [-] [81767] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81767) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 16:12:26.935099 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 16:12:27.003780 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG dbcounter [-] [81767] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81767) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 16:12:27.026592 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 16:12:27.117326 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 16:12:27.147033 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG dbcounter [-] [81767] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=81767) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 16:12:27.375064 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 16:12:27.432736 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG dbcounter [-] [81767] Writing DB stats keystone:SELECT=83,keystone:DELETE=14,keystone:INSERT=3 {{(pid=81767) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 16:12:27.563795 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=91,keystone:DELETE=4,keystone:INSERT=5 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 16:12:27.660847 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG dbcounter [-] [81767] Writing DB stats keystone:SELECT=86,keystone:DELETE=10,keystone:INSERT=4 {{(pid=81767) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 16:12:27.769855 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=98,keystone:INSERT=1,keystone:DELETE=1 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 16:12:27.871918 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG dbcounter [-] [81767] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81767) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 16:12:27.958344 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:DELETE=6,keystone:SELECT=89,keystone:INSERT=5 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 16:12:27.987430 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG dbcounter [-] [81767] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=81767) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 16:12:28.249629 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 16:12:28.285698 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG dbcounter [-] [81767] Writing DB stats keystone:SELECT=82,keystone:DELETE=16,keystone:INSERT=2 {{(pid=81767) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 16:12:28.378154 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=93,keystone:DELETE=6,keystone:INSERT=1 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 16:12:28.524208 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG dbcounter [-] [81767] Writing DB stats keystone:SELECT=93,keystone:INSERT=2,keystone:DELETE=5 {{(pid=81767) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 16:12:28.627036 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:DELETE=1,keystone:SELECT=95,keystone:INSERT=4 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 16:12:28.792368 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG dbcounter [-] [81767] Writing DB stats keystone:DELETE=12,keystone:SELECT=86,keystone:INSERT=2 {{(pid=81767) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 16:12:29.089537 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 16:12:29.252862 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG dbcounter [-] [81767] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=81767) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 16:12:29.284901 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 16:12:29.548136 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 16:12:29.677033 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f0e2d9e-2714-4df7-803e-525ac4c03954 None None] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 16:12:29.679231 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-8f0e2d9e-2714-4df7-803e-525ac4c03954 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 16:12:29.749250 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f0e2d9e-2714-4df7-803e-525ac4c03954 None None] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 16:12:29.754114 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac4e0a21-b10c-4c44-ab28-4cd1b0a3b865 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a98b1e4431940ab8d1fc95df2858a6c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 16:12:29.754740 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac4e0a21-b10c-4c44-ab28-4cd1b0a3b865 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:12:29.755041 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac4e0a21-b10c-4c44-ab28-4cd1b0a3b865 admin admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:12:29.755350 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac4e0a21-b10c-4c44-ab28-4cd1b0a3b865 admin admin] PATH_INFO: `/v3/users/371289161199421ea28a7688ef40d5ad` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:12:29.756476 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac4e0a21-b10c-4c44-ab28-4cd1b0a3b865 admin admin] RBAC: Authorizing `identity:delete_user(user_id=371289161199421ea28a7688ef40d5ad)` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 16:12:29.758453 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac4e0a21-b10c-4c44-ab28-4cd1b0a3b865 admin admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 16:12:29.886809 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=100 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 16:12:29.942396 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-ac4e0a21-b10c-4c44-ab28-4cd1b0a3b865 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '371289161199421ea28a7688ef40d5ad'} {{(pid=81768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 16:12:29.943048 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-ac4e0a21-b10c-4c44-ab28-4cd1b0a3b865 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '371289161199421ea28a7688ef40d5ad'} {{(pid=81768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 16:12:29.943658 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-ac4e0a21-b10c-4c44-ab28-4cd1b0a3b865 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '371289161199421ea28a7688ef40d5ad'} {{(pid=81768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 16:12:29.950732 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-ac4e0a21-b10c-4c44-ab28-4cd1b0a3b865 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '371289161199421ea28a7688ef40d5ad'} {{(pid=81768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 16:12:29.995681 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=100 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 16:12:30.023676 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-ac4e0a21-b10c-4c44-ab28-4cd1b0a3b865 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '371289161199421ea28a7688ef40d5ad'} {{(pid=81768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 16:12:30.042187 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 1194/2353] 10.222.0.15 () {58 vars in 1224 bytes} [Mon Aug 22 16:12:29 2022] DELETE /identity/v3/users/371289161199421ea28a7688ef40d5ad => generated 0 bytes in 366 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 22 16:12:30.047171 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82c7056c-71db-4213-ae24-2c4a306166a1 None None] REQUEST_METHOD: `POST` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:12:30.047492 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82c7056c-71db-4213-ae24-2c4a306166a1 None None] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:12:30.047770 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82c7056c-71db-4213-ae24-2c4a306166a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:12:30.055114 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG dbcounter [-] [81767] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81767) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 16:12:30.068578 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.auth.core [None req-82c7056c-71db-4213-ae24-2c4a306166a1 None None] MFA Rules not processed for user `6b2199ae0fb0480aa917ada391f5ec54`. Rule list: `[]` (Enabled: `True`). {{(pid=81767) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 16:12:30.069888 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.fernet_utils [None req-82c7056c-71db-4213-ae24-2c4a306166a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81767) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 16:12:30.130860 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 1160/2354] 10.222.0.15 () {58 vars in 958 bytes} [Mon Aug 22 16:12:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 22 16:12:30.141626 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac4e0a21-b10c-4c44-ab28-4cd1b0a3b865 admin admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 16:12:30.143708 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-ac4e0a21-b10c-4c44-ab28-4cd1b0a3b865 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 16:12:30.201954 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac4e0a21-b10c-4c44-ab28-4cd1b0a3b865 admin admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 16:12:30.206187 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7147d65-7cd0-4724-b610-386707d9260f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '234687adad594f1aa45775a565ffbdc7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4ee0413f9ef4d4fb1f95a660babe090', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 16:12:30.206787 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7147d65-7cd0-4724-b610-386707d9260f service neutron] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:12:30.207065 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7147d65-7cd0-4724-b610-386707d9260f service neutron] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:12:30.207415 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7147d65-7cd0-4724-b610-386707d9260f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:12:30.217302 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7147d65-7cd0-4724-b610-386707d9260f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 16:12:30.218774 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7147d65-7cd0-4724-b610-386707d9260f service neutron] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 16:12:30.228708 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 1195/2355] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Aug 22 16:12:30 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 16:12:30.626042 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82c7056c-71db-4213-ae24-2c4a306166a1 None None] Authenticating user token {{(pid=81767) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 16:12:30.630580 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.fernet_utils [None req-82c7056c-71db-4213-ae24-2c4a306166a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81767) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 16:12:30.641853 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82c7056c-71db-4213-ae24-2c4a306166a1 None None] Validating token access rules against request {{(pid=81767) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 16:12:30.647150 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-395fca9d-187a-48f5-892f-f6b3ba17db78 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a98b1e4431940ab8d1fc95df2858a6c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81767) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 16:12:30.647883 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-395fca9d-187a-48f5-892f-f6b3ba17db78 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:12:30.648327 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-395fca9d-187a-48f5-892f-f6b3ba17db78 admin admin] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:12:30.648615 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-395fca9d-187a-48f5-892f-f6b3ba17db78 admin admin] PATH_INFO: `/v3/projects/169c9dcaffbe4ede89fbf23768666ad2` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:12:30.650671 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-395fca9d-187a-48f5-892f-f6b3ba17db78 admin admin] RBAC: Authorizing `identity:delete_project(project_id=169c9dcaffbe4ede89fbf23768666ad2)` {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 16:12:30.658943 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-395fca9d-187a-48f5-892f-f6b3ba17db78 admin admin] RBAC: Authorization granted {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 16:12:30.796554 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.notifications [None req-395fca9d-187a-48f5-892f-f6b3ba17db78 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '169c9dcaffbe4ede89fbf23768666ad2', 'request_id': 'req-395fca9d-187a-48f5-892f-f6b3ba17db78'} {{(pid=81767) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 16:12:30.805578 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.notifications [None req-395fca9d-187a-48f5-892f-f6b3ba17db78 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '169c9dcaffbe4ede89fbf23768666ad2', 'request_id': 'req-395fca9d-187a-48f5-892f-f6b3ba17db78'} {{(pid=81767) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 16:12:30.872881 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.notifications [None req-395fca9d-187a-48f5-892f-f6b3ba17db78 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '169c9dcaffbe4ede89fbf23768666ad2', 'request_id': 'req-395fca9d-187a-48f5-892f-f6b3ba17db78'} {{(pid=81767) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 16:12:30.888235 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.notifications [None req-395fca9d-187a-48f5-892f-f6b3ba17db78 admin admin] The token cache is being invalidate because project 169c9dcaffbe4ede89fbf23768666ad2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81767) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 22 16:12:30.888914 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.notifications [None req-395fca9d-187a-48f5-892f-f6b3ba17db78 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81767) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 16:12:30.889750 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.notifications [None req-395fca9d-187a-48f5-892f-f6b3ba17db78 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81767) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 16:12:30.891836 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 1161/2356] 10.222.0.15 () {58 vars in 1233 bytes} [Mon Aug 22 16:12:30 2022] DELETE /identity/v3/projects/169c9dcaffbe4ede89fbf23768666ad2 => generated 0 bytes in 266 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 16:12:30.895803 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7147d65-7cd0-4724-b610-386707d9260f service neutron] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 16:12:30.906426 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7147d65-7cd0-4724-b610-386707d9260f service neutron] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 16:12:30.968405 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c15e0345-911e-4376-98e6-1cb9630b2ed3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a98b1e4431940ab8d1fc95df2858a6c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 16:12:30.969024 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c15e0345-911e-4376-98e6-1cb9630b2ed3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:12:30.969301 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c15e0345-911e-4376-98e6-1cb9630b2ed3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:12:30.969572 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c15e0345-911e-4376-98e6-1cb9630b2ed3 admin admin] PATH_INFO: `/v3/users/622e68277ba74bd98a0fce9a20e6991b` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:12:30.971301 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c15e0345-911e-4376-98e6-1cb9630b2ed3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=622e68277ba74bd98a0fce9a20e6991b)` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 16:12:30.973229 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c15e0345-911e-4376-98e6-1cb9630b2ed3 admin admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 16:12:31.051127 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=84,keystone:DELETE=15,keystone:INSERT=1 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 16:12:31.212614 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-c15e0345-911e-4376-98e6-1cb9630b2ed3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '622e68277ba74bd98a0fce9a20e6991b'} {{(pid=81768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 16:12:31.213247 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-c15e0345-911e-4376-98e6-1cb9630b2ed3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '622e68277ba74bd98a0fce9a20e6991b'} {{(pid=81768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 16:12:31.213857 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-c15e0345-911e-4376-98e6-1cb9630b2ed3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '622e68277ba74bd98a0fce9a20e6991b'} {{(pid=81768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 16:12:31.222078 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-c15e0345-911e-4376-98e6-1cb9630b2ed3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '622e68277ba74bd98a0fce9a20e6991b'} {{(pid=81768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 16:12:31.275601 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-c15e0345-911e-4376-98e6-1cb9630b2ed3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '622e68277ba74bd98a0fce9a20e6991b'} {{(pid=81768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 16:12:31.294531 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 1196/2357] 10.222.0.15 () {58 vars in 1224 bytes} [Mon Aug 22 16:12:30 2022] DELETE /identity/v3/users/622e68277ba74bd98a0fce9a20e6991b => generated 0 bytes in 399 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 16:12:31.759713 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-395fca9d-187a-48f5-892f-f6b3ba17db78 admin admin] Authenticating user token {{(pid=81767) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 16:12:31.759713 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.fernet_utils [None req-395fca9d-187a-48f5-892f-f6b3ba17db78 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81767) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 16:12:31.773153 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-395fca9d-187a-48f5-892f-f6b3ba17db78 admin admin] Validating token access rules against request {{(pid=81767) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 16:12:31.842370 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f7aa3e2-56e7-4311-a0d7-012c838022bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a98b1e4431940ab8d1fc95df2858a6c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81767) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 16:12:31.845982 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f7aa3e2-56e7-4311-a0d7-012c838022bb admin admin] REQUEST_METHOD: `DELETE` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:12:31.845982 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f7aa3e2-56e7-4311-a0d7-012c838022bb admin admin] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:12:31.845982 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f7aa3e2-56e7-4311-a0d7-012c838022bb admin admin] PATH_INFO: `/v3/projects/42eb15f24a664ac997543e7df9fb0205` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:12:31.846430 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f7aa3e2-56e7-4311-a0d7-012c838022bb admin admin] RBAC: Authorizing `identity:delete_project(project_id=42eb15f24a664ac997543e7df9fb0205)` {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 16:12:31.849736 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f7aa3e2-56e7-4311-a0d7-012c838022bb admin admin] RBAC: Authorization granted {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 16:12:31.955365 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG dbcounter [-] [81767] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=81767) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 16:12:31.974961 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.notifications [None req-4f7aa3e2-56e7-4311-a0d7-012c838022bb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '42eb15f24a664ac997543e7df9fb0205', 'request_id': 'req-4f7aa3e2-56e7-4311-a0d7-012c838022bb'} {{(pid=81767) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 16:12:31.981466 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.notifications [None req-4f7aa3e2-56e7-4311-a0d7-012c838022bb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '42eb15f24a664ac997543e7df9fb0205', 'request_id': 'req-4f7aa3e2-56e7-4311-a0d7-012c838022bb'} {{(pid=81767) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 16:12:32.071710 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.notifications [None req-4f7aa3e2-56e7-4311-a0d7-012c838022bb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '42eb15f24a664ac997543e7df9fb0205', 'request_id': 'req-4f7aa3e2-56e7-4311-a0d7-012c838022bb'} {{(pid=81767) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 16:12:32.085368 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.notifications [None req-4f7aa3e2-56e7-4311-a0d7-012c838022bb admin admin] The token cache is being invalidate because project 42eb15f24a664ac997543e7df9fb0205 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81767) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 22 16:12:32.085667 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.notifications [None req-4f7aa3e2-56e7-4311-a0d7-012c838022bb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81767) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 16:12:32.086300 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.notifications [None req-4f7aa3e2-56e7-4311-a0d7-012c838022bb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81767) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 16:12:32.088108 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 1162/2358] 10.222.0.15 () {58 vars in 1233 bytes} [Mon Aug 22 16:12:31 2022] DELETE /identity/v3/projects/42eb15f24a664ac997543e7df9fb0205 => generated 0 bytes in 331 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 16:12:32.092787 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4e31bd5-d9cc-4b0e-8a7e-b495af216061 None None] REQUEST_METHOD: `POST` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:12:32.093340 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4e31bd5-d9cc-4b0e-8a7e-b495af216061 None None] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:12:32.093435 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4e31bd5-d9cc-4b0e-8a7e-b495af216061 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:12:32.113988 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.auth.core [None req-b4e31bd5-d9cc-4b0e-8a7e-b495af216061 None None] MFA Rules not processed for user `6b2199ae0fb0480aa917ada391f5ec54`. Rule list: `[]` (Enabled: `True`). {{(pid=81768) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 16:12:32.115239 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-b4e31bd5-d9cc-4b0e-8a7e-b495af216061 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 16:12:32.175960 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 1197/2359] 10.222.0.15 () {58 vars in 958 bytes} [Mon Aug 22 16:12:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 16:12:32.183596 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a6b98f8-f50d-4dbb-aff4-dc12b1d41ee9 None None] REQUEST_METHOD: `POST` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:12:32.183886 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a6b98f8-f50d-4dbb-aff4-dc12b1d41ee9 None None] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:12:32.184168 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a6b98f8-f50d-4dbb-aff4-dc12b1d41ee9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:12:32.204604 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.auth.core [None req-4a6b98f8-f50d-4dbb-aff4-dc12b1d41ee9 None None] MFA Rules not processed for user `6b2199ae0fb0480aa917ada391f5ec54`. Rule list: `[]` (Enabled: `True`). {{(pid=81767) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 16:12:32.206521 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.fernet_utils [None req-4a6b98f8-f50d-4dbb-aff4-dc12b1d41ee9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81767) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 16:12:32.214878 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 1163/2360] 10.222.0.15 () {58 vars in 958 bytes} [Mon Aug 22 16:12:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 16:12:32.219241 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4e31bd5-d9cc-4b0e-8a7e-b495af216061 None None] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 16:12:32.229090 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4e31bd5-d9cc-4b0e-8a7e-b495af216061 None None] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 16:12:32.233591 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48edfd0d-7fae-48c9-962e-cb065252d161 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a98b1e4431940ab8d1fc95df2858a6c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 16:12:32.234181 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48edfd0d-7fae-48c9-962e-cb065252d161 admin admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:12:32.234458 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48edfd0d-7fae-48c9-962e-cb065252d161 admin admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:12:32.234755 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48edfd0d-7fae-48c9-962e-cb065252d161 admin admin] PATH_INFO: `/v3/domains` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:12:32.235440 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48edfd0d-7fae-48c9-962e-cb065252d161 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81768) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 16:12:32.235777 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48edfd0d-7fae-48c9-962e-cb065252d161 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 16:12:32.237367 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48edfd0d-7fae-48c9-962e-cb065252d161 admin admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 16:12:32.248949 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 1198/2361] 10.222.0.15 () {58 vars in 1153 bytes} [Mon Aug 22 16:12:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 16:12:32.253869 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a6b98f8-f50d-4dbb-aff4-dc12b1d41ee9 None None] Authenticating user token {{(pid=81767) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 16:12:32.263490 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a6b98f8-f50d-4dbb-aff4-dc12b1d41ee9 None None] Validating token access rules against request {{(pid=81767) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 16:12:32.268850 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7285bbe4-827e-45ba-979d-ac8ac6d24ff0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a98b1e4431940ab8d1fc95df2858a6c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81767) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 16:12:32.269449 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7285bbe4-827e-45ba-979d-ac8ac6d24ff0 admin admin] REQUEST_METHOD: `POST` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:12:32.269722 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7285bbe4-827e-45ba-979d-ac8ac6d24ff0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:12:32.269987 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7285bbe4-827e-45ba-979d-ac8ac6d24ff0 admin admin] PATH_INFO: `/v3/projects` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:12:32.270610 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7285bbe4-827e-45ba-979d-ac8ac6d24ff0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 16:12:32.272558 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7285bbe4-827e-45ba-979d-ac8ac6d24ff0 admin admin] RBAC: Authorization granted {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 16:12:32.380379 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 1164/2362] 10.222.0.15 () {60 vars in 1153 bytes} [Mon Aug 22 16:12:32 2022] POST /identity/v3/projects => generated 371 bytes in 127 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 22 16:12:32.384718 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48edfd0d-7fae-48c9-962e-cb065252d161 admin admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 16:12:32.395033 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48edfd0d-7fae-48c9-962e-cb065252d161 admin admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 16:12:32.453239 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8864b227-746e-45a9-a779-8324e0855e97 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a98b1e4431940ab8d1fc95df2858a6c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 16:12:32.453836 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8864b227-746e-45a9-a779-8324e0855e97 admin admin] REQUEST_METHOD: `POST` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:12:32.454108 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8864b227-746e-45a9-a779-8324e0855e97 admin admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:12:32.454383 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8864b227-746e-45a9-a779-8324e0855e97 admin admin] PATH_INFO: `/v3/users` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:12:32.455542 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8864b227-746e-45a9-a779-8324e0855e97 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 16:12:32.457454 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8864b227-746e-45a9-a779-8324e0855e97 admin admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 16:12:32.575257 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 1199/2363] 10.222.0.15 () {60 vars in 1144 bytes} [Mon Aug 22 16:12:32 2022] POST /identity/v3/users => generated 286 bytes in 191 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 22 16:12:32.579341 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7285bbe4-827e-45ba-979d-ac8ac6d24ff0 admin admin] Authenticating user token {{(pid=81767) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 16:12:32.590784 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7285bbe4-827e-45ba-979d-ac8ac6d24ff0 admin admin] Validating token access rules against request {{(pid=81767) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 16:12:32.595974 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f047c2a-c6c9-403c-8300-03c84b52c175 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a98b1e4431940ab8d1fc95df2858a6c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81767) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 16:12:32.596572 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f047c2a-c6c9-403c-8300-03c84b52c175 admin admin] REQUEST_METHOD: `GET` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:12:32.596852 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f047c2a-c6c9-403c-8300-03c84b52c175 admin admin] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:12:32.597132 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f047c2a-c6c9-403c-8300-03c84b52c175 admin admin] PATH_INFO: `/v3/roles` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:12:32.597775 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f047c2a-c6c9-403c-8300-03c84b52c175 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 16:12:32.599383 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f047c2a-c6c9-403c-8300-03c84b52c175 admin admin] RBAC: Authorization granted {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 16:12:32.609729 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 1165/2364] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Aug 22 16:12:32 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 22 16:12:32.613973 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8864b227-746e-45a9-a779-8324e0855e97 admin admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 16:12:32.626703 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8864b227-746e-45a9-a779-8324e0855e97 admin admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 16:12:32.631844 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0cc5592-6610-4c3e-a2b2-c22fce113e89 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a98b1e4431940ab8d1fc95df2858a6c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 16:12:32.632433 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0cc5592-6610-4c3e-a2b2-c22fce113e89 admin admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:12:32.632708 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0cc5592-6610-4c3e-a2b2-c22fce113e89 admin admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:12:32.632971 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0cc5592-6610-4c3e-a2b2-c22fce113e89 admin admin] PATH_INFO: `/v3/roles` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:12:32.633598 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0cc5592-6610-4c3e-a2b2-c22fce113e89 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 16:12:32.635187 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0cc5592-6610-4c3e-a2b2-c22fce113e89 admin admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 16:12:32.646854 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 1200/2365] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Aug 22 16:12:32 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 22 16:12:32.652163 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f047c2a-c6c9-403c-8300-03c84b52c175 admin admin] Authenticating user token {{(pid=81767) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 16:12:32.663056 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f047c2a-c6c9-403c-8300-03c84b52c175 admin admin] Validating token access rules against request {{(pid=81767) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 16:12:32.668046 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9282ea9-9b19-4f58-8c28-37827f0f2ff4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a98b1e4431940ab8d1fc95df2858a6c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81767) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 16:12:32.668694 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9282ea9-9b19-4f58-8c28-37827f0f2ff4 admin admin] REQUEST_METHOD: `PUT` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:12:32.668971 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9282ea9-9b19-4f58-8c28-37827f0f2ff4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:12:32.669245 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9282ea9-9b19-4f58-8c28-37827f0f2ff4 admin admin] PATH_INFO: `/v3/projects/63e19642c827490b8cf5a0898780fa79/users/f94920c3945c4da5a8fd1b02fd90bc5a/roles/e6fc1ee3fb974125ba9b4fe55b12b45e` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:12:32.684967 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9282ea9-9b19-4f58-8c28-37827f0f2ff4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=63e19642c827490b8cf5a0898780fa79, user_id=f94920c3945c4da5a8fd1b02fd90bc5a, role_id=e6fc1ee3fb974125ba9b4fe55b12b45e)` {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 16:12:32.687789 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9282ea9-9b19-4f58-8c28-37827f0f2ff4 admin admin] RBAC: Authorization granted {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 16:12:32.752343 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 1166/2366] 10.222.0.15 () {60 vars in 1483 bytes} [Mon Aug 22 16:12:32 2022] PUT /identity/v3/projects/63e19642c827490b8cf5a0898780fa79/users/f94920c3945c4da5a8fd1b02fd90bc5a/roles/e6fc1ee3fb974125ba9b4fe55b12b45e => generated 0 bytes in 101 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 16:12:32.762958 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6da2ce2-ba4c-45b0-87b1-0e94db23b40b None None] REQUEST_METHOD: `POST` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:12:32.763144 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6da2ce2-ba4c-45b0-87b1-0e94db23b40b None None] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:12:32.763457 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6da2ce2-ba4c-45b0-87b1-0e94db23b40b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:12:32.785902 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=91,keystone:DELETE=5,keystone:INSERT=4 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 16:12:32.793949 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.auth.core [None req-e6da2ce2-ba4c-45b0-87b1-0e94db23b40b None None] MFA Rules not processed for user `f94920c3945c4da5a8fd1b02fd90bc5a`. Rule list: `[]` (Enabled: `True`). {{(pid=81768) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 16:12:32.795173 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-e6da2ce2-ba4c-45b0-87b1-0e94db23b40b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 16:12:32.863320 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 1201/2367] 10.222.0.15 () {58 vars in 958 bytes} [Mon Aug 22 16:12:32 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 16:12:32.875016 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9282ea9-9b19-4f58-8c28-37827f0f2ff4 admin admin] Authenticating user token {{(pid=81767) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 16:12:32.877437 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.fernet_utils [None req-c9282ea9-9b19-4f58-8c28-37827f0f2ff4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81767) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 16:12:32.940449 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9282ea9-9b19-4f58-8c28-37827f0f2ff4 admin admin] Validating token access rules against request {{(pid=81767) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 16:12:32.946034 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad3f585b-e66d-41e3-8378-e2e42d19080d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79706d480f864998b9f8594be407ceab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4ee0413f9ef4d4fb1f95a660babe090', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81767) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 16:12:32.946655 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad3f585b-e66d-41e3-8378-e2e42d19080d service cinder] REQUEST_METHOD: `GET` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:12:32.946941 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad3f585b-e66d-41e3-8378-e2e42d19080d service cinder] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:12:32.947197 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad3f585b-e66d-41e3-8378-e2e42d19080d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:12:32.957305 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad3f585b-e66d-41e3-8378-e2e42d19080d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 16:12:32.958723 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad3f585b-e66d-41e3-8378-e2e42d19080d service cinder] RBAC: Authorization granted {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 16:12:32.968594 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 1167/2368] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Aug 22 16:12:32 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 16:12:32.998157 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6da2ce2-ba4c-45b0-87b1-0e94db23b40b None None] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 16:12:33.062213 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6da2ce2-ba4c-45b0-87b1-0e94db23b40b None None] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 16:12:33.068128 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bb06879-0e6e-4e02-84f0-0cb9c9937d3e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a98b1e4431940ab8d1fc95df2858a6c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 16:12:33.068906 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bb06879-0e6e-4e02-84f0-0cb9c9937d3e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:12:33.069194 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bb06879-0e6e-4e02-84f0-0cb9c9937d3e admin admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:12:33.069474 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bb06879-0e6e-4e02-84f0-0cb9c9937d3e admin admin] PATH_INFO: `/v3/users/f94920c3945c4da5a8fd1b02fd90bc5a` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:12:33.070736 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bb06879-0e6e-4e02-84f0-0cb9c9937d3e admin admin] RBAC: Authorizing `identity:delete_user(user_id=f94920c3945c4da5a8fd1b02fd90bc5a)` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 16:12:33.072709 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bb06879-0e6e-4e02-84f0-0cb9c9937d3e admin admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 16:12:33.308336 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-3bb06879-0e6e-4e02-84f0-0cb9c9937d3e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f94920c3945c4da5a8fd1b02fd90bc5a'} {{(pid=81768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 16:12:33.308995 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-3bb06879-0e6e-4e02-84f0-0cb9c9937d3e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f94920c3945c4da5a8fd1b02fd90bc5a'} {{(pid=81768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 16:12:33.309541 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-3bb06879-0e6e-4e02-84f0-0cb9c9937d3e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f94920c3945c4da5a8fd1b02fd90bc5a'} {{(pid=81768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 16:12:33.317238 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-3bb06879-0e6e-4e02-84f0-0cb9c9937d3e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f94920c3945c4da5a8fd1b02fd90bc5a'} {{(pid=81768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 16:12:33.368754 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-3bb06879-0e6e-4e02-84f0-0cb9c9937d3e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f94920c3945c4da5a8fd1b02fd90bc5a'} {{(pid=81768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 16:12:33.388183 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 1202/2369] 10.222.0.15 () {58 vars in 1224 bytes} [Mon Aug 22 16:12:32 2022] DELETE /identity/v3/users/f94920c3945c4da5a8fd1b02fd90bc5a => generated 0 bytes in 398 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 16:12:33.392964 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac33175-170a-4639-8f7b-c2da4b00645a None None] REQUEST_METHOD: `POST` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:12:33.393253 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac33175-170a-4639-8f7b-c2da4b00645a None None] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:12:33.393531 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac33175-170a-4639-8f7b-c2da4b00645a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:12:33.413368 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.auth.core [None req-dac33175-170a-4639-8f7b-c2da4b00645a None None] MFA Rules not processed for user `6b2199ae0fb0480aa917ada391f5ec54`. Rule list: `[]` (Enabled: `True`). {{(pid=81767) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 16:12:33.414598 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.fernet_utils [None req-dac33175-170a-4639-8f7b-c2da4b00645a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81767) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 16:12:33.466486 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG dbcounter [-] [81767] Writing DB stats keystone:SELECT=92,keystone:DELETE=5,keystone:INSERT=3 {{(pid=81767) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 16:12:33.477873 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 1168/2370] 10.222.0.15 () {58 vars in 958 bytes} [Mon Aug 22 16:12:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 16:12:33.488449 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bb06879-0e6e-4e02-84f0-0cb9c9937d3e admin admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 16:12:33.489902 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-3bb06879-0e6e-4e02-84f0-0cb9c9937d3e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 16:12:33.513925 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 16:12:33.551812 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bb06879-0e6e-4e02-84f0-0cb9c9937d3e admin admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 16:12:33.556241 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c6cf48b-3d05-44e6-88a2-ae3bb4521ec6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '234687adad594f1aa45775a565ffbdc7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4ee0413f9ef4d4fb1f95a660babe090', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 16:12:33.556900 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c6cf48b-3d05-44e6-88a2-ae3bb4521ec6 service neutron] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:12:33.557204 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c6cf48b-3d05-44e6-88a2-ae3bb4521ec6 service neutron] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:12:33.557488 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c6cf48b-3d05-44e6-88a2-ae3bb4521ec6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:12:33.568810 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c6cf48b-3d05-44e6-88a2-ae3bb4521ec6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 16:12:33.570275 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c6cf48b-3d05-44e6-88a2-ae3bb4521ec6 service neutron] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 16:12:33.580849 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 1203/2371] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Aug 22 16:12:33 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 16:12:34.020632 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dac33175-170a-4639-8f7b-c2da4b00645a None None] Authenticating user token {{(pid=81767) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 16:12:34.022193 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.fernet_utils [None req-dac33175-170a-4639-8f7b-c2da4b00645a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81767) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 16:12:34.036045 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dac33175-170a-4639-8f7b-c2da4b00645a None None] Validating token access rules against request {{(pid=81767) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 16:12:34.044355 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec88dc55-e217-4102-afaa-11c56b1682fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a98b1e4431940ab8d1fc95df2858a6c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81767) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 16:12:34.050633 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec88dc55-e217-4102-afaa-11c56b1682fe admin admin] REQUEST_METHOD: `DELETE` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:12:34.050633 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec88dc55-e217-4102-afaa-11c56b1682fe admin admin] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:12:34.050633 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec88dc55-e217-4102-afaa-11c56b1682fe admin admin] PATH_INFO: `/v3/projects/63e19642c827490b8cf5a0898780fa79` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:12:34.050633 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec88dc55-e217-4102-afaa-11c56b1682fe admin admin] RBAC: Authorizing `identity:delete_project(project_id=63e19642c827490b8cf5a0898780fa79)` {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 16:12:34.050633 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec88dc55-e217-4102-afaa-11c56b1682fe admin admin] RBAC: Authorization granted {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 16:12:34.236107 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.notifications [None req-ec88dc55-e217-4102-afaa-11c56b1682fe admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '63e19642c827490b8cf5a0898780fa79', 'request_id': 'req-ec88dc55-e217-4102-afaa-11c56b1682fe'} {{(pid=81767) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 16:12:34.247616 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.notifications [None req-ec88dc55-e217-4102-afaa-11c56b1682fe admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '63e19642c827490b8cf5a0898780fa79', 'request_id': 'req-ec88dc55-e217-4102-afaa-11c56b1682fe'} {{(pid=81767) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 16:12:34.318072 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.notifications [None req-ec88dc55-e217-4102-afaa-11c56b1682fe admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '63e19642c827490b8cf5a0898780fa79', 'request_id': 'req-ec88dc55-e217-4102-afaa-11c56b1682fe'} {{(pid=81767) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 16:12:34.332979 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.notifications [None req-ec88dc55-e217-4102-afaa-11c56b1682fe admin admin] The token cache is being invalidate because project 63e19642c827490b8cf5a0898780fa79 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81767) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 22 16:12:34.333493 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.notifications [None req-ec88dc55-e217-4102-afaa-11c56b1682fe admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81767) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 16:12:34.335175 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.notifications [None req-ec88dc55-e217-4102-afaa-11c56b1682fe admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81767) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 16:12:34.340144 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 1169/2372] 10.222.0.15 () {58 vars in 1233 bytes} [Mon Aug 22 16:12:34 2022] DELETE /identity/v3/projects/63e19642c827490b8cf5a0898780fa79 => generated 0 bytes in 320 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 16:12:34.347460 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b07b153a-59e8-4b6e-8bdb-91a772d777a7 None None] REQUEST_METHOD: `POST` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:12:34.347460 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b07b153a-59e8-4b6e-8bdb-91a772d777a7 None None] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:12:34.347932 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b07b153a-59e8-4b6e-8bdb-91a772d777a7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:12:34.400501 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.auth.core [None req-b07b153a-59e8-4b6e-8bdb-91a772d777a7 None None] MFA Rules not processed for user `6b2199ae0fb0480aa917ada391f5ec54`. Rule list: `[]` (Enabled: `True`). {{(pid=81768) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 16:12:34.401768 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-b07b153a-59e8-4b6e-8bdb-91a772d777a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 16:12:34.466297 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 1204/2373] 10.222.0.15 () {58 vars in 958 bytes} [Mon Aug 22 16:12:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 16:12:34.475065 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af735536-0a45-4f00-b1e7-f4140830b302 None None] REQUEST_METHOD: `POST` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:12:34.475388 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af735536-0a45-4f00-b1e7-f4140830b302 None None] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:12:34.475695 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af735536-0a45-4f00-b1e7-f4140830b302 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:12:34.497630 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.auth.core [None req-af735536-0a45-4f00-b1e7-f4140830b302 None None] MFA Rules not processed for user `6b2199ae0fb0480aa917ada391f5ec54`. Rule list: `[]` (Enabled: `True`). {{(pid=81767) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 16:12:34.499288 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.fernet_utils [None req-af735536-0a45-4f00-b1e7-f4140830b302 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81767) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 16:12:34.508455 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 1170/2374] 10.222.0.15 () {58 vars in 958 bytes} [Mon Aug 22 16:12:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 16:12:34.515644 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b07b153a-59e8-4b6e-8bdb-91a772d777a7 None None] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 16:12:34.526738 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b07b153a-59e8-4b6e-8bdb-91a772d777a7 None None] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 16:12:34.531668 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e5c6ce8-758b-44ab-8da5-7ab7cf3d78ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a98b1e4431940ab8d1fc95df2858a6c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 16:12:34.532309 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e5c6ce8-758b-44ab-8da5-7ab7cf3d78ae admin admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:12:34.534614 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e5c6ce8-758b-44ab-8da5-7ab7cf3d78ae admin admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:12:34.534614 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e5c6ce8-758b-44ab-8da5-7ab7cf3d78ae admin admin] PATH_INFO: `/v3/domains` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:12:34.534614 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e5c6ce8-758b-44ab-8da5-7ab7cf3d78ae admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81768) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 16:12:34.534614 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e5c6ce8-758b-44ab-8da5-7ab7cf3d78ae admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 16:12:34.535614 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e5c6ce8-758b-44ab-8da5-7ab7cf3d78ae admin admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 16:12:34.548249 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 1205/2375] 10.222.0.15 () {58 vars in 1153 bytes} [Mon Aug 22 16:12:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 16:12:34.553511 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af735536-0a45-4f00-b1e7-f4140830b302 None None] Authenticating user token {{(pid=81767) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 16:12:34.570540 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af735536-0a45-4f00-b1e7-f4140830b302 None None] Validating token access rules against request {{(pid=81767) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 16:12:34.575438 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1628264-10cf-447e-a9a2-faddd3e40b6d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a98b1e4431940ab8d1fc95df2858a6c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81767) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 16:12:34.576096 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1628264-10cf-447e-a9a2-faddd3e40b6d admin admin] REQUEST_METHOD: `POST` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:12:34.576372 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1628264-10cf-447e-a9a2-faddd3e40b6d admin admin] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:12:34.576639 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1628264-10cf-447e-a9a2-faddd3e40b6d admin admin] PATH_INFO: `/v3/projects` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:12:34.577253 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1628264-10cf-447e-a9a2-faddd3e40b6d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 16:12:34.579119 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1628264-10cf-447e-a9a2-faddd3e40b6d admin admin] RBAC: Authorization granted {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 16:12:34.683139 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 1171/2376] 10.222.0.15 () {60 vars in 1153 bytes} [Mon Aug 22 16:12:34 2022] POST /identity/v3/projects => generated 363 bytes in 127 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 22 16:12:34.684435 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e5c6ce8-758b-44ab-8da5-7ab7cf3d78ae admin admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 16:12:34.695751 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e5c6ce8-758b-44ab-8da5-7ab7cf3d78ae admin admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 16:12:34.768349 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4306e5fc-a07e-46be-b97b-ae769906bc14 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a98b1e4431940ab8d1fc95df2858a6c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 16:12:34.769034 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4306e5fc-a07e-46be-b97b-ae769906bc14 admin admin] REQUEST_METHOD: `POST` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:12:34.769305 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4306e5fc-a07e-46be-b97b-ae769906bc14 admin admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:12:34.769831 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4306e5fc-a07e-46be-b97b-ae769906bc14 admin admin] PATH_INFO: `/v3/users` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:12:34.771245 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4306e5fc-a07e-46be-b97b-ae769906bc14 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 16:12:34.774056 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4306e5fc-a07e-46be-b97b-ae769906bc14 admin admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 16:12:34.875759 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 1206/2377] 10.222.0.15 () {60 vars in 1144 bytes} [Mon Aug 22 16:12:34 2022] POST /identity/v3/users => generated 282 bytes in 192 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 22 16:12:34.880497 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1628264-10cf-447e-a9a2-faddd3e40b6d admin admin] Authenticating user token {{(pid=81767) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 16:12:34.891118 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1628264-10cf-447e-a9a2-faddd3e40b6d admin admin] Validating token access rules against request {{(pid=81767) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 16:12:34.896175 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc6c4989-d705-4787-80b9-9f2ffc20b34c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a98b1e4431940ab8d1fc95df2858a6c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81767) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 16:12:34.896786 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc6c4989-d705-4787-80b9-9f2ffc20b34c admin admin] REQUEST_METHOD: `GET` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:12:34.897065 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc6c4989-d705-4787-80b9-9f2ffc20b34c admin admin] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:12:34.903418 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc6c4989-d705-4787-80b9-9f2ffc20b34c admin admin] PATH_INFO: `/v3/roles` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:12:34.903418 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc6c4989-d705-4787-80b9-9f2ffc20b34c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 16:12:34.903418 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc6c4989-d705-4787-80b9-9f2ffc20b34c admin admin] RBAC: Authorization granted {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 16:12:34.910638 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 1172/2378] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Aug 22 16:12:34 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 22 16:12:34.914536 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4306e5fc-a07e-46be-b97b-ae769906bc14 admin admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 16:12:34.925427 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4306e5fc-a07e-46be-b97b-ae769906bc14 admin admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 16:12:34.930769 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-212320de-20d9-4c5b-80c6-d55d415b65fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a98b1e4431940ab8d1fc95df2858a6c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 16:12:34.931229 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-212320de-20d9-4c5b-80c6-d55d415b65fa admin admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:12:34.931548 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-212320de-20d9-4c5b-80c6-d55d415b65fa admin admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:12:34.931795 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-212320de-20d9-4c5b-80c6-d55d415b65fa admin admin] PATH_INFO: `/v3/roles` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:12:34.932517 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-212320de-20d9-4c5b-80c6-d55d415b65fa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 16:12:34.934295 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-212320de-20d9-4c5b-80c6-d55d415b65fa admin admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 16:12:34.948145 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 1207/2379] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Aug 22 16:12:34 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 22 16:12:34.953490 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc6c4989-d705-4787-80b9-9f2ffc20b34c admin admin] Authenticating user token {{(pid=81767) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 16:12:34.966029 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc6c4989-d705-4787-80b9-9f2ffc20b34c admin admin] Validating token access rules against request {{(pid=81767) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 16:12:34.971591 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-331a7540-c880-485b-9f29-7005034d8e6a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a98b1e4431940ab8d1fc95df2858a6c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81767) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 16:12:34.972315 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-331a7540-c880-485b-9f29-7005034d8e6a admin admin] REQUEST_METHOD: `PUT` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:12:34.972604 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-331a7540-c880-485b-9f29-7005034d8e6a admin admin] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:12:34.972877 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-331a7540-c880-485b-9f29-7005034d8e6a admin admin] PATH_INFO: `/v3/projects/50af9c822c4343409d1063058379b125/users/5e670b956f0c4e4ea8a5ff139742d956/roles/e6fc1ee3fb974125ba9b4fe55b12b45e` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:12:34.989827 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-331a7540-c880-485b-9f29-7005034d8e6a admin admin] RBAC: Authorizing `identity:create_grant(project_id=50af9c822c4343409d1063058379b125, user_id=5e670b956f0c4e4ea8a5ff139742d956, role_id=e6fc1ee3fb974125ba9b4fe55b12b45e)` {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 16:12:34.992608 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-331a7540-c880-485b-9f29-7005034d8e6a admin admin] RBAC: Authorization granted {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 16:12:35.097948 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 1173/2380] 10.222.0.15 () {60 vars in 1483 bytes} [Mon Aug 22 16:12:34 2022] PUT /identity/v3/projects/50af9c822c4343409d1063058379b125/users/5e670b956f0c4e4ea8a5ff139742d956/roles/e6fc1ee3fb974125ba9b4fe55b12b45e => generated 0 bytes in 146 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 16:12:35.111268 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be1ac223-a915-42e3-910f-dc140efd709f None None] REQUEST_METHOD: `POST` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:12:35.111908 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be1ac223-a915-42e3-910f-dc140efd709f None None] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:12:35.112367 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be1ac223-a915-42e3-910f-dc140efd709f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:12:35.160775 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.auth.core [None req-be1ac223-a915-42e3-910f-dc140efd709f None None] MFA Rules not processed for user `5e670b956f0c4e4ea8a5ff139742d956`. Rule list: `[]` (Enabled: `True`). {{(pid=81768) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 16:12:35.162365 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-be1ac223-a915-42e3-910f-dc140efd709f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 16:12:35.193977 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 16:12:35.251770 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 1208/2381] 10.222.0.15 () {58 vars in 958 bytes} [Mon Aug 22 16:12:35 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 16:12:35.262727 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-331a7540-c880-485b-9f29-7005034d8e6a admin admin] Authenticating user token {{(pid=81767) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 16:12:35.264494 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.fernet_utils [None req-331a7540-c880-485b-9f29-7005034d8e6a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81767) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 16:12:35.331751 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-331a7540-c880-485b-9f29-7005034d8e6a admin admin] Validating token access rules against request {{(pid=81767) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 16:12:35.345565 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0e24152-ebca-4d3d-800d-44d1fc6523cd service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79706d480f864998b9f8594be407ceab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4ee0413f9ef4d4fb1f95a660babe090', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81767) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 16:12:35.346741 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0e24152-ebca-4d3d-800d-44d1fc6523cd service cinder] REQUEST_METHOD: `GET` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:12:35.346741 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0e24152-ebca-4d3d-800d-44d1fc6523cd service cinder] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:12:35.346980 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0e24152-ebca-4d3d-800d-44d1fc6523cd service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:12:35.358399 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0e24152-ebca-4d3d-800d-44d1fc6523cd service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 16:12:35.360376 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0e24152-ebca-4d3d-800d-44d1fc6523cd service cinder] RBAC: Authorization granted {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 16:12:35.373256 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 1174/2382] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Aug 22 16:12:35 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 16:12:36.010546 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be1ac223-a915-42e3-910f-dc140efd709f None None] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 16:12:36.012077 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-be1ac223-a915-42e3-910f-dc140efd709f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 16:12:36.024582 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be1ac223-a915-42e3-910f-dc140efd709f None None] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 16:12:36.028331 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab1ba90d-b6da-402b-853f-6eb74b7941a5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79706d480f864998b9f8594be407ceab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4ee0413f9ef4d4fb1f95a660babe090', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 16:12:36.029013 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab1ba90d-b6da-402b-853f-6eb74b7941a5 service cinder] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:12:36.029358 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab1ba90d-b6da-402b-853f-6eb74b7941a5 service cinder] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:12:36.029698 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab1ba90d-b6da-402b-853f-6eb74b7941a5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:12:36.031823 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-ab1ba90d-b6da-402b-853f-6eb74b7941a5 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 16:12:36.107204 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab1ba90d-b6da-402b-853f-6eb74b7941a5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 16:12:36.108748 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab1ba90d-b6da-402b-853f-6eb74b7941a5 service cinder] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 16:12:36.121003 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 1209/2383] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Aug 22 16:12:36 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 16:12:42.272273 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a558208f-c5c5-477b-b2ef-1aad67b83a48 None None] REQUEST_METHOD: `GET` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:12:42.272945 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a558208f-c5c5-477b-b2ef-1aad67b83a48 None None] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:12:42.272945 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a558208f-c5c5-477b-b2ef-1aad67b83a48 None None] PATH_INFO: `/` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:12:42.275158 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 1175/2384] 10.222.0.15 () {54 vars in 921 bytes} [Mon Aug 22 16:12:42 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 16:12:42.284493 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a68a54-6110-4211-85e4-885a8efaa5f7 None None] REQUEST_METHOD: `POST` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:12:42.285050 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a68a54-6110-4211-85e4-885a8efaa5f7 None None] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:12:42.285050 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a68a54-6110-4211-85e4-885a8efaa5f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:12:42.305525 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.auth.core [None req-07a68a54-6110-4211-85e4-885a8efaa5f7 None None] MFA Rules not processed for user `e4ef569e41fb46d0b721bb1c7663616f`. Rule list: `[]` (Enabled: `True`). {{(pid=81768) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 16:12:42.307352 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-07a68a54-6110-4211-85e4-885a8efaa5f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 16:12:42.316522 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a558208f-c5c5-477b-b2ef-1aad67b83a48 None None] Authenticating user token {{(pid=81767) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 16:12:42.328317 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a558208f-c5c5-477b-b2ef-1aad67b83a48 None None] Validating token access rules against request {{(pid=81767) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 16:12:42.332718 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afb3489a-fd32-4803-85ac-f310c81ea7a6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79706d480f864998b9f8594be407ceab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4ee0413f9ef4d4fb1f95a660babe090', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81767) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 16:12:42.334359 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb3489a-fd32-4803-85ac-f310c81ea7a6 service cinder] REQUEST_METHOD: `GET` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:12:42.334359 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb3489a-fd32-4803-85ac-f310c81ea7a6 service cinder] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:12:42.334359 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb3489a-fd32-4803-85ac-f310c81ea7a6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:12:42.339719 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.fernet_utils [None req-afb3489a-fd32-4803-85ac-f310c81ea7a6 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81767) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 16:12:42.353195 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG dbcounter [-] [81767] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81767) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 16:12:42.387799 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 1210/2385] 10.222.0.15 () {58 vars in 1018 bytes} [Mon Aug 22 16:12:42 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 16:12:42.400454 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07a68a54-6110-4211-85e4-885a8efaa5f7 None None] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 16:12:42.401947 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-07a68a54-6110-4211-85e4-885a8efaa5f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 16:12:42.409338 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afb3489a-fd32-4803-85ac-f310c81ea7a6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 16:12:42.410850 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afb3489a-fd32-4803-85ac-f310c81ea7a6 service cinder] RBAC: Authorization granted {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 16:12:42.419974 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07a68a54-6110-4211-85e4-885a8efaa5f7 None None] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 16:12:42.426583 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-835a4bab-67bf-42e3-97bf-8cade251f38c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4ef569e41fb46d0b721bb1c7663616f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4ee0413f9ef4d4fb1f95a660babe090', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 16:12:42.427403 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-835a4bab-67bf-42e3-97bf-8cade251f38c service nova] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:12:42.427494 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-835a4bab-67bf-42e3-97bf-8cade251f38c service nova] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:12:42.427808 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-835a4bab-67bf-42e3-97bf-8cade251f38c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:12:42.432202 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 1176/2386] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Aug 22 16:12:42 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 116 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 22 16:12:42.442111 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-835a4bab-67bf-42e3-97bf-8cade251f38c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 16:12:42.443930 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-835a4bab-67bf-42e3-97bf-8cade251f38c service nova] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 16:12:42.456970 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 1211/2387] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Aug 22 16:12:42 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 57 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 22 16:12:42.492706 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afb3489a-fd32-4803-85ac-f310c81ea7a6 service cinder] Authenticating user token {{(pid=81767) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 16:12:42.495550 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.fernet_utils [None req-afb3489a-fd32-4803-85ac-f310c81ea7a6 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81767) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 16:12:42.670073 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afb3489a-fd32-4803-85ac-f310c81ea7a6 service cinder] Validating token access rules against request {{(pid=81767) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 16:12:42.675738 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0fa5eba-a605-424c-991e-eb3f5876f4cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a98b1e4431940ab8d1fc95df2858a6c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81767) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 16:12:42.676977 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0fa5eba-a605-424c-991e-eb3f5876f4cf admin admin] REQUEST_METHOD: `DELETE` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:12:42.677444 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0fa5eba-a605-424c-991e-eb3f5876f4cf admin admin] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:12:42.677542 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0fa5eba-a605-424c-991e-eb3f5876f4cf admin admin] PATH_INFO: `/v3/users/e9d4e77022fe483999eefda45f27e410` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:12:42.678791 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0fa5eba-a605-424c-991e-eb3f5876f4cf admin admin] RBAC: Authorizing `identity:delete_user(user_id=e9d4e77022fe483999eefda45f27e410)` {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 16:12:42.680979 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0fa5eba-a605-424c-991e-eb3f5876f4cf admin admin] RBAC: Authorization granted {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 16:12:42.961951 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.notifications [None req-e0fa5eba-a605-424c-991e-eb3f5876f4cf admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e9d4e77022fe483999eefda45f27e410'} {{(pid=81767) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 16:12:42.962534 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.notifications [None req-e0fa5eba-a605-424c-991e-eb3f5876f4cf admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e9d4e77022fe483999eefda45f27e410'} {{(pid=81767) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 16:12:43.060434 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.notifications [None req-e0fa5eba-a605-424c-991e-eb3f5876f4cf admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e9d4e77022fe483999eefda45f27e410'} {{(pid=81767) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 16:12:43.074658 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.notifications [None req-e0fa5eba-a605-424c-991e-eb3f5876f4cf admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e9d4e77022fe483999eefda45f27e410'} {{(pid=81767) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 16:12:43.090805 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.notifications [None req-e0fa5eba-a605-424c-991e-eb3f5876f4cf admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e9d4e77022fe483999eefda45f27e410'} {{(pid=81767) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 16:12:43.097514 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 1177/2388] 10.222.0.15 () {58 vars in 1224 bytes} [Mon Aug 22 16:12:42 2022] DELETE /identity/v3/users/e9d4e77022fe483999eefda45f27e410 => generated 0 bytes in 606 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 22 16:12:43.102242 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5706f27a-3bb9-474e-8257-971cad2cbe2e None None] REQUEST_METHOD: `POST` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:12:43.102645 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5706f27a-3bb9-474e-8257-971cad2cbe2e None None] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:12:43.103058 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5706f27a-3bb9-474e-8257-971cad2cbe2e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:12:43.124102 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.auth.core [None req-5706f27a-3bb9-474e-8257-971cad2cbe2e None None] MFA Rules not processed for user `6b2199ae0fb0480aa917ada391f5ec54`. Rule list: `[]` (Enabled: `True`). {{(pid=81768) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 16:12:43.125449 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-5706f27a-3bb9-474e-8257-971cad2cbe2e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 16:12:43.160279 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=100 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 16:12:43.210893 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 1212/2389] 10.222.0.15 () {58 vars in 958 bytes} [Mon Aug 22 16:12:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 16:12:43.223767 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0fa5eba-a605-424c-991e-eb3f5876f4cf admin admin] Authenticating user token {{(pid=81767) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 16:12:43.225679 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.fernet_utils [None req-e0fa5eba-a605-424c-991e-eb3f5876f4cf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81767) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 16:12:43.267345 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG dbcounter [-] [81767] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81767) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 16:12:43.291922 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0fa5eba-a605-424c-991e-eb3f5876f4cf admin admin] Validating token access rules against request {{(pid=81767) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 16:12:43.296353 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fe5215c-ace7-45c0-8ae7-9b99ad036066 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '234687adad594f1aa45775a565ffbdc7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4ee0413f9ef4d4fb1f95a660babe090', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81767) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 16:12:43.296994 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fe5215c-ace7-45c0-8ae7-9b99ad036066 service neutron] REQUEST_METHOD: `GET` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:12:43.297278 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fe5215c-ace7-45c0-8ae7-9b99ad036066 service neutron] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:12:43.297554 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fe5215c-ace7-45c0-8ae7-9b99ad036066 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:12:43.307377 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fe5215c-ace7-45c0-8ae7-9b99ad036066 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 16:12:43.310502 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fe5215c-ace7-45c0-8ae7-9b99ad036066 service neutron] RBAC: Authorization granted {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 16:12:43.322881 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 1178/2390] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Aug 22 16:12:43 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 16:12:44.151619 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5706f27a-3bb9-474e-8257-971cad2cbe2e None None] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 16:12:44.153226 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-5706f27a-3bb9-474e-8257-971cad2cbe2e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 16:12:44.172192 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5706f27a-3bb9-474e-8257-971cad2cbe2e None None] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 16:12:44.178474 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cdf2e95-f2ec-4286-86ce-7ef198468931 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a98b1e4431940ab8d1fc95df2858a6c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 16:12:44.179116 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cdf2e95-f2ec-4286-86ce-7ef198468931 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:12:44.179863 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cdf2e95-f2ec-4286-86ce-7ef198468931 admin admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:12:44.179863 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cdf2e95-f2ec-4286-86ce-7ef198468931 admin admin] PATH_INFO: `/v3/projects/0e5c1343cecc4c6d9d17bd83d4d95116` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:12:44.181020 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cdf2e95-f2ec-4286-86ce-7ef198468931 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0e5c1343cecc4c6d9d17bd83d4d95116)` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 16:12:44.183079 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cdf2e95-f2ec-4286-86ce-7ef198468931 admin admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 16:12:44.338265 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-4cdf2e95-f2ec-4286-86ce-7ef198468931 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0e5c1343cecc4c6d9d17bd83d4d95116', 'request_id': 'req-4cdf2e95-f2ec-4286-86ce-7ef198468931'} {{(pid=81768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 16:12:44.345858 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-4cdf2e95-f2ec-4286-86ce-7ef198468931 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0e5c1343cecc4c6d9d17bd83d4d95116', 'request_id': 'req-4cdf2e95-f2ec-4286-86ce-7ef198468931'} {{(pid=81768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 16:12:44.443617 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-4cdf2e95-f2ec-4286-86ce-7ef198468931 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0e5c1343cecc4c6d9d17bd83d4d95116', 'request_id': 'req-4cdf2e95-f2ec-4286-86ce-7ef198468931'} {{(pid=81768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 16:12:44.467227 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-4cdf2e95-f2ec-4286-86ce-7ef198468931 admin admin] The token cache is being invalidate because project 0e5c1343cecc4c6d9d17bd83d4d95116 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81768) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 22 16:12:44.467771 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-4cdf2e95-f2ec-4286-86ce-7ef198468931 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 16:12:44.468788 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-4cdf2e95-f2ec-4286-86ce-7ef198468931 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 16:12:44.472975 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 1213/2391] 10.222.0.15 () {58 vars in 1233 bytes} [Mon Aug 22 16:12:44 2022] DELETE /identity/v3/projects/0e5c1343cecc4c6d9d17bd83d4d95116 => generated 0 bytes in 322 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 16:12:44.478665 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fe5215c-ace7-45c0-8ae7-9b99ad036066 service neutron] Authenticating user token {{(pid=81767) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 16:12:44.489107 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fe5215c-ace7-45c0-8ae7-9b99ad036066 service neutron] Validating token access rules against request {{(pid=81767) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 16:12:44.548253 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-311f69e5-0337-4d7f-8f32-b03e6fcb96bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a98b1e4431940ab8d1fc95df2858a6c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81767) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 16:12:44.549142 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-311f69e5-0337-4d7f-8f32-b03e6fcb96bf admin admin] REQUEST_METHOD: `DELETE` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:12:44.549424 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-311f69e5-0337-4d7f-8f32-b03e6fcb96bf admin admin] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:12:44.549729 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-311f69e5-0337-4d7f-8f32-b03e6fcb96bf admin admin] PATH_INFO: `/v3/users/fcdeaaa4687340deafb946b7d3f7fac1` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:12:44.550925 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-311f69e5-0337-4d7f-8f32-b03e6fcb96bf admin admin] RBAC: Authorizing `identity:delete_user(user_id=fcdeaaa4687340deafb946b7d3f7fac1)` {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 16:12:44.553083 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-311f69e5-0337-4d7f-8f32-b03e6fcb96bf admin admin] RBAC: Authorization granted {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 16:12:44.918748 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.notifications [None req-311f69e5-0337-4d7f-8f32-b03e6fcb96bf admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fcdeaaa4687340deafb946b7d3f7fac1'} {{(pid=81767) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 16:12:44.919571 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.notifications [None req-311f69e5-0337-4d7f-8f32-b03e6fcb96bf admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fcdeaaa4687340deafb946b7d3f7fac1'} {{(pid=81767) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 16:12:45.018685 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.notifications [None req-311f69e5-0337-4d7f-8f32-b03e6fcb96bf admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fcdeaaa4687340deafb946b7d3f7fac1'} {{(pid=81767) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 16:12:45.026076 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.notifications [None req-311f69e5-0337-4d7f-8f32-b03e6fcb96bf admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fcdeaaa4687340deafb946b7d3f7fac1'} {{(pid=81767) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 16:12:45.039902 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.notifications [None req-311f69e5-0337-4d7f-8f32-b03e6fcb96bf admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fcdeaaa4687340deafb946b7d3f7fac1'} {{(pid=81767) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 16:12:45.048799 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 1179/2392] 10.222.0.15 () {58 vars in 1224 bytes} [Mon Aug 22 16:12:44 2022] DELETE /identity/v3/users/fcdeaaa4687340deafb946b7d3f7fac1 => generated 0 bytes in 568 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 16:12:45.523160 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cdf2e95-f2ec-4286-86ce-7ef198468931 admin admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 16:12:45.524919 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-4cdf2e95-f2ec-4286-86ce-7ef198468931 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 16:12:45.546008 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cdf2e95-f2ec-4286-86ce-7ef198468931 admin admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 16:12:45.624309 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09930c60-8321-4eee-97dc-fabd45290abf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a98b1e4431940ab8d1fc95df2858a6c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 16:12:45.627973 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09930c60-8321-4eee-97dc-fabd45290abf admin admin] REQUEST_METHOD: `DELETE` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:12:45.627973 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09930c60-8321-4eee-97dc-fabd45290abf admin admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:12:45.627973 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09930c60-8321-4eee-97dc-fabd45290abf admin admin] PATH_INFO: `/v3/projects/86b4b257251d4bf0ad86d874b55c8515` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:12:45.627973 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09930c60-8321-4eee-97dc-fabd45290abf admin admin] RBAC: Authorizing `identity:delete_project(project_id=86b4b257251d4bf0ad86d874b55c8515)` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 16:12:45.629559 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09930c60-8321-4eee-97dc-fabd45290abf admin admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 16:12:45.820353 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-09930c60-8321-4eee-97dc-fabd45290abf admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '86b4b257251d4bf0ad86d874b55c8515', 'request_id': 'req-09930c60-8321-4eee-97dc-fabd45290abf'} {{(pid=81768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 16:12:45.826858 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 16:12:45.829898 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-09930c60-8321-4eee-97dc-fabd45290abf admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '86b4b257251d4bf0ad86d874b55c8515', 'request_id': 'req-09930c60-8321-4eee-97dc-fabd45290abf'} {{(pid=81768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 16:12:45.950206 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-09930c60-8321-4eee-97dc-fabd45290abf admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '86b4b257251d4bf0ad86d874b55c8515', 'request_id': 'req-09930c60-8321-4eee-97dc-fabd45290abf'} {{(pid=81768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 16:12:45.968419 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-09930c60-8321-4eee-97dc-fabd45290abf admin admin] The token cache is being invalidate because project 86b4b257251d4bf0ad86d874b55c8515 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81768) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 22 16:12:45.968982 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-09930c60-8321-4eee-97dc-fabd45290abf admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 16:12:45.969863 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-09930c60-8321-4eee-97dc-fabd45290abf admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 16:12:45.972288 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 1214/2393] 10.222.0.15 () {58 vars in 1233 bytes} [Mon Aug 22 16:12:45 2022] DELETE /identity/v3/projects/86b4b257251d4bf0ad86d874b55c8515 => generated 0 bytes in 450 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 16:12:45.980021 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a15fcff5-96de-4963-b1c9-16153efd0308 None None] REQUEST_METHOD: `POST` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:12:45.981007 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a15fcff5-96de-4963-b1c9-16153efd0308 None None] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:12:45.981462 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a15fcff5-96de-4963-b1c9-16153efd0308 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:12:46.005529 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.auth.core [None req-a15fcff5-96de-4963-b1c9-16153efd0308 None None] MFA Rules not processed for user `6b2199ae0fb0480aa917ada391f5ec54`. Rule list: `[]` (Enabled: `True`). {{(pid=81767) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 16:12:46.007176 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.fernet_utils [None req-a15fcff5-96de-4963-b1c9-16153efd0308 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81767) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 16:12:46.075072 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG dbcounter [-] [81767] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81767) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 16:12:46.105905 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 1180/2394] 10.222.0.15 () {58 vars in 958 bytes} [Mon Aug 22 16:12:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 16:12:46.128434 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-260b3104-dcb4-42bf-8acc-413057929157 None None] REQUEST_METHOD: `POST` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:12:46.128767 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-260b3104-dcb4-42bf-8acc-413057929157 None None] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:12:46.129006 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-260b3104-dcb4-42bf-8acc-413057929157 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:12:46.150017 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.auth.core [None req-260b3104-dcb4-42bf-8acc-413057929157 None None] MFA Rules not processed for user `6b2199ae0fb0480aa917ada391f5ec54`. Rule list: `[]` (Enabled: `True`). {{(pid=81768) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 16:12:46.151410 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-260b3104-dcb4-42bf-8acc-413057929157 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 16:12:46.160026 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 1215/2395] 10.222.0.15 () {58 vars in 958 bytes} [Mon Aug 22 16:12:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 16:12:46.165688 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a15fcff5-96de-4963-b1c9-16153efd0308 None None] Authenticating user token {{(pid=81767) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 16:12:46.176773 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a15fcff5-96de-4963-b1c9-16153efd0308 None None] Validating token access rules against request {{(pid=81767) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 16:12:46.181874 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3faf88c-f99a-443a-b8a7-c55ed1de0dee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a98b1e4431940ab8d1fc95df2858a6c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81767) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 16:12:46.182618 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3faf88c-f99a-443a-b8a7-c55ed1de0dee admin admin] REQUEST_METHOD: `GET` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:12:46.182978 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3faf88c-f99a-443a-b8a7-c55ed1de0dee admin admin] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:12:46.183659 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3faf88c-f99a-443a-b8a7-c55ed1de0dee admin admin] PATH_INFO: `/v3/domains` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:12:46.184276 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3faf88c-f99a-443a-b8a7-c55ed1de0dee admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81767) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 16:12:46.184686 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3faf88c-f99a-443a-b8a7-c55ed1de0dee admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 16:12:46.186361 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3faf88c-f99a-443a-b8a7-c55ed1de0dee admin admin] RBAC: Authorization granted {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 16:12:46.200327 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 1181/2396] 10.222.0.15 () {58 vars in 1153 bytes} [Mon Aug 22 16:12:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 16:12:46.205599 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-260b3104-dcb4-42bf-8acc-413057929157 None None] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 16:12:46.215671 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-260b3104-dcb4-42bf-8acc-413057929157 None None] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 16:12:46.229426 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e78e4d9-dcef-40ce-a7b6-7e0a7079c6d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a98b1e4431940ab8d1fc95df2858a6c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 16:12:46.233798 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e78e4d9-dcef-40ce-a7b6-7e0a7079c6d1 admin admin] REQUEST_METHOD: `POST` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:12:46.234433 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e78e4d9-dcef-40ce-a7b6-7e0a7079c6d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:12:46.235880 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e78e4d9-dcef-40ce-a7b6-7e0a7079c6d1 admin admin] PATH_INFO: `/v3/projects` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:12:46.236761 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e78e4d9-dcef-40ce-a7b6-7e0a7079c6d1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 16:12:46.239104 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e78e4d9-dcef-40ce-a7b6-7e0a7079c6d1 admin admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 16:12:46.405070 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 1216/2397] 10.222.0.15 () {60 vars in 1153 bytes} [Mon Aug 22 16:12:46 2022] POST /identity/v3/projects => generated 387 bytes in 200 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 22 16:12:46.410266 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3faf88c-f99a-443a-b8a7-c55ed1de0dee admin admin] Authenticating user token {{(pid=81767) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 16:12:46.422686 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3faf88c-f99a-443a-b8a7-c55ed1de0dee admin admin] Validating token access rules against request {{(pid=81767) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 16:12:46.514066 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eb91399-f1ca-4031-9586-be8a547b7875 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a98b1e4431940ab8d1fc95df2858a6c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81767) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 16:12:46.514519 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eb91399-f1ca-4031-9586-be8a547b7875 admin admin] REQUEST_METHOD: `POST` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:12:46.514790 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eb91399-f1ca-4031-9586-be8a547b7875 admin admin] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:12:46.519795 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eb91399-f1ca-4031-9586-be8a547b7875 admin admin] PATH_INFO: `/v3/users` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:12:46.521021 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eb91399-f1ca-4031-9586-be8a547b7875 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 16:12:46.523264 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eb91399-f1ca-4031-9586-be8a547b7875 admin admin] RBAC: Authorization granted {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 16:12:46.648403 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 1182/2398] 10.222.0.15 () {60 vars in 1144 bytes} [Mon Aug 22 16:12:46 2022] POST /identity/v3/users => generated 294 bytes in 238 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 22 16:12:46.652436 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e78e4d9-dcef-40ce-a7b6-7e0a7079c6d1 admin admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 16:12:46.663861 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e78e4d9-dcef-40ce-a7b6-7e0a7079c6d1 admin admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 16:12:46.668590 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31652ad1-eb7a-4755-933f-bfde748eaab0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a98b1e4431940ab8d1fc95df2858a6c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 16:12:46.669202 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31652ad1-eb7a-4755-933f-bfde748eaab0 admin admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:12:46.669481 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31652ad1-eb7a-4755-933f-bfde748eaab0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:12:46.669805 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31652ad1-eb7a-4755-933f-bfde748eaab0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:12:46.670530 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31652ad1-eb7a-4755-933f-bfde748eaab0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 16:12:46.672411 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31652ad1-eb7a-4755-933f-bfde748eaab0 admin admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 16:12:46.684067 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 1217/2399] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Aug 22 16:12:46 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 22 16:12:46.700508 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eb91399-f1ca-4031-9586-be8a547b7875 admin admin] Authenticating user token {{(pid=81767) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 16:12:46.712395 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eb91399-f1ca-4031-9586-be8a547b7875 admin admin] Validating token access rules against request {{(pid=81767) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 16:12:46.717873 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-677fd106-e50c-4b9f-92cd-53ac8685c3f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a98b1e4431940ab8d1fc95df2858a6c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81767) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 16:12:46.719629 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-677fd106-e50c-4b9f-92cd-53ac8685c3f3 admin admin] REQUEST_METHOD: `GET` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:12:46.719629 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-677fd106-e50c-4b9f-92cd-53ac8685c3f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:12:46.719629 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-677fd106-e50c-4b9f-92cd-53ac8685c3f3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:12:46.720535 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-677fd106-e50c-4b9f-92cd-53ac8685c3f3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 16:12:46.722121 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-677fd106-e50c-4b9f-92cd-53ac8685c3f3 admin admin] RBAC: Authorization granted {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 16:12:46.734279 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 1183/2400] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Aug 22 16:12:46 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 22 16:12:46.738534 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31652ad1-eb7a-4755-933f-bfde748eaab0 admin admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 16:12:46.749940 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31652ad1-eb7a-4755-933f-bfde748eaab0 admin admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 16:12:46.754797 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9a7e8e9-0d5d-4a00-8400-3bb524570e5c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a98b1e4431940ab8d1fc95df2858a6c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 16:12:46.755519 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9a7e8e9-0d5d-4a00-8400-3bb524570e5c admin admin] REQUEST_METHOD: `PUT` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:12:46.755804 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9a7e8e9-0d5d-4a00-8400-3bb524570e5c admin admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:12:46.756096 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9a7e8e9-0d5d-4a00-8400-3bb524570e5c admin admin] PATH_INFO: `/v3/projects/3d0304f96b814c709e4c14dfc302bd95/users/c266cccf6d5f4fe8b2bea15b55b26b7a/roles/e6fc1ee3fb974125ba9b4fe55b12b45e` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:12:46.774307 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9a7e8e9-0d5d-4a00-8400-3bb524570e5c admin admin] RBAC: Authorizing `identity:create_grant(project_id=3d0304f96b814c709e4c14dfc302bd95, user_id=c266cccf6d5f4fe8b2bea15b55b26b7a, role_id=e6fc1ee3fb974125ba9b4fe55b12b45e)` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 16:12:46.777583 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9a7e8e9-0d5d-4a00-8400-3bb524570e5c admin admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 16:12:46.941900 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 1218/2401] 10.222.0.15 () {60 vars in 1483 bytes} [Mon Aug 22 16:12:46 2022] PUT /identity/v3/projects/3d0304f96b814c709e4c14dfc302bd95/users/c266cccf6d5f4fe8b2bea15b55b26b7a/roles/e6fc1ee3fb974125ba9b4fe55b12b45e => generated 0 bytes in 201 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 16:12:46.952208 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28b481d6-d59f-4f05-92f5-687cf014b835 None None] REQUEST_METHOD: `POST` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:12:46.952577 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28b481d6-d59f-4f05-92f5-687cf014b835 None None] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:12:46.952847 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28b481d6-d59f-4f05-92f5-687cf014b835 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:12:46.985283 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.auth.core [None req-28b481d6-d59f-4f05-92f5-687cf014b835 None None] MFA Rules not processed for user `c266cccf6d5f4fe8b2bea15b55b26b7a`. Rule list: `[]` (Enabled: `True`). {{(pid=81767) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 16:12:46.988103 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.fernet_utils [None req-28b481d6-d59f-4f05-92f5-687cf014b835 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81767) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 16:12:47.066733 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 1184/2402] 10.222.0.15 () {58 vars in 958 bytes} [Mon Aug 22 16:12:46 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 16:12:47.069139 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9a7e8e9-0d5d-4a00-8400-3bb524570e5c admin admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 16:12:47.143526 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9a7e8e9-0d5d-4a00-8400-3bb524570e5c admin admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 16:12:47.148264 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b65a7bc-84ce-413b-a275-326ec30fffac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a98b1e4431940ab8d1fc95df2858a6c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 16:12:47.148873 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b65a7bc-84ce-413b-a275-326ec30fffac admin admin] REQUEST_METHOD: `POST` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:12:47.149804 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b65a7bc-84ce-413b-a275-326ec30fffac admin admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:12:47.150126 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b65a7bc-84ce-413b-a275-326ec30fffac admin admin] PATH_INFO: `/v3/projects` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:12:47.150747 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b65a7bc-84ce-413b-a275-326ec30fffac admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 16:12:47.152717 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b65a7bc-84ce-413b-a275-326ec30fffac admin admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 16:12:47.249372 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 1219/2403] 10.222.0.15 () {60 vars in 1153 bytes} [Mon Aug 22 16:12:47 2022] POST /identity/v3/projects => generated 387 bytes in 181 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 22 16:12:47.258253 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28b481d6-d59f-4f05-92f5-687cf014b835 None None] Authenticating user token {{(pid=81767) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 16:12:47.269154 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28b481d6-d59f-4f05-92f5-687cf014b835 None None] Validating token access rules against request {{(pid=81767) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 16:12:47.330218 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1caccd4-f992-4e54-9aaf-2bd4bab94e32 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a98b1e4431940ab8d1fc95df2858a6c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81767) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 16:12:47.330790 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1caccd4-f992-4e54-9aaf-2bd4bab94e32 admin admin] REQUEST_METHOD: `POST` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:12:47.331071 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1caccd4-f992-4e54-9aaf-2bd4bab94e32 admin admin] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:12:47.331378 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1caccd4-f992-4e54-9aaf-2bd4bab94e32 admin admin] PATH_INFO: `/v3/users` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:12:47.332050 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1caccd4-f992-4e54-9aaf-2bd4bab94e32 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 16:12:47.334018 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1caccd4-f992-4e54-9aaf-2bd4bab94e32 admin admin] RBAC: Authorization granted {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 16:12:47.463137 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 1185/2404] 10.222.0.15 () {60 vars in 1144 bytes} [Mon Aug 22 16:12:47 2022] POST /identity/v3/users => generated 300 bytes in 202 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 22 16:12:47.464377 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b65a7bc-84ce-413b-a275-326ec30fffac admin admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 16:12:47.476066 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b65a7bc-84ce-413b-a275-326ec30fffac admin admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 16:12:47.480820 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76706ee0-b368-4f2b-8931-ff7d21c1b360 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a98b1e4431940ab8d1fc95df2858a6c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 16:12:47.481614 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76706ee0-b368-4f2b-8931-ff7d21c1b360 admin admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:12:47.481890 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76706ee0-b368-4f2b-8931-ff7d21c1b360 admin admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:12:47.482385 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76706ee0-b368-4f2b-8931-ff7d21c1b360 admin admin] PATH_INFO: `/v3/roles` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:12:47.483196 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76706ee0-b368-4f2b-8931-ff7d21c1b360 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 16:12:47.485036 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76706ee0-b368-4f2b-8931-ff7d21c1b360 admin admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 16:12:47.498254 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 1220/2405] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Aug 22 16:12:47 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 22 16:12:47.504091 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1caccd4-f992-4e54-9aaf-2bd4bab94e32 admin admin] Authenticating user token {{(pid=81767) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 16:12:47.517920 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1caccd4-f992-4e54-9aaf-2bd4bab94e32 admin admin] Validating token access rules against request {{(pid=81767) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 16:12:47.523263 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-653053ab-a434-499b-8d1e-5315a504d443 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a98b1e4431940ab8d1fc95df2858a6c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81767) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 16:12:47.524124 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-653053ab-a434-499b-8d1e-5315a504d443 admin admin] REQUEST_METHOD: `PUT` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:12:47.524407 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-653053ab-a434-499b-8d1e-5315a504d443 admin admin] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:12:47.524677 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-653053ab-a434-499b-8d1e-5315a504d443 admin admin] PATH_INFO: `/v3/projects/3d454535ca3143088b019a5785cd1a7f/users/36ac64fac03147d0bc8d73207aed614c/roles/9d9848cfd22e4b089db443c350e56670` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:12:47.546643 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-653053ab-a434-499b-8d1e-5315a504d443 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3d454535ca3143088b019a5785cd1a7f, user_id=36ac64fac03147d0bc8d73207aed614c, role_id=9d9848cfd22e4b089db443c350e56670)` {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 16:12:47.549520 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-653053ab-a434-499b-8d1e-5315a504d443 admin admin] RBAC: Authorization granted {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 16:12:47.651039 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 1186/2406] 10.222.0.15 () {60 vars in 1483 bytes} [Mon Aug 22 16:12:47 2022] PUT /identity/v3/projects/3d454535ca3143088b019a5785cd1a7f/users/36ac64fac03147d0bc8d73207aed614c/roles/9d9848cfd22e4b089db443c350e56670 => generated 0 bytes in 147 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 16:12:47.655411 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76706ee0-b368-4f2b-8931-ff7d21c1b360 admin admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 16:12:47.666704 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76706ee0-b368-4f2b-8931-ff7d21c1b360 admin admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 16:12:47.730639 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b8867c2-f637-4132-8791-f75a11bd9318 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a98b1e4431940ab8d1fc95df2858a6c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 16:12:47.731321 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b8867c2-f637-4132-8791-f75a11bd9318 admin admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:12:47.731664 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b8867c2-f637-4132-8791-f75a11bd9318 admin admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:12:47.731955 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b8867c2-f637-4132-8791-f75a11bd9318 admin admin] PATH_INFO: `/v3/roles` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:12:47.732597 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b8867c2-f637-4132-8791-f75a11bd9318 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 16:12:47.734254 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b8867c2-f637-4132-8791-f75a11bd9318 admin admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 16:12:47.744387 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 1221/2407] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Aug 22 16:12:47 2022] GET /identity/v3/roles => generated 1420 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 22 16:12:47.748687 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-653053ab-a434-499b-8d1e-5315a504d443 admin admin] Authenticating user token {{(pid=81767) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 16:12:47.759296 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-653053ab-a434-499b-8d1e-5315a504d443 admin admin] Validating token access rules against request {{(pid=81767) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 16:12:47.764042 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f99d18b-d6f9-4263-aa11-37d89f972a8e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a98b1e4431940ab8d1fc95df2858a6c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81767) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 16:12:47.764743 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f99d18b-d6f9-4263-aa11-37d89f972a8e admin admin] REQUEST_METHOD: `PUT` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:12:47.765029 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f99d18b-d6f9-4263-aa11-37d89f972a8e admin admin] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:12:47.765306 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f99d18b-d6f9-4263-aa11-37d89f972a8e admin admin] PATH_INFO: `/v3/projects/3d454535ca3143088b019a5785cd1a7f/users/36ac64fac03147d0bc8d73207aed614c/roles/9d9848cfd22e4b089db443c350e56670` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:12:47.767775 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f99d18b-d6f9-4263-aa11-37d89f972a8e admin admin] RBAC: Authorizing `identity:create_grant(project_id=3d454535ca3143088b019a5785cd1a7f, user_id=36ac64fac03147d0bc8d73207aed614c, role_id=9d9848cfd22e4b089db443c350e56670)` {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 16:12:47.770857 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f99d18b-d6f9-4263-aa11-37d89f972a8e admin admin] RBAC: Authorization granted {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 16:12:47.783148 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 1187/2408] 10.222.0.15 () {60 vars in 1483 bytes} [Mon Aug 22 16:12:47 2022] PUT /identity/v3/projects/3d454535ca3143088b019a5785cd1a7f/users/36ac64fac03147d0bc8d73207aed614c/roles/9d9848cfd22e4b089db443c350e56670 => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 16:12:47.794084 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13c65369-727e-4349-9d20-323c0ff41c9b None None] REQUEST_METHOD: `POST` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:12:47.794369 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13c65369-727e-4349-9d20-323c0ff41c9b None None] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:12:47.794665 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13c65369-727e-4349-9d20-323c0ff41c9b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:12:47.825371 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.auth.core [None req-13c65369-727e-4349-9d20-323c0ff41c9b None None] MFA Rules not processed for user `36ac64fac03147d0bc8d73207aed614c`. Rule list: `[]` (Enabled: `True`). {{(pid=81768) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 16:12:47.826594 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-13c65369-727e-4349-9d20-323c0ff41c9b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 16:12:47.901060 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=95,keystone:INSERT=4,keystone:DELETE=1 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 16:12:47.909442 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 1222/2409] 10.222.0.15 () {58 vars in 958 bytes} [Mon Aug 22 16:12:47 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 16:12:47.920865 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f99d18b-d6f9-4263-aa11-37d89f972a8e admin admin] Authenticating user token {{(pid=81767) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 16:12:47.922630 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.fernet_utils [None req-7f99d18b-d6f9-4263-aa11-37d89f972a8e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81767) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 16:12:47.986600 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f99d18b-d6f9-4263-aa11-37d89f972a8e admin admin] Validating token access rules against request {{(pid=81767) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 16:12:47.992164 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ac08a91-6c1f-402b-b27f-894856dc99b6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79706d480f864998b9f8594be407ceab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4ee0413f9ef4d4fb1f95a660babe090', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81767) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 16:12:47.996003 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac08a91-6c1f-402b-b27f-894856dc99b6 service cinder] REQUEST_METHOD: `GET` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:12:47.996003 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac08a91-6c1f-402b-b27f-894856dc99b6 service cinder] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:12:47.996003 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac08a91-6c1f-402b-b27f-894856dc99b6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:12:48.004331 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ac08a91-6c1f-402b-b27f-894856dc99b6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 16:12:48.006097 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ac08a91-6c1f-402b-b27f-894856dc99b6 service cinder] RBAC: Authorization granted {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 16:12:48.015909 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 1188/2410] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Aug 22 16:12:47 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 16:12:48.169433 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG dbcounter [-] [81767] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81767) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 16:12:49.295134 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13c65369-727e-4349-9d20-323c0ff41c9b None None] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 16:12:49.444613 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13c65369-727e-4349-9d20-323c0ff41c9b None None] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 16:12:49.457772 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3043c196-0fa3-4787-ae37-a6e03487ff11 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a98b1e4431940ab8d1fc95df2858a6c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 16:12:49.458423 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3043c196-0fa3-4787-ae37-a6e03487ff11 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:12:49.459912 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3043c196-0fa3-4787-ae37-a6e03487ff11 admin admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:12:49.459912 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3043c196-0fa3-4787-ae37-a6e03487ff11 admin admin] PATH_INFO: `/v3/users/c266cccf6d5f4fe8b2bea15b55b26b7a` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:12:49.460597 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3043c196-0fa3-4787-ae37-a6e03487ff11 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c266cccf6d5f4fe8b2bea15b55b26b7a)` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 16:12:49.462669 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3043c196-0fa3-4787-ae37-a6e03487ff11 admin admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 16:12:49.715157 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-3043c196-0fa3-4787-ae37-a6e03487ff11 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c266cccf6d5f4fe8b2bea15b55b26b7a'} {{(pid=81768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 16:12:49.715778 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-3043c196-0fa3-4787-ae37-a6e03487ff11 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c266cccf6d5f4fe8b2bea15b55b26b7a'} {{(pid=81768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 16:12:49.716365 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-3043c196-0fa3-4787-ae37-a6e03487ff11 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c266cccf6d5f4fe8b2bea15b55b26b7a'} {{(pid=81768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 16:12:49.724064 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-3043c196-0fa3-4787-ae37-a6e03487ff11 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c266cccf6d5f4fe8b2bea15b55b26b7a'} {{(pid=81768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 16:12:49.809361 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-3043c196-0fa3-4787-ae37-a6e03487ff11 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c266cccf6d5f4fe8b2bea15b55b26b7a'} {{(pid=81768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 16:12:49.822162 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 16:12:49.829459 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 1223/2411] 10.222.0.15 () {58 vars in 1224 bytes} [Mon Aug 22 16:12:49 2022] DELETE /identity/v3/users/c266cccf6d5f4fe8b2bea15b55b26b7a => generated 0 bytes in 535 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 16:12:49.834375 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50b220a3-cdcf-446c-b397-b92c5e4be814 None None] REQUEST_METHOD: `POST` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:12:49.834895 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50b220a3-cdcf-446c-b397-b92c5e4be814 None None] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:12:49.835021 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50b220a3-cdcf-446c-b397-b92c5e4be814 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:12:49.855644 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.auth.core [None req-50b220a3-cdcf-446c-b397-b92c5e4be814 None None] MFA Rules not processed for user `6b2199ae0fb0480aa917ada391f5ec54`. Rule list: `[]` (Enabled: `True`). {{(pid=81767) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 16:12:49.856860 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.fernet_utils [None req-50b220a3-cdcf-446c-b397-b92c5e4be814 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81767) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 16:12:49.916503 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 1189/2412] 10.222.0.15 () {58 vars in 958 bytes} [Mon Aug 22 16:12:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 16:12:49.926280 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3043c196-0fa3-4787-ae37-a6e03487ff11 admin admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 16:12:49.927784 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-3043c196-0fa3-4787-ae37-a6e03487ff11 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 16:12:49.997979 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3043c196-0fa3-4787-ae37-a6e03487ff11 admin admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 16:12:50.002246 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5564e497-326a-4b52-9f2c-5cd8993d6483 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '234687adad594f1aa45775a565ffbdc7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4ee0413f9ef4d4fb1f95a660babe090', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 16:12:50.003623 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5564e497-326a-4b52-9f2c-5cd8993d6483 service neutron] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:12:50.003623 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5564e497-326a-4b52-9f2c-5cd8993d6483 service neutron] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:12:50.003623 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5564e497-326a-4b52-9f2c-5cd8993d6483 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:12:50.012988 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5564e497-326a-4b52-9f2c-5cd8993d6483 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 16:12:50.014972 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5564e497-326a-4b52-9f2c-5cd8993d6483 service neutron] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 16:12:50.026635 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 1224/2413] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Aug 22 16:12:49 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 16:12:50.447874 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50b220a3-cdcf-446c-b397-b92c5e4be814 None None] Authenticating user token {{(pid=81767) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 16:12:50.449519 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.fernet_utils [None req-50b220a3-cdcf-446c-b397-b92c5e4be814 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81767) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 16:12:50.463653 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50b220a3-cdcf-446c-b397-b92c5e4be814 None None] Validating token access rules against request {{(pid=81767) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 16:12:50.468813 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-313c1a3f-6ef1-4fc5-afcc-f19fbb05f477 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a98b1e4431940ab8d1fc95df2858a6c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81767) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 16:12:50.469614 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-313c1a3f-6ef1-4fc5-afcc-f19fbb05f477 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:12:50.470125 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-313c1a3f-6ef1-4fc5-afcc-f19fbb05f477 admin admin] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:12:50.470588 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-313c1a3f-6ef1-4fc5-afcc-f19fbb05f477 admin admin] PATH_INFO: `/v3/projects/3d0304f96b814c709e4c14dfc302bd95` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:12:50.471850 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-313c1a3f-6ef1-4fc5-afcc-f19fbb05f477 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3d0304f96b814c709e4c14dfc302bd95)` {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 16:12:50.473741 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-313c1a3f-6ef1-4fc5-afcc-f19fbb05f477 admin admin] RBAC: Authorization granted {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 16:12:50.603216 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG dbcounter [-] [81767] Writing DB stats keystone:SELECT=92,keystone:INSERT=5,keystone:DELETE=3 {{(pid=81767) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 16:12:50.612284 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.notifications [None req-313c1a3f-6ef1-4fc5-afcc-f19fbb05f477 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3d0304f96b814c709e4c14dfc302bd95', 'request_id': 'req-313c1a3f-6ef1-4fc5-afcc-f19fbb05f477'} {{(pid=81767) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 16:12:50.618020 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.notifications [None req-313c1a3f-6ef1-4fc5-afcc-f19fbb05f477 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3d0304f96b814c709e4c14dfc302bd95', 'request_id': 'req-313c1a3f-6ef1-4fc5-afcc-f19fbb05f477'} {{(pid=81767) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 16:12:50.705916 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.notifications [None req-313c1a3f-6ef1-4fc5-afcc-f19fbb05f477 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3d0304f96b814c709e4c14dfc302bd95', 'request_id': 'req-313c1a3f-6ef1-4fc5-afcc-f19fbb05f477'} {{(pid=81767) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 16:12:50.718026 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.notifications [None req-313c1a3f-6ef1-4fc5-afcc-f19fbb05f477 admin admin] The token cache is being invalidate because project 3d0304f96b814c709e4c14dfc302bd95 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81767) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 22 16:12:50.718330 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.notifications [None req-313c1a3f-6ef1-4fc5-afcc-f19fbb05f477 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81767) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 16:12:50.719109 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.notifications [None req-313c1a3f-6ef1-4fc5-afcc-f19fbb05f477 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81767) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 16:12:50.721134 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 1190/2414] 10.222.0.15 () {58 vars in 1233 bytes} [Mon Aug 22 16:12:50 2022] DELETE /identity/v3/projects/3d0304f96b814c709e4c14dfc302bd95 => generated 0 bytes in 274 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 16:12:50.736677 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5564e497-326a-4b52-9f2c-5cd8993d6483 service neutron] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 16:12:50.746767 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5564e497-326a-4b52-9f2c-5cd8993d6483 service neutron] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 16:12:50.803838 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfb977ca-190b-4fe2-8bc3-e79a6c21ae93 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a98b1e4431940ab8d1fc95df2858a6c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 16:12:50.804462 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfb977ca-190b-4fe2-8bc3-e79a6c21ae93 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:12:50.804737 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfb977ca-190b-4fe2-8bc3-e79a6c21ae93 admin admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:12:50.805082 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfb977ca-190b-4fe2-8bc3-e79a6c21ae93 admin admin] PATH_INFO: `/v3/users/36ac64fac03147d0bc8d73207aed614c` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:12:50.806128 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfb977ca-190b-4fe2-8bc3-e79a6c21ae93 admin admin] RBAC: Authorizing `identity:delete_user(user_id=36ac64fac03147d0bc8d73207aed614c)` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 16:12:50.808077 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfb977ca-190b-4fe2-8bc3-e79a6c21ae93 admin admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 16:12:51.045166 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-cfb977ca-190b-4fe2-8bc3-e79a6c21ae93 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '36ac64fac03147d0bc8d73207aed614c'} {{(pid=81768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 16:12:51.045821 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-cfb977ca-190b-4fe2-8bc3-e79a6c21ae93 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '36ac64fac03147d0bc8d73207aed614c'} {{(pid=81768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 16:12:51.046438 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-cfb977ca-190b-4fe2-8bc3-e79a6c21ae93 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '36ac64fac03147d0bc8d73207aed614c'} {{(pid=81768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 16:12:51.054117 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-cfb977ca-190b-4fe2-8bc3-e79a6c21ae93 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '36ac64fac03147d0bc8d73207aed614c'} {{(pid=81768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 16:12:51.114785 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-cfb977ca-190b-4fe2-8bc3-e79a6c21ae93 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '36ac64fac03147d0bc8d73207aed614c'} {{(pid=81768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 16:12:51.134065 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 1225/2415] 10.222.0.15 () {58 vars in 1224 bytes} [Mon Aug 22 16:12:50 2022] DELETE /identity/v3/users/36ac64fac03147d0bc8d73207aed614c => generated 0 bytes in 404 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 16:12:51.580903 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-313c1a3f-6ef1-4fc5-afcc-f19fbb05f477 admin admin] Authenticating user token {{(pid=81767) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 16:12:51.582373 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.fernet_utils [None req-313c1a3f-6ef1-4fc5-afcc-f19fbb05f477 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81767) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 16:12:51.597473 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-313c1a3f-6ef1-4fc5-afcc-f19fbb05f477 admin admin] Validating token access rules against request {{(pid=81767) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 16:12:51.659259 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50ce984f-bb14-4348-9ab8-cb8a837d2a76 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a98b1e4431940ab8d1fc95df2858a6c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81767) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 16:12:51.659913 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50ce984f-bb14-4348-9ab8-cb8a837d2a76 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:12:51.660192 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50ce984f-bb14-4348-9ab8-cb8a837d2a76 admin admin] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:12:51.660463 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50ce984f-bb14-4348-9ab8-cb8a837d2a76 admin admin] PATH_INFO: `/v3/projects/3d454535ca3143088b019a5785cd1a7f` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:12:51.661580 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50ce984f-bb14-4348-9ab8-cb8a837d2a76 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3d454535ca3143088b019a5785cd1a7f)` {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 16:12:51.663495 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50ce984f-bb14-4348-9ab8-cb8a837d2a76 admin admin] RBAC: Authorization granted {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 16:12:51.790963 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.notifications [None req-50ce984f-bb14-4348-9ab8-cb8a837d2a76 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3d454535ca3143088b019a5785cd1a7f', 'request_id': 'req-50ce984f-bb14-4348-9ab8-cb8a837d2a76'} {{(pid=81767) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 16:12:51.793245 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.notifications [None req-50ce984f-bb14-4348-9ab8-cb8a837d2a76 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3d454535ca3143088b019a5785cd1a7f', 'request_id': 'req-50ce984f-bb14-4348-9ab8-cb8a837d2a76'} {{(pid=81767) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 16:12:51.917773 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.notifications [None req-50ce984f-bb14-4348-9ab8-cb8a837d2a76 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3d454535ca3143088b019a5785cd1a7f', 'request_id': 'req-50ce984f-bb14-4348-9ab8-cb8a837d2a76'} {{(pid=81767) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 16:12:51.929763 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.notifications [None req-50ce984f-bb14-4348-9ab8-cb8a837d2a76 admin admin] The token cache is being invalidate because project 3d454535ca3143088b019a5785cd1a7f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81767) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 22 16:12:51.933483 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.notifications [None req-50ce984f-bb14-4348-9ab8-cb8a837d2a76 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81767) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 16:12:51.933483 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.notifications [None req-50ce984f-bb14-4348-9ab8-cb8a837d2a76 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81767) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 16:12:51.933483 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 1191/2416] 10.222.0.15 () {58 vars in 1233 bytes} [Mon Aug 22 16:12:51 2022] DELETE /identity/v3/projects/3d454535ca3143088b019a5785cd1a7f => generated 0 bytes in 352 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 16:12:51.936990 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a14363-c3dc-4a54-8830-89ad44830915 None None] REQUEST_METHOD: `POST` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:12:51.937447 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a14363-c3dc-4a54-8830-89ad44830915 None None] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:12:51.937539 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a14363-c3dc-4a54-8830-89ad44830915 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:12:51.958537 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.auth.core [None req-d0a14363-c3dc-4a54-8830-89ad44830915 None None] MFA Rules not processed for user `6b2199ae0fb0480aa917ada391f5ec54`. Rule list: `[]` (Enabled: `True`). {{(pid=81768) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 16:12:51.960026 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-d0a14363-c3dc-4a54-8830-89ad44830915 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 16:12:51.972602 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 16:12:52.021453 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 1226/2417] 10.222.0.15 () {58 vars in 958 bytes} [Mon Aug 22 16:12:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 16:12:52.029361 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9d522a3-3ea0-4603-af44-e7e5af99312a None None] REQUEST_METHOD: `POST` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:12:52.030151 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9d522a3-3ea0-4603-af44-e7e5af99312a None None] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:12:52.030151 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9d522a3-3ea0-4603-af44-e7e5af99312a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:12:52.051208 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.auth.core [None req-d9d522a3-3ea0-4603-af44-e7e5af99312a None None] MFA Rules not processed for user `6b2199ae0fb0480aa917ada391f5ec54`. Rule list: `[]` (Enabled: `True`). {{(pid=81767) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 16:12:52.052591 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.fernet_utils [None req-d9d522a3-3ea0-4603-af44-e7e5af99312a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81767) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 16:12:52.060690 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 1192/2418] 10.222.0.15 () {58 vars in 958 bytes} [Mon Aug 22 16:12:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 16:12:52.065503 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0a14363-c3dc-4a54-8830-89ad44830915 None None] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 16:12:52.075221 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0a14363-c3dc-4a54-8830-89ad44830915 None None] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 16:12:52.079699 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fa29364-2ec1-4fe6-b67e-05a0df44b8f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a98b1e4431940ab8d1fc95df2858a6c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 16:12:52.080285 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fa29364-2ec1-4fe6-b67e-05a0df44b8f9 admin admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:12:52.080559 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fa29364-2ec1-4fe6-b67e-05a0df44b8f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:12:52.080825 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fa29364-2ec1-4fe6-b67e-05a0df44b8f9 admin admin] PATH_INFO: `/v3/domains` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:12:52.081509 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fa29364-2ec1-4fe6-b67e-05a0df44b8f9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81768) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 16:12:52.082022 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fa29364-2ec1-4fe6-b67e-05a0df44b8f9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 16:12:52.084472 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fa29364-2ec1-4fe6-b67e-05a0df44b8f9 admin admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 16:12:52.096014 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 1227/2419] 10.222.0.15 () {58 vars in 1153 bytes} [Mon Aug 22 16:12:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 16:12:52.100427 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9d522a3-3ea0-4603-af44-e7e5af99312a None None] Authenticating user token {{(pid=81767) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 16:12:52.109801 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9d522a3-3ea0-4603-af44-e7e5af99312a None None] Validating token access rules against request {{(pid=81767) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 16:12:52.115027 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-942b2260-5561-44fd-83de-e2577294ce83 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a98b1e4431940ab8d1fc95df2858a6c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81767) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 16:12:52.115656 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-942b2260-5561-44fd-83de-e2577294ce83 admin admin] REQUEST_METHOD: `POST` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:12:52.115933 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-942b2260-5561-44fd-83de-e2577294ce83 admin admin] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:12:52.116199 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-942b2260-5561-44fd-83de-e2577294ce83 admin admin] PATH_INFO: `/v3/projects` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:12:52.116808 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-942b2260-5561-44fd-83de-e2577294ce83 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 16:12:52.118715 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-942b2260-5561-44fd-83de-e2577294ce83 admin admin] RBAC: Authorization granted {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 16:12:52.222289 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 1193/2420] 10.222.0.15 () {60 vars in 1153 bytes} [Mon Aug 22 16:12:52 2022] POST /identity/v3/projects => generated 383 bytes in 122 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 22 16:12:52.226387 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fa29364-2ec1-4fe6-b67e-05a0df44b8f9 admin admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 16:12:52.237226 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fa29364-2ec1-4fe6-b67e-05a0df44b8f9 admin admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 16:12:52.299493 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e011ede-cab1-4001-9a5e-eeaeef0917fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a98b1e4431940ab8d1fc95df2858a6c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 16:12:52.300102 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e011ede-cab1-4001-9a5e-eeaeef0917fb admin admin] REQUEST_METHOD: `POST` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:12:52.300384 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e011ede-cab1-4001-9a5e-eeaeef0917fb admin admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:12:52.300655 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e011ede-cab1-4001-9a5e-eeaeef0917fb admin admin] PATH_INFO: `/v3/users` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:12:52.301277 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e011ede-cab1-4001-9a5e-eeaeef0917fb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 16:12:52.303418 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e011ede-cab1-4001-9a5e-eeaeef0917fb admin admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 16:12:52.385070 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 1228/2421] 10.222.0.15 () {60 vars in 1144 bytes} [Mon Aug 22 16:12:52 2022] POST /identity/v3/users => generated 292 bytes in 159 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 22 16:12:52.389528 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-942b2260-5561-44fd-83de-e2577294ce83 admin admin] Authenticating user token {{(pid=81767) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 16:12:52.400518 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-942b2260-5561-44fd-83de-e2577294ce83 admin admin] Validating token access rules against request {{(pid=81767) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 16:12:52.405470 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b47fdb6f-0791-4d72-876b-5a95f62d7a39 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a98b1e4431940ab8d1fc95df2858a6c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81767) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 16:12:52.411040 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b47fdb6f-0791-4d72-876b-5a95f62d7a39 admin admin] REQUEST_METHOD: `GET` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:12:52.411040 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b47fdb6f-0791-4d72-876b-5a95f62d7a39 admin admin] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:12:52.411040 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b47fdb6f-0791-4d72-876b-5a95f62d7a39 admin admin] PATH_INFO: `/v3/roles` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:12:52.411040 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b47fdb6f-0791-4d72-876b-5a95f62d7a39 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 16:12:52.411040 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b47fdb6f-0791-4d72-876b-5a95f62d7a39 admin admin] RBAC: Authorization granted {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 16:12:52.419801 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 1194/2422] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Aug 22 16:12:52 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 22 16:12:52.424864 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e011ede-cab1-4001-9a5e-eeaeef0917fb admin admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 16:12:52.441208 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e011ede-cab1-4001-9a5e-eeaeef0917fb admin admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 16:12:52.447083 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1761a09d-beeb-46fb-afe2-aac1446c4d2c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a98b1e4431940ab8d1fc95df2858a6c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 16:12:52.447735 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1761a09d-beeb-46fb-afe2-aac1446c4d2c admin admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:12:52.452125 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1761a09d-beeb-46fb-afe2-aac1446c4d2c admin admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:12:52.452125 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1761a09d-beeb-46fb-afe2-aac1446c4d2c admin admin] PATH_INFO: `/v3/roles` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:12:52.452125 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1761a09d-beeb-46fb-afe2-aac1446c4d2c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 16:12:52.452125 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1761a09d-beeb-46fb-afe2-aac1446c4d2c admin admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 16:12:52.474459 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 1229/2423] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Aug 22 16:12:52 2022] GET /identity/v3/roles => generated 1420 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 22 16:12:52.478827 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b47fdb6f-0791-4d72-876b-5a95f62d7a39 admin admin] Authenticating user token {{(pid=81767) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 16:12:52.490922 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b47fdb6f-0791-4d72-876b-5a95f62d7a39 admin admin] Validating token access rules against request {{(pid=81767) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 16:12:52.496139 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-844eb472-d956-4f7b-8e76-2f0b6e37edba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a98b1e4431940ab8d1fc95df2858a6c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81767) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 16:12:52.496833 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-844eb472-d956-4f7b-8e76-2f0b6e37edba admin admin] REQUEST_METHOD: `PUT` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:12:52.497123 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-844eb472-d956-4f7b-8e76-2f0b6e37edba admin admin] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:12:52.497400 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-844eb472-d956-4f7b-8e76-2f0b6e37edba admin admin] PATH_INFO: `/v3/projects/8b642a9ea972449792a4705ec7580eb1/users/743cbbd27fd34e169b9bc2741a2a403e/roles/e6fc1ee3fb974125ba9b4fe55b12b45e` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:12:52.514835 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-844eb472-d956-4f7b-8e76-2f0b6e37edba admin admin] RBAC: Authorizing `identity:create_grant(project_id=8b642a9ea972449792a4705ec7580eb1, user_id=743cbbd27fd34e169b9bc2741a2a403e, role_id=e6fc1ee3fb974125ba9b4fe55b12b45e)` {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 16:12:52.517584 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-844eb472-d956-4f7b-8e76-2f0b6e37edba admin admin] RBAC: Authorization granted {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 16:12:52.520094 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG dbcounter [-] [81767] Writing DB stats keystone:SELECT=86,keystone:DELETE=11,keystone:INSERT=3 {{(pid=81767) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 16:12:52.604355 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 1195/2424] 10.222.0.15 () {60 vars in 1483 bytes} [Mon Aug 22 16:12:52 2022] PUT /identity/v3/projects/8b642a9ea972449792a4705ec7580eb1/users/743cbbd27fd34e169b9bc2741a2a403e/roles/e6fc1ee3fb974125ba9b4fe55b12b45e => generated 0 bytes in 117 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 22 16:12:52.605723 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66302f3d-6441-4c6c-97f6-3f09ef942eb7 None None] REQUEST_METHOD: `POST` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:12:52.606022 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66302f3d-6441-4c6c-97f6-3f09ef942eb7 None None] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:12:52.606291 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66302f3d-6441-4c6c-97f6-3f09ef942eb7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:12:52.635422 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.auth.core [None req-66302f3d-6441-4c6c-97f6-3f09ef942eb7 None None] MFA Rules not processed for user `743cbbd27fd34e169b9bc2741a2a403e`. Rule list: `[]` (Enabled: `True`). {{(pid=81768) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 16:12:52.636680 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-66302f3d-6441-4c6c-97f6-3f09ef942eb7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 16:12:52.706187 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 1230/2425] 10.222.0.15 () {58 vars in 958 bytes} [Mon Aug 22 16:12:52 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 22 16:12:52.711001 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-844eb472-d956-4f7b-8e76-2f0b6e37edba admin admin] Authenticating user token {{(pid=81767) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 16:12:52.795199 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-844eb472-d956-4f7b-8e76-2f0b6e37edba admin admin] Validating token access rules against request {{(pid=81767) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 16:12:52.800339 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54fe52a6-bfd0-4f50-bbd1-a5421fa7899c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a98b1e4431940ab8d1fc95df2858a6c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81767) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 16:12:52.800943 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54fe52a6-bfd0-4f50-bbd1-a5421fa7899c admin admin] REQUEST_METHOD: `POST` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:12:52.802086 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54fe52a6-bfd0-4f50-bbd1-a5421fa7899c admin admin] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:12:52.802086 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54fe52a6-bfd0-4f50-bbd1-a5421fa7899c admin admin] PATH_INFO: `/v3/projects` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:12:52.802409 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54fe52a6-bfd0-4f50-bbd1-a5421fa7899c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 16:12:52.804325 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54fe52a6-bfd0-4f50-bbd1-a5421fa7899c admin admin] RBAC: Authorization granted {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 16:12:52.934327 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 1196/2426] 10.222.0.15 () {60 vars in 1153 bytes} [Mon Aug 22 16:12:52 2022] POST /identity/v3/projects => generated 381 bytes in 224 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 22 16:12:52.938493 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66302f3d-6441-4c6c-97f6-3f09ef942eb7 None None] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 16:12:52.959424 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66302f3d-6441-4c6c-97f6-3f09ef942eb7 None None] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 16:12:52.962710 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 16:12:53.011766 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc6ef09a-5c81-4191-af96-ec6a5979297e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a98b1e4431940ab8d1fc95df2858a6c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 16:12:53.012265 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc6ef09a-5c81-4191-af96-ec6a5979297e admin admin] REQUEST_METHOD: `POST` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:12:53.012544 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc6ef09a-5c81-4191-af96-ec6a5979297e admin admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:12:53.012815 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc6ef09a-5c81-4191-af96-ec6a5979297e admin admin] PATH_INFO: `/v3/users` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:12:53.013432 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc6ef09a-5c81-4191-af96-ec6a5979297e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 16:12:53.015376 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc6ef09a-5c81-4191-af96-ec6a5979297e admin admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 16:12:53.161649 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 1231/2427] 10.222.0.15 () {60 vars in 1144 bytes} [Mon Aug 22 16:12:52 2022] POST /identity/v3/users => generated 297 bytes in 224 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 22 16:12:53.165928 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54fe52a6-bfd0-4f50-bbd1-a5421fa7899c admin admin] Authenticating user token {{(pid=81767) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 16:12:53.176430 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54fe52a6-bfd0-4f50-bbd1-a5421fa7899c admin admin] Validating token access rules against request {{(pid=81767) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 16:12:53.181548 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e1c402b-9881-4986-8d7b-682a4527606e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a98b1e4431940ab8d1fc95df2858a6c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81767) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 16:12:53.182145 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e1c402b-9881-4986-8d7b-682a4527606e admin admin] REQUEST_METHOD: `GET` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:12:53.182420 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e1c402b-9881-4986-8d7b-682a4527606e admin admin] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:12:53.182687 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e1c402b-9881-4986-8d7b-682a4527606e admin admin] PATH_INFO: `/v3/roles` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:12:53.183345 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e1c402b-9881-4986-8d7b-682a4527606e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 16:12:53.184918 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e1c402b-9881-4986-8d7b-682a4527606e admin admin] RBAC: Authorization granted {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 16:12:53.196192 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 1197/2428] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Aug 22 16:12:53 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 22 16:12:53.200952 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc6ef09a-5c81-4191-af96-ec6a5979297e admin admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 16:12:53.212051 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc6ef09a-5c81-4191-af96-ec6a5979297e admin admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 16:12:53.216769 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1173c325-d088-4bd2-adbe-653535395041 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a98b1e4431940ab8d1fc95df2858a6c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 16:12:53.217422 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1173c325-d088-4bd2-adbe-653535395041 admin admin] REQUEST_METHOD: `PUT` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:12:53.217693 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1173c325-d088-4bd2-adbe-653535395041 admin admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:12:53.217964 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1173c325-d088-4bd2-adbe-653535395041 admin admin] PATH_INFO: `/v3/projects/fb1b1e2aa24c4915ab8ae6b17239ccf8/users/8f3199989e13421794632708200cb79c/roles/9d9848cfd22e4b089db443c350e56670` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:12:53.238642 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1173c325-d088-4bd2-adbe-653535395041 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fb1b1e2aa24c4915ab8ae6b17239ccf8, user_id=8f3199989e13421794632708200cb79c, role_id=9d9848cfd22e4b089db443c350e56670)` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 16:12:53.242019 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1173c325-d088-4bd2-adbe-653535395041 admin admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 16:12:53.346695 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 1232/2429] 10.222.0.15 () {60 vars in 1483 bytes} [Mon Aug 22 16:12:53 2022] PUT /identity/v3/projects/fb1b1e2aa24c4915ab8ae6b17239ccf8/users/8f3199989e13421794632708200cb79c/roles/9d9848cfd22e4b089db443c350e56670 => generated 0 bytes in 147 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 16:12:53.350895 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e1c402b-9881-4986-8d7b-682a4527606e admin admin] Authenticating user token {{(pid=81767) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 16:12:53.361790 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e1c402b-9881-4986-8d7b-682a4527606e admin admin] Validating token access rules against request {{(pid=81767) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 16:12:53.430933 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e547812c-cb1f-4d9e-bf03-6d02bf022693 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a98b1e4431940ab8d1fc95df2858a6c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81767) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 16:12:53.431577 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e547812c-cb1f-4d9e-bf03-6d02bf022693 admin admin] REQUEST_METHOD: `GET` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:12:53.431856 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e547812c-cb1f-4d9e-bf03-6d02bf022693 admin admin] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:12:53.432126 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e547812c-cb1f-4d9e-bf03-6d02bf022693 admin admin] PATH_INFO: `/v3/roles` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:12:53.432773 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e547812c-cb1f-4d9e-bf03-6d02bf022693 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 16:12:53.434350 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e547812c-cb1f-4d9e-bf03-6d02bf022693 admin admin] RBAC: Authorization granted {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 16:12:53.444544 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 1198/2430] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Aug 22 16:12:53 2022] GET /identity/v3/roles => generated 1420 bytes in 94 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 22 16:12:53.448863 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1173c325-d088-4bd2-adbe-653535395041 admin admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 16:12:53.460721 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1173c325-d088-4bd2-adbe-653535395041 admin admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 16:12:53.465676 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47e00bc7-4b79-4cd3-b061-48455617d377 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a98b1e4431940ab8d1fc95df2858a6c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 16:12:53.466370 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e00bc7-4b79-4cd3-b061-48455617d377 admin admin] REQUEST_METHOD: `PUT` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:12:53.466661 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e00bc7-4b79-4cd3-b061-48455617d377 admin admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:12:53.466929 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e00bc7-4b79-4cd3-b061-48455617d377 admin admin] PATH_INFO: `/v3/projects/fb1b1e2aa24c4915ab8ae6b17239ccf8/users/8f3199989e13421794632708200cb79c/roles/9d9848cfd22e4b089db443c350e56670` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:12:53.469240 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47e00bc7-4b79-4cd3-b061-48455617d377 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fb1b1e2aa24c4915ab8ae6b17239ccf8, user_id=8f3199989e13421794632708200cb79c, role_id=9d9848cfd22e4b089db443c350e56670)` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 16:12:53.472033 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47e00bc7-4b79-4cd3-b061-48455617d377 admin admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 16:12:53.483343 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 1233/2431] 10.222.0.15 () {60 vars in 1483 bytes} [Mon Aug 22 16:12:53 2022] PUT /identity/v3/projects/fb1b1e2aa24c4915ab8ae6b17239ccf8/users/8f3199989e13421794632708200cb79c/roles/9d9848cfd22e4b089db443c350e56670 => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 16:12:53.495404 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a18144ad-5d5a-4021-9184-b67e9683ba14 None None] REQUEST_METHOD: `POST` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:12:53.495756 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a18144ad-5d5a-4021-9184-b67e9683ba14 None None] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:12:53.496039 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a18144ad-5d5a-4021-9184-b67e9683ba14 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:12:53.525281 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.auth.core [None req-a18144ad-5d5a-4021-9184-b67e9683ba14 None None] MFA Rules not processed for user `8f3199989e13421794632708200cb79c`. Rule list: `[]` (Enabled: `True`). {{(pid=81767) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 16:12:53.526503 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.fernet_utils [None req-a18144ad-5d5a-4021-9184-b67e9683ba14 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81767) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 16:12:53.580794 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG dbcounter [-] [81767] Writing DB stats keystone:INSERT=2,keystone:SELECT=98 {{(pid=81767) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 16:12:53.599760 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 1199/2432] 10.222.0.15 () {58 vars in 958 bytes} [Mon Aug 22 16:12:53 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 16:12:53.611580 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47e00bc7-4b79-4cd3-b061-48455617d377 admin admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 16:12:53.613433 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-47e00bc7-4b79-4cd3-b061-48455617d377 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 16:12:53.692823 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47e00bc7-4b79-4cd3-b061-48455617d377 admin admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 16:12:53.696935 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cd20a12-8ace-4740-b72e-b33651a32c6f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79706d480f864998b9f8594be407ceab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4ee0413f9ef4d4fb1f95a660babe090', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 16:12:53.697548 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cd20a12-8ace-4740-b72e-b33651a32c6f service cinder] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:12:53.697858 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cd20a12-8ace-4740-b72e-b33651a32c6f service cinder] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:12:53.698095 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cd20a12-8ace-4740-b72e-b33651a32c6f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:12:53.708258 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cd20a12-8ace-4740-b72e-b33651a32c6f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 16:12:53.710028 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cd20a12-8ace-4740-b72e-b33651a32c6f service cinder] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 16:12:53.720876 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 1234/2433] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Aug 22 16:12:53 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 16:12:53.971385 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a18144ad-5d5a-4021-9184-b67e9683ba14 None None] Authenticating user token {{(pid=81767) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 16:12:53.981446 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a18144ad-5d5a-4021-9184-b67e9683ba14 None None] Validating token access rules against request {{(pid=81767) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 16:12:53.985723 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9494fbe1-766d-4f05-8396-990d1d5172e4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79706d480f864998b9f8594be407ceab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4ee0413f9ef4d4fb1f95a660babe090', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81767) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 16:12:53.986333 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9494fbe1-766d-4f05-8396-990d1d5172e4 service cinder] REQUEST_METHOD: `GET` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:12:53.986609 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9494fbe1-766d-4f05-8396-990d1d5172e4 service cinder] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:12:53.986875 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9494fbe1-766d-4f05-8396-990d1d5172e4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:12:54.051161 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9494fbe1-766d-4f05-8396-990d1d5172e4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 16:12:54.052645 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9494fbe1-766d-4f05-8396-990d1d5172e4 service cinder] RBAC: Authorization granted {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 16:12:54.062340 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 1200/2434] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Aug 22 16:12:53 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 16:12:54.595539 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cd20a12-8ace-4740-b72e-b33651a32c6f service cinder] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 16:12:54.660297 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cd20a12-8ace-4740-b72e-b33651a32c6f service cinder] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 16:12:54.664882 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e178dfc1-4147-41b7-8e9d-e201e0d73362 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a98b1e4431940ab8d1fc95df2858a6c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 16:12:54.665503 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e178dfc1-4147-41b7-8e9d-e201e0d73362 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:12:54.665775 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e178dfc1-4147-41b7-8e9d-e201e0d73362 admin admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:12:54.666043 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e178dfc1-4147-41b7-8e9d-e201e0d73362 admin admin] PATH_INFO: `/v3/users/743cbbd27fd34e169b9bc2741a2a403e` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:12:54.667418 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e178dfc1-4147-41b7-8e9d-e201e0d73362 admin admin] RBAC: Authorizing `identity:delete_user(user_id=743cbbd27fd34e169b9bc2741a2a403e)` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 16:12:54.669184 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e178dfc1-4147-41b7-8e9d-e201e0d73362 admin admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 16:12:54.680940 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 16:12:54.880195 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-e178dfc1-4147-41b7-8e9d-e201e0d73362 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '743cbbd27fd34e169b9bc2741a2a403e'} {{(pid=81768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 16:12:54.880992 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-e178dfc1-4147-41b7-8e9d-e201e0d73362 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '743cbbd27fd34e169b9bc2741a2a403e'} {{(pid=81768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 16:12:54.881601 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-e178dfc1-4147-41b7-8e9d-e201e0d73362 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '743cbbd27fd34e169b9bc2741a2a403e'} {{(pid=81768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 16:12:54.888977 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-e178dfc1-4147-41b7-8e9d-e201e0d73362 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '743cbbd27fd34e169b9bc2741a2a403e'} {{(pid=81768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 16:12:54.943896 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-e178dfc1-4147-41b7-8e9d-e201e0d73362 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '743cbbd27fd34e169b9bc2741a2a403e'} {{(pid=81768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 16:12:54.963906 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 1235/2435] 10.222.0.15 () {58 vars in 1224 bytes} [Mon Aug 22 16:12:54 2022] DELETE /identity/v3/users/743cbbd27fd34e169b9bc2741a2a403e => generated 0 bytes in 369 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 16:12:54.968564 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-319c0584-d854-40db-b900-dd4907c0cab9 None None] REQUEST_METHOD: `POST` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:12:54.968852 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-319c0584-d854-40db-b900-dd4907c0cab9 None None] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:12:54.969964 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-319c0584-d854-40db-b900-dd4907c0cab9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:12:54.990114 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.auth.core [None req-319c0584-d854-40db-b900-dd4907c0cab9 None None] MFA Rules not processed for user `6b2199ae0fb0480aa917ada391f5ec54`. Rule list: `[]` (Enabled: `True`). {{(pid=81767) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 16:12:54.991520 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.fernet_utils [None req-319c0584-d854-40db-b900-dd4907c0cab9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81767) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 16:12:55.051302 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 1201/2436] 10.222.0.15 () {58 vars in 958 bytes} [Mon Aug 22 16:12:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 22 16:12:55.061381 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e178dfc1-4147-41b7-8e9d-e201e0d73362 admin admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 16:12:55.063650 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-e178dfc1-4147-41b7-8e9d-e201e0d73362 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 16:12:55.132863 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e178dfc1-4147-41b7-8e9d-e201e0d73362 admin admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 16:12:55.136869 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b46b705e-c45f-46f8-b64a-32392e017f31 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '234687adad594f1aa45775a565ffbdc7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4ee0413f9ef4d4fb1f95a660babe090', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 16:12:55.137557 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b46b705e-c45f-46f8-b64a-32392e017f31 service neutron] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:12:55.137879 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b46b705e-c45f-46f8-b64a-32392e017f31 service neutron] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:12:55.138153 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b46b705e-c45f-46f8-b64a-32392e017f31 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:12:55.149475 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b46b705e-c45f-46f8-b64a-32392e017f31 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 16:12:55.150942 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b46b705e-c45f-46f8-b64a-32392e017f31 service neutron] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 16:12:55.162189 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 1236/2437] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Aug 22 16:12:55 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 16:12:55.564408 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-319c0584-d854-40db-b900-dd4907c0cab9 None None] Authenticating user token {{(pid=81767) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 16:12:55.565966 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.fernet_utils [None req-319c0584-d854-40db-b900-dd4907c0cab9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81767) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 16:12:55.579503 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-319c0584-d854-40db-b900-dd4907c0cab9 None None] Validating token access rules against request {{(pid=81767) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 16:12:55.585262 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c22bb650-6173-45a8-bbea-7e0adad22f84 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a98b1e4431940ab8d1fc95df2858a6c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81767) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 16:12:55.585911 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c22bb650-6173-45a8-bbea-7e0adad22f84 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:12:55.586189 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c22bb650-6173-45a8-bbea-7e0adad22f84 admin admin] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:12:55.586638 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c22bb650-6173-45a8-bbea-7e0adad22f84 admin admin] PATH_INFO: `/v3/projects/8b642a9ea972449792a4705ec7580eb1` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:12:55.588208 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c22bb650-6173-45a8-bbea-7e0adad22f84 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8b642a9ea972449792a4705ec7580eb1)` {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 16:12:55.591895 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c22bb650-6173-45a8-bbea-7e0adad22f84 admin admin] RBAC: Authorization granted {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 16:12:55.725441 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.notifications [None req-c22bb650-6173-45a8-bbea-7e0adad22f84 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8b642a9ea972449792a4705ec7580eb1', 'request_id': 'req-c22bb650-6173-45a8-bbea-7e0adad22f84'} {{(pid=81767) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 16:12:55.730715 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.notifications [None req-c22bb650-6173-45a8-bbea-7e0adad22f84 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8b642a9ea972449792a4705ec7580eb1', 'request_id': 'req-c22bb650-6173-45a8-bbea-7e0adad22f84'} {{(pid=81767) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 16:12:55.812161 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.notifications [None req-c22bb650-6173-45a8-bbea-7e0adad22f84 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8b642a9ea972449792a4705ec7580eb1', 'request_id': 'req-c22bb650-6173-45a8-bbea-7e0adad22f84'} {{(pid=81767) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 16:12:55.824067 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.notifications [None req-c22bb650-6173-45a8-bbea-7e0adad22f84 admin admin] The token cache is being invalidate because project 8b642a9ea972449792a4705ec7580eb1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81767) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 22 16:12:55.824496 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.notifications [None req-c22bb650-6173-45a8-bbea-7e0adad22f84 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81767) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 16:12:55.825054 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.notifications [None req-c22bb650-6173-45a8-bbea-7e0adad22f84 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81767) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 16:12:55.826799 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 1202/2438] 10.222.0.15 () {58 vars in 1233 bytes} [Mon Aug 22 16:12:55 2022] DELETE /identity/v3/projects/8b642a9ea972449792a4705ec7580eb1 => generated 0 bytes in 263 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 22 16:12:55.832507 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b46b705e-c45f-46f8-b64a-32392e017f31 service neutron] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 16:12:55.843918 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b46b705e-c45f-46f8-b64a-32392e017f31 service neutron] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 16:12:55.905200 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2426afe8-1ebf-45a3-a4c8-db3b66915144 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a98b1e4431940ab8d1fc95df2858a6c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 16:12:55.905836 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2426afe8-1ebf-45a3-a4c8-db3b66915144 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:12:55.906115 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2426afe8-1ebf-45a3-a4c8-db3b66915144 admin admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:12:55.906386 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2426afe8-1ebf-45a3-a4c8-db3b66915144 admin admin] PATH_INFO: `/v3/users/8f3199989e13421794632708200cb79c` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:12:55.907581 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2426afe8-1ebf-45a3-a4c8-db3b66915144 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8f3199989e13421794632708200cb79c)` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 16:12:55.909522 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2426afe8-1ebf-45a3-a4c8-db3b66915144 admin admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 16:12:56.099916 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:DELETE=15,keystone:SELECT=84,keystone:INSERT=1 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 16:12:56.113031 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-2426afe8-1ebf-45a3-a4c8-db3b66915144 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8f3199989e13421794632708200cb79c'} {{(pid=81768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 16:12:56.116617 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-2426afe8-1ebf-45a3-a4c8-db3b66915144 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8f3199989e13421794632708200cb79c'} {{(pid=81768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 16:12:56.116617 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-2426afe8-1ebf-45a3-a4c8-db3b66915144 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8f3199989e13421794632708200cb79c'} {{(pid=81768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 16:12:56.122378 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-2426afe8-1ebf-45a3-a4c8-db3b66915144 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8f3199989e13421794632708200cb79c'} {{(pid=81768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 16:12:56.189419 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-2426afe8-1ebf-45a3-a4c8-db3b66915144 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8f3199989e13421794632708200cb79c'} {{(pid=81768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 16:12:56.210586 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 1237/2439] 10.222.0.15 () {58 vars in 1224 bytes} [Mon Aug 22 16:12:55 2022] DELETE /identity/v3/users/8f3199989e13421794632708200cb79c => generated 0 bytes in 379 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 22 16:12:56.633926 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c22bb650-6173-45a8-bbea-7e0adad22f84 admin admin] Authenticating user token {{(pid=81767) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 16:12:56.635489 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.fernet_utils [None req-c22bb650-6173-45a8-bbea-7e0adad22f84 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81767) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 16:12:56.651776 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c22bb650-6173-45a8-bbea-7e0adad22f84 admin admin] Validating token access rules against request {{(pid=81767) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 16:12:56.667994 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG dbcounter [-] [81767] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81767) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 16:12:56.719305 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1699ba34-3ac3-4717-9b3b-833021c3ae52 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a98b1e4431940ab8d1fc95df2858a6c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81767) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 16:12:56.719935 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1699ba34-3ac3-4717-9b3b-833021c3ae52 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:12:56.720216 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1699ba34-3ac3-4717-9b3b-833021c3ae52 admin admin] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:12:56.720488 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1699ba34-3ac3-4717-9b3b-833021c3ae52 admin admin] PATH_INFO: `/v3/projects/fb1b1e2aa24c4915ab8ae6b17239ccf8` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:12:56.721624 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1699ba34-3ac3-4717-9b3b-833021c3ae52 admin admin] RBAC: Authorizing `identity:delete_project(project_id=fb1b1e2aa24c4915ab8ae6b17239ccf8)` {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 16:12:56.723611 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1699ba34-3ac3-4717-9b3b-833021c3ae52 admin admin] RBAC: Authorization granted {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 16:12:56.858795 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.notifications [None req-1699ba34-3ac3-4717-9b3b-833021c3ae52 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fb1b1e2aa24c4915ab8ae6b17239ccf8', 'request_id': 'req-1699ba34-3ac3-4717-9b3b-833021c3ae52'} {{(pid=81767) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 16:12:56.866803 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.notifications [None req-1699ba34-3ac3-4717-9b3b-833021c3ae52 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fb1b1e2aa24c4915ab8ae6b17239ccf8', 'request_id': 'req-1699ba34-3ac3-4717-9b3b-833021c3ae52'} {{(pid=81767) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 16:12:56.972198 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.notifications [None req-1699ba34-3ac3-4717-9b3b-833021c3ae52 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fb1b1e2aa24c4915ab8ae6b17239ccf8', 'request_id': 'req-1699ba34-3ac3-4717-9b3b-833021c3ae52'} {{(pid=81767) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 16:12:56.987040 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.notifications [None req-1699ba34-3ac3-4717-9b3b-833021c3ae52 admin admin] The token cache is being invalidate because project fb1b1e2aa24c4915ab8ae6b17239ccf8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81767) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 22 16:12:56.987371 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.notifications [None req-1699ba34-3ac3-4717-9b3b-833021c3ae52 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81767) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 16:12:56.988044 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.notifications [None req-1699ba34-3ac3-4717-9b3b-833021c3ae52 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81767) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 16:12:56.989769 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 1203/2440] 10.222.0.15 () {58 vars in 1233 bytes} [Mon Aug 22 16:12:56 2022] DELETE /identity/v3/projects/fb1b1e2aa24c4915ab8ae6b17239ccf8 => generated 0 bytes in 356 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 16:12:56.994511 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be6e1d85-22c2-462c-87c0-1f1356652dfb None None] REQUEST_METHOD: `POST` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:12:56.995013 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be6e1d85-22c2-462c-87c0-1f1356652dfb None None] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:12:56.995131 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be6e1d85-22c2-462c-87c0-1f1356652dfb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:12:57.014891 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.auth.core [None req-be6e1d85-22c2-462c-87c0-1f1356652dfb None None] MFA Rules not processed for user `6b2199ae0fb0480aa917ada391f5ec54`. Rule list: `[]` (Enabled: `True`). {{(pid=81768) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 16:12:57.016234 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-be6e1d85-22c2-462c-87c0-1f1356652dfb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 16:12:57.102560 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 1238/2441] 10.222.0.15 () {58 vars in 958 bytes} [Mon Aug 22 16:12:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 16:12:57.111165 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bed0ba49-2063-4c7a-aca4-ab45370b7f0f None None] REQUEST_METHOD: `POST` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:12:57.111701 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bed0ba49-2063-4c7a-aca4-ab45370b7f0f None None] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:12:57.111988 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bed0ba49-2063-4c7a-aca4-ab45370b7f0f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:12:57.132830 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.auth.core [None req-bed0ba49-2063-4c7a-aca4-ab45370b7f0f None None] MFA Rules not processed for user `6b2199ae0fb0480aa917ada391f5ec54`. Rule list: `[]` (Enabled: `True`). {{(pid=81767) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 16:12:57.134479 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.fernet_utils [None req-bed0ba49-2063-4c7a-aca4-ab45370b7f0f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81767) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 16:12:57.144964 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 1204/2442] 10.222.0.15 () {58 vars in 958 bytes} [Mon Aug 22 16:12:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 16:12:57.150150 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be6e1d85-22c2-462c-87c0-1f1356652dfb None None] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 16:12:57.166204 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be6e1d85-22c2-462c-87c0-1f1356652dfb None None] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 16:12:57.173343 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04aeb98b-92a2-4a61-87b3-840ccdca2d99 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a98b1e4431940ab8d1fc95df2858a6c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 16:12:57.174154 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04aeb98b-92a2-4a61-87b3-840ccdca2d99 admin admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:12:57.174525 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04aeb98b-92a2-4a61-87b3-840ccdca2d99 admin admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:12:57.174905 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04aeb98b-92a2-4a61-87b3-840ccdca2d99 admin admin] PATH_INFO: `/v3/domains` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:12:57.175736 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04aeb98b-92a2-4a61-87b3-840ccdca2d99 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81768) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 16:12:57.176146 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04aeb98b-92a2-4a61-87b3-840ccdca2d99 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 16:12:57.177893 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04aeb98b-92a2-4a61-87b3-840ccdca2d99 admin admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 16:12:57.192350 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 1239/2443] 10.222.0.15 () {58 vars in 1153 bytes} [Mon Aug 22 16:12:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 22 16:12:57.197130 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bed0ba49-2063-4c7a-aca4-ab45370b7f0f None None] Authenticating user token {{(pid=81767) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 16:12:57.207739 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bed0ba49-2063-4c7a-aca4-ab45370b7f0f None None] Validating token access rules against request {{(pid=81767) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 16:12:57.213746 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-304f05da-9cfa-42c7-81bc-99aca5b51ed7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a98b1e4431940ab8d1fc95df2858a6c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81767) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 16:12:57.214349 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-304f05da-9cfa-42c7-81bc-99aca5b51ed7 admin admin] REQUEST_METHOD: `POST` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:12:57.215658 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-304f05da-9cfa-42c7-81bc-99aca5b51ed7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:12:57.215658 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-304f05da-9cfa-42c7-81bc-99aca5b51ed7 admin admin] PATH_INFO: `/v3/projects` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:12:57.215658 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-304f05da-9cfa-42c7-81bc-99aca5b51ed7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 16:12:57.217387 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-304f05da-9cfa-42c7-81bc-99aca5b51ed7 admin admin] RBAC: Authorization granted {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 16:12:57.288826 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 1205/2444] 10.222.0.15 () {60 vars in 1153 bytes} [Mon Aug 22 16:12:57 2022] POST /identity/v3/projects => generated 373 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 22 16:12:57.294056 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04aeb98b-92a2-4a61-87b3-840ccdca2d99 admin admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 16:12:57.304805 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04aeb98b-92a2-4a61-87b3-840ccdca2d99 admin admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 16:12:57.385762 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dc03128-0e13-4f2f-b767-3f8e2402415f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a98b1e4431940ab8d1fc95df2858a6c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 16:12:57.386365 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc03128-0e13-4f2f-b767-3f8e2402415f admin admin] REQUEST_METHOD: `POST` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:12:57.386641 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc03128-0e13-4f2f-b767-3f8e2402415f admin admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:12:57.386912 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc03128-0e13-4f2f-b767-3f8e2402415f admin admin] PATH_INFO: `/v3/users` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:12:57.387568 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dc03128-0e13-4f2f-b767-3f8e2402415f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 16:12:57.389474 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dc03128-0e13-4f2f-b767-3f8e2402415f admin admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 16:12:57.532373 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 1240/2445] 10.222.0.15 () {60 vars in 1144 bytes} [Mon Aug 22 16:12:57 2022] POST /identity/v3/users => generated 287 bytes in 232 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 22 16:12:57.533058 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-304f05da-9cfa-42c7-81bc-99aca5b51ed7 admin admin] Authenticating user token {{(pid=81767) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 16:12:57.543876 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-304f05da-9cfa-42c7-81bc-99aca5b51ed7 admin admin] Validating token access rules against request {{(pid=81767) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 16:12:57.549505 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3904d067-6658-419e-9fbe-2ad97a3d65a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a98b1e4431940ab8d1fc95df2858a6c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81767) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 16:12:57.550244 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3904d067-6658-419e-9fbe-2ad97a3d65a8 admin admin] REQUEST_METHOD: `GET` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:12:57.550522 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3904d067-6658-419e-9fbe-2ad97a3d65a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:12:57.550859 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3904d067-6658-419e-9fbe-2ad97a3d65a8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:12:57.551534 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3904d067-6658-419e-9fbe-2ad97a3d65a8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 16:12:57.553112 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3904d067-6658-419e-9fbe-2ad97a3d65a8 admin admin] RBAC: Authorization granted {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 16:12:57.570652 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 1206/2446] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Aug 22 16:12:57 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 22 16:12:57.578269 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dc03128-0e13-4f2f-b767-3f8e2402415f admin admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 16:12:57.589225 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dc03128-0e13-4f2f-b767-3f8e2402415f admin admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 16:12:57.594660 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dd3e33e-6faa-4b50-a547-6a1e9bfa280f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a98b1e4431940ab8d1fc95df2858a6c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 16:12:57.595290 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd3e33e-6faa-4b50-a547-6a1e9bfa280f admin admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:12:57.595727 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd3e33e-6faa-4b50-a547-6a1e9bfa280f admin admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:12:57.596033 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd3e33e-6faa-4b50-a547-6a1e9bfa280f admin admin] PATH_INFO: `/v3/roles` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:12:57.596680 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dd3e33e-6faa-4b50-a547-6a1e9bfa280f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 16:12:57.598289 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dd3e33e-6faa-4b50-a547-6a1e9bfa280f admin admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 16:12:57.610765 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 1241/2447] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Aug 22 16:12:57 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 22 16:12:57.615148 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3904d067-6658-419e-9fbe-2ad97a3d65a8 admin admin] Authenticating user token {{(pid=81767) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 16:12:57.625338 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3904d067-6658-419e-9fbe-2ad97a3d65a8 admin admin] Validating token access rules against request {{(pid=81767) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 16:12:57.630076 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bbcdc4d-6ba6-4be9-ac85-0810e3f31871 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a98b1e4431940ab8d1fc95df2858a6c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81767) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 16:12:57.630731 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bbcdc4d-6ba6-4be9-ac85-0810e3f31871 admin admin] REQUEST_METHOD: `PUT` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:12:57.631010 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bbcdc4d-6ba6-4be9-ac85-0810e3f31871 admin admin] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:12:57.631376 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bbcdc4d-6ba6-4be9-ac85-0810e3f31871 admin admin] PATH_INFO: `/v3/projects/c7c13dfa47ae41439398559bbdfe591a/users/344a1e1a3d234730bb3974282c27fefb/roles/e6fc1ee3fb974125ba9b4fe55b12b45e` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:12:57.646944 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bbcdc4d-6ba6-4be9-ac85-0810e3f31871 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c7c13dfa47ae41439398559bbdfe591a, user_id=344a1e1a3d234730bb3974282c27fefb, role_id=e6fc1ee3fb974125ba9b4fe55b12b45e)` {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 16:12:57.649663 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bbcdc4d-6ba6-4be9-ac85-0810e3f31871 admin admin] RBAC: Authorization granted {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 16:12:57.689460 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d23e2c7-ff21-4f51-adbb-aff5a7856de0 None None] REQUEST_METHOD: `POST` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:12:57.689739 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d23e2c7-ff21-4f51-adbb-aff5a7856de0 None None] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:12:57.690014 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d23e2c7-ff21-4f51-adbb-aff5a7856de0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:12:57.709801 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.auth.core [None req-5d23e2c7-ff21-4f51-adbb-aff5a7856de0 None None] MFA Rules not processed for user `058e76354dce412cb78b2875c88f2791`. Rule list: `[]` (Enabled: `True`). {{(pid=81768) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 16:12:57.711044 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-5d23e2c7-ff21-4f51-adbb-aff5a7856de0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 16:12:57.721513 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=91,keystone:DELETE=5,keystone:INSERT=4 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 16:12:57.769442 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 1207/2448] 10.222.0.15 () {60 vars in 1483 bytes} [Mon Aug 22 16:12:57 2022] PUT /identity/v3/projects/c7c13dfa47ae41439398559bbdfe591a/users/344a1e1a3d234730bb3974282c27fefb/roles/e6fc1ee3fb974125ba9b4fe55b12b45e => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 16:12:57.783155 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 1242/2449] 10.222.0.15 () {58 vars in 1009 bytes} [Mon Aug 22 16:12:57 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 22 16:12:57.783697 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6021d593-b5bf-4ec5-831b-2ed40fd35059 None None] REQUEST_METHOD: `POST` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:12:57.784195 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6021d593-b5bf-4ec5-831b-2ed40fd35059 None None] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:12:57.784304 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6021d593-b5bf-4ec5-831b-2ed40fd35059 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:12:57.818945 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.auth.core [None req-6021d593-b5bf-4ec5-831b-2ed40fd35059 None None] MFA Rules not processed for user `344a1e1a3d234730bb3974282c27fefb`. Rule list: `[]` (Enabled: `True`). {{(pid=81767) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 16:12:57.820201 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.fernet_utils [None req-6021d593-b5bf-4ec5-831b-2ed40fd35059 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81767) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 16:12:57.836955 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d23e2c7-ff21-4f51-adbb-aff5a7856de0 None None] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 16:12:57.838560 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-5d23e2c7-ff21-4f51-adbb-aff5a7856de0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 16:12:57.852322 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG dbcounter [-] [81767] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81767) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 16:12:57.898945 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 1208/2450] 10.222.0.15 () {58 vars in 958 bytes} [Mon Aug 22 16:12:57 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 16:12:57.903616 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6021d593-b5bf-4ec5-831b-2ed40fd35059 None None] Authenticating user token {{(pid=81767) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 16:12:57.939076 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d23e2c7-ff21-4f51-adbb-aff5a7856de0 None None] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 16:12:57.971942 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6021d593-b5bf-4ec5-831b-2ed40fd35059 None None] Validating token access rules against request {{(pid=81767) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 16:12:57.976662 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7277a9d9-6370-481b-9dda-a12127627194 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a98b1e4431940ab8d1fc95df2858a6c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81767) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 16:12:57.977268 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7277a9d9-6370-481b-9dda-a12127627194 admin admin] REQUEST_METHOD: `POST` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:12:57.977545 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7277a9d9-6370-481b-9dda-a12127627194 admin admin] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:12:57.977813 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7277a9d9-6370-481b-9dda-a12127627194 admin admin] PATH_INFO: `/v3/projects` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:12:57.978440 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7277a9d9-6370-481b-9dda-a12127627194 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 16:12:57.980406 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7277a9d9-6370-481b-9dda-a12127627194 admin admin] RBAC: Authorization granted {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 16:12:58.004338 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fca132c-1eae-47c2-9f81-4fac75cbb17c service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a09d128dee9438283633e51f4834643', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4ee0413f9ef4d4fb1f95a660babe090', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 16:12:58.004949 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fca132c-1eae-47c2-9f81-4fac75cbb17c service swift] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:12:58.005215 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fca132c-1eae-47c2-9f81-4fac75cbb17c service swift] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:12:58.005650 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fca132c-1eae-47c2-9f81-4fac75cbb17c service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:12:58.063482 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fca132c-1eae-47c2-9f81-4fac75cbb17c service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 16:12:58.065009 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fca132c-1eae-47c2-9f81-4fac75cbb17c service swift] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 16:12:58.074057 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 1243/2451] 10.222.0.15 () {60 vars in 1384 bytes} [Mon Aug 22 16:12:57 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 238 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Aug 22 16:12:58.110937 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 1209/2452] 10.222.0.15 () {60 vars in 1153 bytes} [Mon Aug 22 16:12:57 2022] POST /identity/v3/projects => generated 375 bytes in 205 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 22 16:12:58.112151 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fca132c-1eae-47c2-9f81-4fac75cbb17c service swift] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 16:12:58.124582 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=100 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 16:12:58.126105 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fca132c-1eae-47c2-9f81-4fac75cbb17c service swift] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 16:12:58.214145 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd78395f-7029-4974-ba2a-49604862ef67 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a98b1e4431940ab8d1fc95df2858a6c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 16:12:58.214650 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd78395f-7029-4974-ba2a-49604862ef67 admin admin] REQUEST_METHOD: `POST` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:12:58.214926 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd78395f-7029-4974-ba2a-49604862ef67 admin admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:12:58.215198 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd78395f-7029-4974-ba2a-49604862ef67 admin admin] PATH_INFO: `/v3/users` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:12:58.215865 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd78395f-7029-4974-ba2a-49604862ef67 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 16:12:58.217816 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd78395f-7029-4974-ba2a-49604862ef67 admin admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 16:12:58.327079 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 1244/2453] 10.222.0.15 () {60 vars in 1144 bytes} [Mon Aug 22 16:12:58 2022] POST /identity/v3/users => generated 294 bytes in 215 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 22 16:12:58.331767 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7277a9d9-6370-481b-9dda-a12127627194 admin admin] Authenticating user token {{(pid=81767) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 16:12:58.348076 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7277a9d9-6370-481b-9dda-a12127627194 admin admin] Validating token access rules against request {{(pid=81767) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 16:12:58.354294 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a15446b-1bbb-4027-ba24-f8ca3acf096e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a98b1e4431940ab8d1fc95df2858a6c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81767) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 16:12:58.355646 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a15446b-1bbb-4027-ba24-f8ca3acf096e admin admin] REQUEST_METHOD: `GET` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:12:58.355989 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a15446b-1bbb-4027-ba24-f8ca3acf096e admin admin] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:12:58.356386 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a15446b-1bbb-4027-ba24-f8ca3acf096e admin admin] PATH_INFO: `/v3/roles` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:12:58.357096 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a15446b-1bbb-4027-ba24-f8ca3acf096e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 16:12:58.359575 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a15446b-1bbb-4027-ba24-f8ca3acf096e admin admin] RBAC: Authorization granted {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 16:12:58.370699 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 1210/2454] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Aug 22 16:12:58 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 22 16:12:58.376206 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd78395f-7029-4974-ba2a-49604862ef67 admin admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 16:12:58.393607 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd78395f-7029-4974-ba2a-49604862ef67 admin admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 16:12:58.399980 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8223ced9-f083-473c-9803-a3abb4014bd4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a98b1e4431940ab8d1fc95df2858a6c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 16:12:58.400689 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8223ced9-f083-473c-9803-a3abb4014bd4 admin admin] REQUEST_METHOD: `PUT` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:12:58.400982 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8223ced9-f083-473c-9803-a3abb4014bd4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:12:58.401587 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8223ced9-f083-473c-9803-a3abb4014bd4 admin admin] PATH_INFO: `/v3/projects/ef2d9591bc134ba18519a0e19ede6253/users/c2648c3da398449bb38f2c578faa8266/roles/9d9848cfd22e4b089db443c350e56670` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:12:58.422826 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8223ced9-f083-473c-9803-a3abb4014bd4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ef2d9591bc134ba18519a0e19ede6253, user_id=c2648c3da398449bb38f2c578faa8266, role_id=9d9848cfd22e4b089db443c350e56670)` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 16:12:58.426003 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8223ced9-f083-473c-9803-a3abb4014bd4 admin admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 16:12:58.591688 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 1245/2455] 10.222.0.15 () {60 vars in 1483 bytes} [Mon Aug 22 16:12:58 2022] PUT /identity/v3/projects/ef2d9591bc134ba18519a0e19ede6253/users/c2648c3da398449bb38f2c578faa8266/roles/9d9848cfd22e4b089db443c350e56670 => generated 0 bytes in 214 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 16:12:58.593571 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a15446b-1bbb-4027-ba24-f8ca3acf096e admin admin] Authenticating user token {{(pid=81767) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 16:12:58.603945 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a15446b-1bbb-4027-ba24-f8ca3acf096e admin admin] Validating token access rules against request {{(pid=81767) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 16:12:58.667842 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f101d7e-c54b-49f9-bf00-cb1657161fcc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a98b1e4431940ab8d1fc95df2858a6c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81767) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 16:12:58.668488 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f101d7e-c54b-49f9-bf00-cb1657161fcc admin admin] REQUEST_METHOD: `GET` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:12:58.668726 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f101d7e-c54b-49f9-bf00-cb1657161fcc admin admin] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:12:58.668998 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f101d7e-c54b-49f9-bf00-cb1657161fcc admin admin] PATH_INFO: `/v3/roles` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:12:58.670239 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f101d7e-c54b-49f9-bf00-cb1657161fcc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 16:12:58.671509 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f101d7e-c54b-49f9-bf00-cb1657161fcc admin admin] RBAC: Authorization granted {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 16:12:58.683211 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 1211/2456] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Aug 22 16:12:58 2022] GET /identity/v3/roles => generated 1420 bytes in 91 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 22 16:12:58.689754 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8223ced9-f083-473c-9803-a3abb4014bd4 admin admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 16:12:58.700686 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8223ced9-f083-473c-9803-a3abb4014bd4 admin admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 16:12:58.709601 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42f78888-5c6f-456b-bb7e-ff69a2c0bc38 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a98b1e4431940ab8d1fc95df2858a6c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 16:12:58.710354 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42f78888-5c6f-456b-bb7e-ff69a2c0bc38 admin admin] REQUEST_METHOD: `PUT` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:12:58.710905 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42f78888-5c6f-456b-bb7e-ff69a2c0bc38 admin admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:12:58.716511 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42f78888-5c6f-456b-bb7e-ff69a2c0bc38 admin admin] PATH_INFO: `/v3/projects/ef2d9591bc134ba18519a0e19ede6253/users/c2648c3da398449bb38f2c578faa8266/roles/9d9848cfd22e4b089db443c350e56670` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:12:58.716511 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42f78888-5c6f-456b-bb7e-ff69a2c0bc38 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ef2d9591bc134ba18519a0e19ede6253, user_id=c2648c3da398449bb38f2c578faa8266, role_id=9d9848cfd22e4b089db443c350e56670)` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 16:12:58.716511 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42f78888-5c6f-456b-bb7e-ff69a2c0bc38 admin admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 16:12:58.745153 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 1246/2457] 10.222.0.15 () {60 vars in 1483 bytes} [Mon Aug 22 16:12:58 2022] PUT /identity/v3/projects/ef2d9591bc134ba18519a0e19ede6253/users/c2648c3da398449bb38f2c578faa8266/roles/9d9848cfd22e4b089db443c350e56670 => generated 0 bytes in 45 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 16:12:58.746749 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a383f1db-22c5-419d-8204-71bf436bea87 None None] REQUEST_METHOD: `POST` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:12:58.747127 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a383f1db-22c5-419d-8204-71bf436bea87 None None] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:12:58.747510 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a383f1db-22c5-419d-8204-71bf436bea87 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:12:58.779678 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.auth.core [None req-a383f1db-22c5-419d-8204-71bf436bea87 None None] MFA Rules not processed for user `c2648c3da398449bb38f2c578faa8266`. Rule list: `[]` (Enabled: `True`). {{(pid=81767) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 16:12:58.779678 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.fernet_utils [None req-a383f1db-22c5-419d-8204-71bf436bea87 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81767) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 16:12:58.819382 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG dbcounter [-] [81767] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81767) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 16:12:58.912283 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 1212/2458] 10.222.0.15 () {58 vars in 958 bytes} [Mon Aug 22 16:12:58 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 16:12:59.026850 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42f78888-5c6f-456b-bb7e-ff69a2c0bc38 admin admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 16:12:59.029355 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-42f78888-5c6f-456b-bb7e-ff69a2c0bc38 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 16:12:59.096873 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42f78888-5c6f-456b-bb7e-ff69a2c0bc38 admin admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 16:12:59.100755 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00625419-72f0-42bb-8828-7c1c7fe751e6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79706d480f864998b9f8594be407ceab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4ee0413f9ef4d4fb1f95a660babe090', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 16:12:59.101406 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00625419-72f0-42bb-8828-7c1c7fe751e6 service cinder] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:12:59.105504 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00625419-72f0-42bb-8828-7c1c7fe751e6 service cinder] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:12:59.105504 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00625419-72f0-42bb-8828-7c1c7fe751e6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:12:59.172925 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d00bf2c-fbee-4d77-9893-80f357e289dd None None] REQUEST_METHOD: `POST` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:12:59.173294 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d00bf2c-fbee-4d77-9893-80f357e289dd None None] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:12:59.173525 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d00bf2c-fbee-4d77-9893-80f357e289dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:12:59.177697 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00625419-72f0-42bb-8828-7c1c7fe751e6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 16:12:59.179246 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00625419-72f0-42bb-8828-7c1c7fe751e6 service cinder] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 16:12:59.190955 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 1247/2459] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Aug 22 16:12:59 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 165 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 16:12:59.199095 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.auth.core [None req-8d00bf2c-fbee-4d77-9893-80f357e289dd None None] MFA Rules not processed for user `058e76354dce412cb78b2875c88f2791`. Rule list: `[]` (Enabled: `True`). {{(pid=81767) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 16:12:59.200628 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.fernet_utils [None req-8d00bf2c-fbee-4d77-9893-80f357e289dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81767) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 16:12:59.277684 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 1213/2460] 10.222.0.15 () {58 vars in 967 bytes} [Mon Aug 22 16:12:59 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 16:12:59.293888 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00625419-72f0-42bb-8828-7c1c7fe751e6 service cinder] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 16:12:59.307348 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 16:12:59.418134 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00625419-72f0-42bb-8828-7c1c7fe751e6 service cinder] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 16:12:59.423220 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93a94909-54ee-45b4-a6a5-65882432ca32 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a09d128dee9438283633e51f4834643', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4ee0413f9ef4d4fb1f95a660babe090', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 16:12:59.423860 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93a94909-54ee-45b4-a6a5-65882432ca32 service swift] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:12:59.424166 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93a94909-54ee-45b4-a6a5-65882432ca32 service swift] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:12:59.424414 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93a94909-54ee-45b4-a6a5-65882432ca32 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:12:59.433502 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93a94909-54ee-45b4-a6a5-65882432ca32 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 16:12:59.435329 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93a94909-54ee-45b4-a6a5-65882432ca32 service swift] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 16:12:59.446530 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 1248/2461] 10.222.0.15 () {60 vars in 1384 bytes} [Mon Aug 22 16:12:59 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 154 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Aug 22 16:13:00.667341 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea37ac39-a982-4043-8479-2d9cd15e552c None None] REQUEST_METHOD: `POST` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:13:00.667886 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea37ac39-a982-4043-8479-2d9cd15e552c None None] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:13:00.667886 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea37ac39-a982-4043-8479-2d9cd15e552c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:13:00.688689 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.auth.core [None req-ea37ac39-a982-4043-8479-2d9cd15e552c None None] MFA Rules not processed for user `058e76354dce412cb78b2875c88f2791`. Rule list: `[]` (Enabled: `True`). {{(pid=81767) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 16:13:00.689767 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.fernet_utils [None req-ea37ac39-a982-4043-8479-2d9cd15e552c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81767) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 16:13:00.697288 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 1214/2462] 10.222.0.15 () {58 vars in 1009 bytes} [Mon Aug 22 16:13:00 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 16:13:00.714659 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93a94909-54ee-45b4-a6a5-65882432ca32 service swift] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 16:13:00.729261 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93a94909-54ee-45b4-a6a5-65882432ca32 service swift] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 16:13:00.729261 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d935816-7acf-4fb5-8f9a-fbce69e62005 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a09d128dee9438283633e51f4834643', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4ee0413f9ef4d4fb1f95a660babe090', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 16:13:00.729799 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d935816-7acf-4fb5-8f9a-fbce69e62005 service swift] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:13:00.730083 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d935816-7acf-4fb5-8f9a-fbce69e62005 service swift] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:13:00.730341 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d935816-7acf-4fb5-8f9a-fbce69e62005 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:13:00.739700 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d935816-7acf-4fb5-8f9a-fbce69e62005 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 16:13:00.741326 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d935816-7acf-4fb5-8f9a-fbce69e62005 service swift] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 16:13:00.750834 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 1249/2463] 10.222.0.15 () {60 vars in 1384 bytes} [Mon Aug 22 16:13:00 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Aug 22 16:13:01.523955 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c03fd80f-ce2e-4b7d-ac75-d57b9f15ae91 None None] REQUEST_METHOD: `POST` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:13:01.524240 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c03fd80f-ce2e-4b7d-ac75-d57b9f15ae91 None None] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:13:01.524513 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c03fd80f-ce2e-4b7d-ac75-d57b9f15ae91 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:13:01.545378 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.auth.core [None req-c03fd80f-ce2e-4b7d-ac75-d57b9f15ae91 None None] MFA Rules not processed for user `058e76354dce412cb78b2875c88f2791`. Rule list: `[]` (Enabled: `True`). {{(pid=81767) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 16:13:01.546555 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.fernet_utils [None req-c03fd80f-ce2e-4b7d-ac75-d57b9f15ae91 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81767) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 16:13:01.554322 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 1215/2464] 10.222.0.15 () {58 vars in 967 bytes} [Mon Aug 22 16:13:01 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 16:13:01.571549 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d935816-7acf-4fb5-8f9a-fbce69e62005 service swift] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 16:13:01.577573 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d935816-7acf-4fb5-8f9a-fbce69e62005 service swift] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 16:13:01.582575 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fc6d6df-1bb5-4b99-8d04-8508e21c2b9a service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a09d128dee9438283633e51f4834643', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4ee0413f9ef4d4fb1f95a660babe090', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 16:13:01.583364 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fc6d6df-1bb5-4b99-8d04-8508e21c2b9a service swift] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:13:01.583641 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fc6d6df-1bb5-4b99-8d04-8508e21c2b9a service swift] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:13:01.583911 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fc6d6df-1bb5-4b99-8d04-8508e21c2b9a service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:13:01.592302 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fc6d6df-1bb5-4b99-8d04-8508e21c2b9a service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 16:13:01.593905 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fc6d6df-1bb5-4b99-8d04-8508e21c2b9a service swift] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 16:13:01.602698 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 1250/2465] 10.222.0.15 () {60 vars in 1384 bytes} [Mon Aug 22 16:13:01 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 22 16:13:07.687912 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c03fd80f-ce2e-4b7d-ac75-d57b9f15ae91 None None] Authenticating user token {{(pid=81767) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 16:13:07.697051 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.fernet_utils [None req-c03fd80f-ce2e-4b7d-ac75-d57b9f15ae91 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81767) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 16:13:07.791378 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c03fd80f-ce2e-4b7d-ac75-d57b9f15ae91 None None] Validating token access rules against request {{(pid=81767) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 16:13:07.796118 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60b03f2c-ef66-431b-a164-f98c9d2819f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a98b1e4431940ab8d1fc95df2858a6c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81767) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 16:13:07.796783 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b03f2c-ef66-431b-a164-f98c9d2819f6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:13:07.797060 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b03f2c-ef66-431b-a164-f98c9d2819f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:13:07.797330 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b03f2c-ef66-431b-a164-f98c9d2819f6 admin admin] PATH_INFO: `/v3/users/04529e2a334542fcaa4b64a26ae74ed7` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:13:07.798994 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60b03f2c-ef66-431b-a164-f98c9d2819f6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=04529e2a334542fcaa4b64a26ae74ed7)` {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 16:13:07.800951 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60b03f2c-ef66-431b-a164-f98c9d2819f6 admin admin] RBAC: Authorization granted {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 16:13:08.024986 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.notifications [None req-60b03f2c-ef66-431b-a164-f98c9d2819f6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '04529e2a334542fcaa4b64a26ae74ed7'} {{(pid=81767) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 16:13:08.025666 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.notifications [None req-60b03f2c-ef66-431b-a164-f98c9d2819f6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '04529e2a334542fcaa4b64a26ae74ed7'} {{(pid=81767) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 16:13:08.069851 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fc6d6df-1bb5-4b99-8d04-8508e21c2b9a service swift] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 16:13:08.072339 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-4fc6d6df-1bb5-4b99-8d04-8508e21c2b9a service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 16:13:08.086688 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fc6d6df-1bb5-4b99-8d04-8508e21c2b9a service swift] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 16:13:08.091376 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ee7de04-cee1-4941-b0ad-2adb07ea83cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a98b1e4431940ab8d1fc95df2858a6c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 16:13:08.092002 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee7de04-cee1-4941-b0ad-2adb07ea83cf admin admin] REQUEST_METHOD: `DELETE` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:13:08.092282 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee7de04-cee1-4941-b0ad-2adb07ea83cf admin admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:13:08.092557 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee7de04-cee1-4941-b0ad-2adb07ea83cf admin admin] PATH_INFO: `/v3/users/ddb50764dc754749b67c4bdb6d156efc` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:13:08.093688 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ee7de04-cee1-4941-b0ad-2adb07ea83cf admin admin] RBAC: Authorizing `identity:delete_user(user_id=ddb50764dc754749b67c4bdb6d156efc)` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 16:13:08.096137 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ee7de04-cee1-4941-b0ad-2adb07ea83cf admin admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 16:13:08.126824 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.notifications [None req-60b03f2c-ef66-431b-a164-f98c9d2819f6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '04529e2a334542fcaa4b64a26ae74ed7'} {{(pid=81767) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 16:13:08.130936 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG dbcounter [-] [81767] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=81767) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 16:13:08.139179 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.notifications [None req-60b03f2c-ef66-431b-a164-f98c9d2819f6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '04529e2a334542fcaa4b64a26ae74ed7'} {{(pid=81767) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 16:13:08.154961 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.notifications [None req-60b03f2c-ef66-431b-a164-f98c9d2819f6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '04529e2a334542fcaa4b64a26ae74ed7'} {{(pid=81767) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 16:13:08.160953 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 1216/2466] 10.222.0.15 () {58 vars in 1224 bytes} [Mon Aug 22 16:13:07 2022] DELETE /identity/v3/users/04529e2a334542fcaa4b64a26ae74ed7 => generated 0 bytes in 474 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 22 16:13:08.165216 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57a3fa26-b56e-4881-a36c-dab0d7ccd4da None None] REQUEST_METHOD: `POST` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:13:08.165507 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57a3fa26-b56e-4881-a36c-dab0d7ccd4da None None] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:13:08.168765 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57a3fa26-b56e-4881-a36c-dab0d7ccd4da None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:13:08.187013 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.auth.core [None req-57a3fa26-b56e-4881-a36c-dab0d7ccd4da None None] MFA Rules not processed for user `6b2199ae0fb0480aa917ada391f5ec54`. Rule list: `[]` (Enabled: `True`). {{(pid=81767) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 16:13:08.188887 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.fernet_utils [None req-57a3fa26-b56e-4881-a36c-dab0d7ccd4da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81767) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 16:13:08.348713 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-5ee7de04-cee1-4941-b0ad-2adb07ea83cf admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ddb50764dc754749b67c4bdb6d156efc'} {{(pid=81768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 16:13:08.349460 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-5ee7de04-cee1-4941-b0ad-2adb07ea83cf admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ddb50764dc754749b67c4bdb6d156efc'} {{(pid=81768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 16:13:08.350124 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-5ee7de04-cee1-4941-b0ad-2adb07ea83cf admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ddb50764dc754749b67c4bdb6d156efc'} {{(pid=81768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 16:13:08.362898 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-5ee7de04-cee1-4941-b0ad-2adb07ea83cf admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ddb50764dc754749b67c4bdb6d156efc'} {{(pid=81768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 16:13:08.383896 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 1217/2467] 10.222.0.15 () {58 vars in 958 bytes} [Mon Aug 22 16:13:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 220 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 22 16:13:08.384883 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57a3fa26-b56e-4881-a36c-dab0d7ccd4da None None] Authenticating user token {{(pid=81767) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 16:13:08.386292 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.fernet_utils [None req-57a3fa26-b56e-4881-a36c-dab0d7ccd4da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81767) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 16:13:08.402446 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57a3fa26-b56e-4881-a36c-dab0d7ccd4da None None] Validating token access rules against request {{(pid=81767) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 16:13:08.407627 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9440ec2-9da1-438a-bcc1-59e610606277 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a98b1e4431940ab8d1fc95df2858a6c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81767) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 16:13:08.408285 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9440ec2-9da1-438a-bcc1-59e610606277 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:13:08.408558 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9440ec2-9da1-438a-bcc1-59e610606277 admin admin] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:13:08.408850 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9440ec2-9da1-438a-bcc1-59e610606277 admin admin] PATH_INFO: `/v3/users/344a1e1a3d234730bb3974282c27fefb` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:13:08.409986 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9440ec2-9da1-438a-bcc1-59e610606277 admin admin] RBAC: Authorizing `identity:delete_user(user_id=344a1e1a3d234730bb3974282c27fefb)` {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 16:13:08.411916 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9440ec2-9da1-438a-bcc1-59e610606277 admin admin] RBAC: Authorization granted {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 16:13:08.451842 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-5ee7de04-cee1-4941-b0ad-2adb07ea83cf admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ddb50764dc754749b67c4bdb6d156efc'} {{(pid=81768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 16:13:08.474216 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 1251/2468] 10.222.0.15 () {58 vars in 1224 bytes} [Mon Aug 22 16:13:08 2022] DELETE /identity/v3/users/ddb50764dc754749b67c4bdb6d156efc => generated 0 bytes in 405 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 22 16:13:08.475371 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ee7de04-cee1-4941-b0ad-2adb07ea83cf admin admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 16:13:08.477085 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-5ee7de04-cee1-4941-b0ad-2adb07ea83cf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 16:13:08.534393 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 16:13:08.539431 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ee7de04-cee1-4941-b0ad-2adb07ea83cf admin admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 16:13:08.543265 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f780399e-9699-44a3-8d47-c13aa3fce8d3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '234687adad594f1aa45775a565ffbdc7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4ee0413f9ef4d4fb1f95a660babe090', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 16:13:08.544040 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f780399e-9699-44a3-8d47-c13aa3fce8d3 service neutron] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:13:08.544217 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f780399e-9699-44a3-8d47-c13aa3fce8d3 service neutron] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:13:08.549836 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f780399e-9699-44a3-8d47-c13aa3fce8d3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:13:08.608964 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f780399e-9699-44a3-8d47-c13aa3fce8d3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 16:13:08.611095 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f780399e-9699-44a3-8d47-c13aa3fce8d3 service neutron] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 16:13:08.621272 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 1252/2469] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Aug 22 16:13:08 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 146 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 16:13:08.622632 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71585050-5003-4c3e-9396-bd96ec271bea None None] REQUEST_METHOD: `POST` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:13:08.622921 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71585050-5003-4c3e-9396-bd96ec271bea None None] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:13:08.623197 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71585050-5003-4c3e-9396-bd96ec271bea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:13:08.643729 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.auth.core [None req-71585050-5003-4c3e-9396-bd96ec271bea None None] MFA Rules not processed for user `6b2199ae0fb0480aa917ada391f5ec54`. Rule list: `[]` (Enabled: `True`). {{(pid=81768) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 16:13:08.644957 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-71585050-5003-4c3e-9396-bd96ec271bea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 16:13:08.653425 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 1253/2470] 10.222.0.15 () {58 vars in 958 bytes} [Mon Aug 22 16:13:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 16:13:08.663692 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71585050-5003-4c3e-9396-bd96ec271bea None None] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 16:13:08.665394 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-71585050-5003-4c3e-9396-bd96ec271bea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 16:13:08.679720 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.notifications [None req-f9440ec2-9da1-438a-bcc1-59e610606277 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '344a1e1a3d234730bb3974282c27fefb'} {{(pid=81767) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 16:13:08.680409 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.notifications [None req-f9440ec2-9da1-438a-bcc1-59e610606277 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '344a1e1a3d234730bb3974282c27fefb'} {{(pid=81767) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 16:13:08.682727 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71585050-5003-4c3e-9396-bd96ec271bea None None] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 16:13:08.686617 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e588b756-5f7d-41f4-80c6-e72157c958c0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '234687adad594f1aa45775a565ffbdc7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4ee0413f9ef4d4fb1f95a660babe090', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 16:13:08.687227 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e588b756-5f7d-41f4-80c6-e72157c958c0 service neutron] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:13:08.687603 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e588b756-5f7d-41f4-80c6-e72157c958c0 service neutron] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:13:08.687877 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e588b756-5f7d-41f4-80c6-e72157c958c0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:13:08.690467 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-e588b756-5f7d-41f4-80c6-e72157c958c0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 16:13:08.703656 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e588b756-5f7d-41f4-80c6-e72157c958c0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 16:13:08.705303 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e588b756-5f7d-41f4-80c6-e72157c958c0 service neutron] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 16:13:08.717299 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 1254/2471] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Aug 22 16:13:08 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 54 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 22 16:13:08.821486 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.notifications [None req-f9440ec2-9da1-438a-bcc1-59e610606277 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '344a1e1a3d234730bb3974282c27fefb'} {{(pid=81767) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 16:13:08.842953 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.notifications [None req-f9440ec2-9da1-438a-bcc1-59e610606277 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '344a1e1a3d234730bb3974282c27fefb'} {{(pid=81767) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 16:13:08.874144 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.notifications [None req-f9440ec2-9da1-438a-bcc1-59e610606277 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '344a1e1a3d234730bb3974282c27fefb'} {{(pid=81767) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 16:13:08.885730 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 1218/2472] 10.222.0.15 () {58 vars in 1224 bytes} [Mon Aug 22 16:13:08 2022] DELETE /identity/v3/users/344a1e1a3d234730bb3974282c27fefb => generated 0 bytes in 501 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 16:13:08.890545 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e04093-e931-4d1c-9a71-db4bb1252561 None None] REQUEST_METHOD: `POST` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:13:08.890849 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e04093-e931-4d1c-9a71-db4bb1252561 None None] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:13:08.891151 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e04093-e931-4d1c-9a71-db4bb1252561 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:13:08.923923 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.auth.core [None req-d4e04093-e931-4d1c-9a71-db4bb1252561 None None] MFA Rules not processed for user `6b2199ae0fb0480aa917ada391f5ec54`. Rule list: `[]` (Enabled: `True`). {{(pid=81768) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 16:13:08.923923 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-d4e04093-e931-4d1c-9a71-db4bb1252561 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 16:13:09.033142 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 1255/2473] 10.222.0.15 () {58 vars in 958 bytes} [Mon Aug 22 16:13:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 22 16:13:09.061993 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9440ec2-9da1-438a-bcc1-59e610606277 admin admin] Authenticating user token {{(pid=81767) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 16:13:09.066353 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.fernet_utils [None req-f9440ec2-9da1-438a-bcc1-59e610606277 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81767) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 16:13:09.126522 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG dbcounter [-] [81767] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=81767) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 16:13:09.132306 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9440ec2-9da1-438a-bcc1-59e610606277 admin admin] Validating token access rules against request {{(pid=81767) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 16:13:09.142435 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de9b0228-f726-4c28-80f9-59324ae853ad service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '234687adad594f1aa45775a565ffbdc7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4ee0413f9ef4d4fb1f95a660babe090', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81767) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 16:13:09.143219 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de9b0228-f726-4c28-80f9-59324ae853ad service neutron] REQUEST_METHOD: `GET` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:13:09.143554 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de9b0228-f726-4c28-80f9-59324ae853ad service neutron] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:13:09.144766 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de9b0228-f726-4c28-80f9-59324ae853ad service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:13:09.155882 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de9b0228-f726-4c28-80f9-59324ae853ad service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 16:13:09.158373 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de9b0228-f726-4c28-80f9-59324ae853ad service neutron] RBAC: Authorization granted {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 16:13:09.170623 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 1219/2474] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Aug 22 16:13:09 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 16:13:09.194630 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4e04093-e931-4d1c-9a71-db4bb1252561 None None] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 16:13:09.196574 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-d4e04093-e931-4d1c-9a71-db4bb1252561 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 16:13:09.214936 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4e04093-e931-4d1c-9a71-db4bb1252561 None None] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 16:13:09.221631 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee627f6b-2461-410f-99ac-93bd91025463 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a98b1e4431940ab8d1fc95df2858a6c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 16:13:09.222492 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee627f6b-2461-410f-99ac-93bd91025463 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:13:09.222922 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee627f6b-2461-410f-99ac-93bd91025463 admin admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:13:09.223390 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee627f6b-2461-410f-99ac-93bd91025463 admin admin] PATH_INFO: `/v3/projects/581d22bece1544e5addcc7f7205093ec` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:13:09.225128 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee627f6b-2461-410f-99ac-93bd91025463 admin admin] RBAC: Authorizing `identity:delete_project(project_id=581d22bece1544e5addcc7f7205093ec)` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 16:13:09.229702 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee627f6b-2461-410f-99ac-93bd91025463 admin admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 16:13:09.279988 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de9b0228-f726-4c28-80f9-59324ae853ad service neutron] Authenticating user token {{(pid=81767) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 16:13:09.281574 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.fernet_utils [None req-de9b0228-f726-4c28-80f9-59324ae853ad service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81767) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 16:13:09.309761 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de9b0228-f726-4c28-80f9-59324ae853ad service neutron] Validating token access rules against request {{(pid=81767) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 16:13:09.316899 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-147dafe1-629e-4808-841a-69d9de6e757c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a98b1e4431940ab8d1fc95df2858a6c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81767) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 16:13:09.317691 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-147dafe1-629e-4808-841a-69d9de6e757c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:13:09.318020 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-147dafe1-629e-4808-841a-69d9de6e757c admin admin] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:13:09.322049 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-147dafe1-629e-4808-841a-69d9de6e757c admin admin] PATH_INFO: `/v3/projects/6902a92f7a29499cb5315eb1ef9dee79` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:13:09.322049 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-147dafe1-629e-4808-841a-69d9de6e757c admin admin] RBAC: Authorizing `identity:delete_project(project_id=6902a92f7a29499cb5315eb1ef9dee79)` {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 16:13:09.322804 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-147dafe1-629e-4808-841a-69d9de6e757c admin admin] RBAC: Authorization granted {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 16:13:09.367488 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-ee627f6b-2461-410f-99ac-93bd91025463 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '581d22bece1544e5addcc7f7205093ec', 'request_id': 'req-ee627f6b-2461-410f-99ac-93bd91025463'} {{(pid=81768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 16:13:09.375610 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-ee627f6b-2461-410f-99ac-93bd91025463 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '581d22bece1544e5addcc7f7205093ec', 'request_id': 'req-ee627f6b-2461-410f-99ac-93bd91025463'} {{(pid=81768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 16:13:09.461366 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-ee627f6b-2461-410f-99ac-93bd91025463 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '581d22bece1544e5addcc7f7205093ec', 'request_id': 'req-ee627f6b-2461-410f-99ac-93bd91025463'} {{(pid=81768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 16:13:09.477252 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 16:13:09.481817 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-ee627f6b-2461-410f-99ac-93bd91025463 admin admin] The token cache is being invalidate because project 581d22bece1544e5addcc7f7205093ec was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81768) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 22 16:13:09.482223 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-ee627f6b-2461-410f-99ac-93bd91025463 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 16:13:09.482916 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-ee627f6b-2461-410f-99ac-93bd91025463 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 16:13:09.484979 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 1256/2475] 10.222.0.15 () {58 vars in 1233 bytes} [Mon Aug 22 16:13:09 2022] DELETE /identity/v3/projects/581d22bece1544e5addcc7f7205093ec => generated 0 bytes in 292 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 16:13:09.489956 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca256f24-b3a1-4a00-a55c-9747d1faf84e None None] REQUEST_METHOD: `POST` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:13:09.490774 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca256f24-b3a1-4a00-a55c-9747d1faf84e None None] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:13:09.491078 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca256f24-b3a1-4a00-a55c-9747d1faf84e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:13:09.497088 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.notifications [None req-147dafe1-629e-4808-841a-69d9de6e757c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6902a92f7a29499cb5315eb1ef9dee79', 'request_id': 'req-147dafe1-629e-4808-841a-69d9de6e757c'} {{(pid=81767) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 16:13:09.503815 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.notifications [None req-147dafe1-629e-4808-841a-69d9de6e757c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6902a92f7a29499cb5315eb1ef9dee79', 'request_id': 'req-147dafe1-629e-4808-841a-69d9de6e757c'} {{(pid=81767) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 16:13:09.517897 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.auth.core [None req-ca256f24-b3a1-4a00-a55c-9747d1faf84e None None] MFA Rules not processed for user `6b2199ae0fb0480aa917ada391f5ec54`. Rule list: `[]` (Enabled: `True`). {{(pid=81768) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 16:13:09.519750 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-ca256f24-b3a1-4a00-a55c-9747d1faf84e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 16:13:09.594362 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.notifications [None req-147dafe1-629e-4808-841a-69d9de6e757c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6902a92f7a29499cb5315eb1ef9dee79', 'request_id': 'req-147dafe1-629e-4808-841a-69d9de6e757c'} {{(pid=81767) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 16:13:09.610933 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 1257/2476] 10.222.0.15 () {58 vars in 958 bytes} [Mon Aug 22 16:13:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 16:13:09.612800 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.notifications [None req-147dafe1-629e-4808-841a-69d9de6e757c admin admin] The token cache is being invalidate because project 6902a92f7a29499cb5315eb1ef9dee79 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81767) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 22 16:13:09.613095 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.notifications [None req-147dafe1-629e-4808-841a-69d9de6e757c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81767) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 16:13:09.613795 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.notifications [None req-147dafe1-629e-4808-841a-69d9de6e757c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81767) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 16:13:09.615580 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 1220/2477] 10.222.0.15 () {58 vars in 1233 bytes} [Mon Aug 22 16:13:09 2022] DELETE /identity/v3/projects/6902a92f7a29499cb5315eb1ef9dee79 => generated 0 bytes in 337 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 16:13:09.617922 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0421e918-d9cc-4eb4-8e19-72b1e286b063 None None] REQUEST_METHOD: `POST` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:13:09.618439 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0421e918-d9cc-4eb4-8e19-72b1e286b063 None None] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:13:09.618701 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0421e918-d9cc-4eb4-8e19-72b1e286b063 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:13:09.622161 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-147dafe1-629e-4808-841a-69d9de6e757c admin admin] Authenticating user token {{(pid=81767) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 16:13:09.623812 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.fernet_utils [None req-147dafe1-629e-4808-841a-69d9de6e757c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81767) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 16:13:09.643071 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.auth.core [None req-0421e918-d9cc-4eb4-8e19-72b1e286b063 None None] MFA Rules not processed for user `6b2199ae0fb0480aa917ada391f5ec54`. Rule list: `[]` (Enabled: `True`). {{(pid=81768) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 16:13:09.644406 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-0421e918-d9cc-4eb4-8e19-72b1e286b063 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 16:13:09.650898 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-147dafe1-629e-4808-841a-69d9de6e757c admin admin] Validating token access rules against request {{(pid=81767) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 16:13:09.658072 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 1258/2478] 10.222.0.15 () {58 vars in 958 bytes} [Mon Aug 22 16:13:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 16:13:09.658242 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-396c886f-7fa2-45d1-a97b-cf16d6aa68af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a98b1e4431940ab8d1fc95df2858a6c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81767) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 16:13:09.658242 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-396c886f-7fa2-45d1-a97b-cf16d6aa68af admin admin] REQUEST_METHOD: `DELETE` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:13:09.658242 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-396c886f-7fa2-45d1-a97b-cf16d6aa68af admin admin] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:13:09.658531 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-396c886f-7fa2-45d1-a97b-cf16d6aa68af admin admin] PATH_INFO: `/v3/users/8efc4f0227b6447696ce159f946d0428` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:13:09.659853 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0421e918-d9cc-4eb4-8e19-72b1e286b063 None None] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 16:13:09.660431 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-396c886f-7fa2-45d1-a97b-cf16d6aa68af admin admin] RBAC: Authorizing `identity:delete_user(user_id=8efc4f0227b6447696ce159f946d0428)` {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 16:13:09.663159 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-396c886f-7fa2-45d1-a97b-cf16d6aa68af admin admin] RBAC: Authorization granted {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 16:13:09.669954 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0421e918-d9cc-4eb4-8e19-72b1e286b063 None None] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 16:13:09.675974 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56636767-02f4-4e1f-9c50-800e19bece53 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a98b1e4431940ab8d1fc95df2858a6c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 16:13:09.676589 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56636767-02f4-4e1f-9c50-800e19bece53 admin admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:13:09.677174 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56636767-02f4-4e1f-9c50-800e19bece53 admin admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:13:09.677405 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56636767-02f4-4e1f-9c50-800e19bece53 admin admin] PATH_INFO: `/v3/domains` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:13:09.678306 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56636767-02f4-4e1f-9c50-800e19bece53 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81768) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 16:13:09.678614 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56636767-02f4-4e1f-9c50-800e19bece53 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 16:13:09.680252 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56636767-02f4-4e1f-9c50-800e19bece53 admin admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 16:13:09.698996 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 1259/2479] 10.222.0.15 () {58 vars in 1153 bytes} [Mon Aug 22 16:13:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 16:13:09.706187 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56636767-02f4-4e1f-9c50-800e19bece53 admin admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 16:13:09.715589 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56636767-02f4-4e1f-9c50-800e19bece53 admin admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 16:13:09.720600 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aca1b84-0584-4d44-99de-06941a2de16d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a98b1e4431940ab8d1fc95df2858a6c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 16:13:09.721271 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aca1b84-0584-4d44-99de-06941a2de16d admin admin] REQUEST_METHOD: `POST` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:13:09.721582 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aca1b84-0584-4d44-99de-06941a2de16d admin admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:13:09.721900 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aca1b84-0584-4d44-99de-06941a2de16d admin admin] PATH_INFO: `/v3/projects` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:13:09.722513 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aca1b84-0584-4d44-99de-06941a2de16d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 16:13:09.724504 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aca1b84-0584-4d44-99de-06941a2de16d admin admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 16:13:09.842784 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 1260/2480] 10.222.0.15 () {60 vars in 1153 bytes} [Mon Aug 22 16:13:09 2022] POST /identity/v3/projects => generated 381 bytes in 139 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 22 16:13:09.842784 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aca1b84-0584-4d44-99de-06941a2de16d admin admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 16:13:09.853783 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-8aca1b84-0584-4d44-99de-06941a2de16d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 16:13:09.890845 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.notifications [None req-396c886f-7fa2-45d1-a97b-cf16d6aa68af admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8efc4f0227b6447696ce159f946d0428'} {{(pid=81767) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 16:13:09.892200 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.notifications [None req-396c886f-7fa2-45d1-a97b-cf16d6aa68af admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8efc4f0227b6447696ce159f946d0428'} {{(pid=81767) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 16:13:09.947161 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aca1b84-0584-4d44-99de-06941a2de16d admin admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 16:13:09.953153 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-286060c3-08ec-4273-b993-67ea62a94432 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a98b1e4431940ab8d1fc95df2858a6c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 16:13:09.954694 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-286060c3-08ec-4273-b993-67ea62a94432 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:13:09.955043 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-286060c3-08ec-4273-b993-67ea62a94432 admin admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:13:09.955408 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-286060c3-08ec-4273-b993-67ea62a94432 admin admin] PATH_INFO: `/v3/users/514a45348e12415abaee3d2e29406689` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:13:09.956827 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-286060c3-08ec-4273-b993-67ea62a94432 admin admin] RBAC: Authorizing `identity:delete_user(user_id=514a45348e12415abaee3d2e29406689)` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 16:13:09.958832 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-286060c3-08ec-4273-b993-67ea62a94432 admin admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 16:13:09.975941 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.notifications [None req-396c886f-7fa2-45d1-a97b-cf16d6aa68af admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8efc4f0227b6447696ce159f946d0428'} {{(pid=81767) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 16:13:09.991882 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.notifications [None req-396c886f-7fa2-45d1-a97b-cf16d6aa68af admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8efc4f0227b6447696ce159f946d0428'} {{(pid=81767) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 16:13:10.014216 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.notifications [None req-396c886f-7fa2-45d1-a97b-cf16d6aa68af admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8efc4f0227b6447696ce159f946d0428'} {{(pid=81767) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 16:13:10.023568 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 1221/2481] 10.222.0.15 () {58 vars in 1224 bytes} [Mon Aug 22 16:13:09 2022] DELETE /identity/v3/users/8efc4f0227b6447696ce159f946d0428 => generated 0 bytes in 402 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 16:13:10.024396 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-396c886f-7fa2-45d1-a97b-cf16d6aa68af admin admin] Authenticating user token {{(pid=81767) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 16:13:10.025733 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.fernet_utils [None req-396c886f-7fa2-45d1-a97b-cf16d6aa68af admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81767) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 16:13:10.077205 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG dbcounter [-] [81767] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=81767) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 16:13:10.102366 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-396c886f-7fa2-45d1-a97b-cf16d6aa68af admin admin] Validating token access rules against request {{(pid=81767) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 16:13:10.110457 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb3e5efb-392a-4872-821c-fbcd00badab1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a98b1e4431940ab8d1fc95df2858a6c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81767) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 16:13:10.111104 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb3e5efb-392a-4872-821c-fbcd00badab1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:13:10.111412 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb3e5efb-392a-4872-821c-fbcd00badab1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:13:10.111677 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb3e5efb-392a-4872-821c-fbcd00badab1 admin admin] PATH_INFO: `/v3/projects/c7c13dfa47ae41439398559bbdfe591a` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:13:10.112845 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb3e5efb-392a-4872-821c-fbcd00badab1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c7c13dfa47ae41439398559bbdfe591a)` {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 16:13:10.114838 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb3e5efb-392a-4872-821c-fbcd00badab1 admin admin] RBAC: Authorization granted {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 16:13:10.195233 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-286060c3-08ec-4273-b993-67ea62a94432 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '514a45348e12415abaee3d2e29406689'} {{(pid=81768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 16:13:10.195906 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-286060c3-08ec-4273-b993-67ea62a94432 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '514a45348e12415abaee3d2e29406689'} {{(pid=81768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 16:13:10.196503 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-286060c3-08ec-4273-b993-67ea62a94432 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '514a45348e12415abaee3d2e29406689'} {{(pid=81768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 16:13:10.203395 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-286060c3-08ec-4273-b993-67ea62a94432 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '514a45348e12415abaee3d2e29406689'} {{(pid=81768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 16:13:10.207227 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=90,keystone:INSERT=2,keystone:DELETE=8 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 16:13:10.267788 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.notifications [None req-cb3e5efb-392a-4872-821c-fbcd00badab1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c7c13dfa47ae41439398559bbdfe591a', 'request_id': 'req-cb3e5efb-392a-4872-821c-fbcd00badab1'} {{(pid=81767) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 16:13:10.273340 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.notifications [None req-cb3e5efb-392a-4872-821c-fbcd00badab1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c7c13dfa47ae41439398559bbdfe591a', 'request_id': 'req-cb3e5efb-392a-4872-821c-fbcd00badab1'} {{(pid=81767) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 16:13:10.283871 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-286060c3-08ec-4273-b993-67ea62a94432 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '514a45348e12415abaee3d2e29406689'} {{(pid=81768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 16:13:10.309622 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 1261/2482] 10.222.0.15 () {58 vars in 1224 bytes} [Mon Aug 22 16:13:09 2022] DELETE /identity/v3/users/514a45348e12415abaee3d2e29406689 => generated 0 bytes in 470 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 16:13:10.312987 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-286060c3-08ec-4273-b993-67ea62a94432 admin admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 16:13:10.312987 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-286060c3-08ec-4273-b993-67ea62a94432 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 16:13:10.387393 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-286060c3-08ec-4273-b993-67ea62a94432 admin admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 16:13:10.392976 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86ca434e-1d52-4db6-a6ca-9bc7032b596f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a98b1e4431940ab8d1fc95df2858a6c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 16:13:10.393706 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86ca434e-1d52-4db6-a6ca-9bc7032b596f admin admin] REQUEST_METHOD: `POST` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:13:10.393989 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86ca434e-1d52-4db6-a6ca-9bc7032b596f admin admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:13:10.394302 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86ca434e-1d52-4db6-a6ca-9bc7032b596f admin admin] PATH_INFO: `/v3/users` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:13:10.394928 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86ca434e-1d52-4db6-a6ca-9bc7032b596f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 16:13:10.397095 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86ca434e-1d52-4db6-a6ca-9bc7032b596f admin admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 16:13:10.421022 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.notifications [None req-cb3e5efb-392a-4872-821c-fbcd00badab1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c7c13dfa47ae41439398559bbdfe591a', 'request_id': 'req-cb3e5efb-392a-4872-821c-fbcd00badab1'} {{(pid=81767) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 16:13:10.435513 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.notifications [None req-cb3e5efb-392a-4872-821c-fbcd00badab1 admin admin] The token cache is being invalidate because project c7c13dfa47ae41439398559bbdfe591a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81767) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 22 16:13:10.435811 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.notifications [None req-cb3e5efb-392a-4872-821c-fbcd00badab1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81767) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 16:13:10.436491 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.notifications [None req-cb3e5efb-392a-4872-821c-fbcd00badab1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81767) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 16:13:10.438535 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 1222/2483] 10.222.0.15 () {58 vars in 1233 bytes} [Mon Aug 22 16:13:10 2022] DELETE /identity/v3/projects/c7c13dfa47ae41439398559bbdfe591a => generated 0 bytes in 414 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 16:13:10.439336 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb3e5efb-392a-4872-821c-fbcd00badab1 admin admin] Authenticating user token {{(pid=81767) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 16:13:10.440630 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.fernet_utils [None req-cb3e5efb-392a-4872-821c-fbcd00badab1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81767) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 16:13:10.507858 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb3e5efb-392a-4872-821c-fbcd00badab1 admin admin] Validating token access rules against request {{(pid=81767) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 16:13:10.512859 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8d0e6a3-2328-476f-af40-b2e90bd317bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a98b1e4431940ab8d1fc95df2858a6c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81767) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 16:13:10.513555 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8d0e6a3-2328-476f-af40-b2e90bd317bb admin admin] REQUEST_METHOD: `DELETE` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:13:10.513810 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8d0e6a3-2328-476f-af40-b2e90bd317bb admin admin] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:13:10.514083 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8d0e6a3-2328-476f-af40-b2e90bd317bb admin admin] PATH_INFO: `/v3/users/3922ae7a8fb44e11bc3d83a57feb5ea0` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:13:10.515248 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8d0e6a3-2328-476f-af40-b2e90bd317bb admin admin] RBAC: Authorizing `identity:delete_user(user_id=3922ae7a8fb44e11bc3d83a57feb5ea0)` {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 16:13:10.518854 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 1262/2484] 10.222.0.15 () {60 vars in 1144 bytes} [Mon Aug 22 16:13:10 2022] POST /identity/v3/users => generated 291 bytes in 206 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 22 16:13:10.518983 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8d0e6a3-2328-476f-af40-b2e90bd317bb admin admin] RBAC: Authorization granted {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 16:13:10.519065 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cfdfa85-39ef-4ed0-9456-d98d57bd3598 None None] REQUEST_METHOD: `POST` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:13:10.519065 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cfdfa85-39ef-4ed0-9456-d98d57bd3598 None None] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:13:10.519065 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cfdfa85-39ef-4ed0-9456-d98d57bd3598 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:13:10.542279 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.auth.core [None req-3cfdfa85-39ef-4ed0-9456-d98d57bd3598 None None] MFA Rules not processed for user `6b2199ae0fb0480aa917ada391f5ec54`. Rule list: `[]` (Enabled: `True`). {{(pid=81768) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 16:13:10.543672 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-3cfdfa85-39ef-4ed0-9456-d98d57bd3598 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 16:13:10.553294 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 1263/2485] 10.222.0.15 () {58 vars in 958 bytes} [Mon Aug 22 16:13:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 16:13:10.554054 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cfdfa85-39ef-4ed0-9456-d98d57bd3598 None None] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 16:13:10.557816 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-3cfdfa85-39ef-4ed0-9456-d98d57bd3598 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 16:13:10.573301 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cfdfa85-39ef-4ed0-9456-d98d57bd3598 None None] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 16:13:10.575659 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e91afe0-dd1d-40b4-a9d4-32b954459dc8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a98b1e4431940ab8d1fc95df2858a6c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 16:13:10.576296 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e91afe0-dd1d-40b4-a9d4-32b954459dc8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:13:10.579299 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e91afe0-dd1d-40b4-a9d4-32b954459dc8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:13:10.579299 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e91afe0-dd1d-40b4-a9d4-32b954459dc8 admin admin] PATH_INFO: `/v3/users/c2648c3da398449bb38f2c578faa8266` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:13:10.579299 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e91afe0-dd1d-40b4-a9d4-32b954459dc8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c2648c3da398449bb38f2c578faa8266)` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 16:13:10.580565 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e91afe0-dd1d-40b4-a9d4-32b954459dc8 admin admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 16:13:10.763851 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.notifications [None req-a8d0e6a3-2328-476f-af40-b2e90bd317bb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3922ae7a8fb44e11bc3d83a57feb5ea0'} {{(pid=81767) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 16:13:10.764592 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.notifications [None req-a8d0e6a3-2328-476f-af40-b2e90bd317bb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3922ae7a8fb44e11bc3d83a57feb5ea0'} {{(pid=81767) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 16:13:10.815854 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-2e91afe0-dd1d-40b4-a9d4-32b954459dc8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c2648c3da398449bb38f2c578faa8266'} {{(pid=81768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 16:13:10.816411 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-2e91afe0-dd1d-40b4-a9d4-32b954459dc8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c2648c3da398449bb38f2c578faa8266'} {{(pid=81768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 16:13:10.817010 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-2e91afe0-dd1d-40b4-a9d4-32b954459dc8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c2648c3da398449bb38f2c578faa8266'} {{(pid=81768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 16:13:10.823887 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-2e91afe0-dd1d-40b4-a9d4-32b954459dc8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c2648c3da398449bb38f2c578faa8266'} {{(pid=81768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 16:13:10.828916 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.notifications [None req-a8d0e6a3-2328-476f-af40-b2e90bd317bb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3922ae7a8fb44e11bc3d83a57feb5ea0'} {{(pid=81767) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 16:13:10.836786 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.notifications [None req-a8d0e6a3-2328-476f-af40-b2e90bd317bb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3922ae7a8fb44e11bc3d83a57feb5ea0'} {{(pid=81767) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 16:13:10.847531 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG dbcounter [-] [81767] Writing DB stats keystone:SELECT=82,keystone:DELETE=16,keystone:INSERT=2 {{(pid=81767) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 16:13:10.849912 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.notifications [None req-a8d0e6a3-2328-476f-af40-b2e90bd317bb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3922ae7a8fb44e11bc3d83a57feb5ea0'} {{(pid=81767) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 16:13:10.856449 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 1223/2486] 10.222.0.15 () {58 vars in 1224 bytes} [Mon Aug 22 16:13:10 2022] DELETE /identity/v3/users/3922ae7a8fb44e11bc3d83a57feb5ea0 => generated 0 bytes in 417 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 16:13:10.857291 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8d0e6a3-2328-476f-af40-b2e90bd317bb admin admin] Authenticating user token {{(pid=81767) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 16:13:10.858642 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.fernet_utils [None req-a8d0e6a3-2328-476f-af40-b2e90bd317bb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81767) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 16:13:10.923268 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-2e91afe0-dd1d-40b4-a9d4-32b954459dc8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c2648c3da398449bb38f2c578faa8266'} {{(pid=81768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 16:13:10.926194 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8d0e6a3-2328-476f-af40-b2e90bd317bb admin admin] Validating token access rules against request {{(pid=81767) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 16:13:10.931843 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e07a594-131f-4370-969a-cb1dcd9d37c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a98b1e4431940ab8d1fc95df2858a6c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81767) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 16:13:10.932417 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e07a594-131f-4370-969a-cb1dcd9d37c5 admin admin] REQUEST_METHOD: `GET` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:13:10.932758 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e07a594-131f-4370-969a-cb1dcd9d37c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:13:10.933036 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e07a594-131f-4370-969a-cb1dcd9d37c5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:13:10.933682 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e07a594-131f-4370-969a-cb1dcd9d37c5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 16:13:10.935308 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e07a594-131f-4370-969a-cb1dcd9d37c5 admin admin] RBAC: Authorization granted {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 16:13:10.949898 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 1224/2488] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Aug 22 16:13:10 2022] GET /identity/v3/roles => generated 1420 bytes in 92 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 22 16:13:10.949898 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e07a594-131f-4370-969a-cb1dcd9d37c5 admin admin] Authenticating user token {{(pid=81767) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 16:13:10.950909 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 1264/2488] 10.222.0.15 () {58 vars in 1224 bytes} [Mon Aug 22 16:13:10 2022] DELETE /identity/v3/users/c2648c3da398449bb38f2c578faa8266 => generated 0 bytes in 393 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 16:13:10.952267 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e91afe0-dd1d-40b4-a9d4-32b954459dc8 admin admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 16:13:10.954129 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-2e91afe0-dd1d-40b4-a9d4-32b954459dc8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 16:13:10.962401 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.fernet_utils [None req-3e07a594-131f-4370-969a-cb1dcd9d37c5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81767) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 16:13:10.981003 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e07a594-131f-4370-969a-cb1dcd9d37c5 admin admin] Validating token access rules against request {{(pid=81767) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 16:13:10.995449 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=84,keystone:DELETE=12,keystone:INSERT=4 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 16:13:11.036276 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e91afe0-dd1d-40b4-a9d4-32b954459dc8 admin admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 16:13:11.041413 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77d3e4e6-32fa-43bc-9810-c379d921ee0e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '234687adad594f1aa45775a565ffbdc7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4ee0413f9ef4d4fb1f95a660babe090', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 16:13:11.042295 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77d3e4e6-32fa-43bc-9810-c379d921ee0e service neutron] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:13:11.043069 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77d3e4e6-32fa-43bc-9810-c379d921ee0e service neutron] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:13:11.043069 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77d3e4e6-32fa-43bc-9810-c379d921ee0e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:13:11.045455 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-77d3e4e6-32fa-43bc-9810-c379d921ee0e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 16:13:11.065958 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6791b54-4ff4-4eff-9447-80b09f34ba78 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a98b1e4431940ab8d1fc95df2858a6c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81767) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 16:13:11.066710 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6791b54-4ff4-4eff-9447-80b09f34ba78 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:13:11.067034 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6791b54-4ff4-4eff-9447-80b09f34ba78 admin admin] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:13:11.067334 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6791b54-4ff4-4eff-9447-80b09f34ba78 admin admin] PATH_INFO: `/v3/projects/27fe16fa1d8845718830fa26a9dcff7e` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:13:11.068765 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6791b54-4ff4-4eff-9447-80b09f34ba78 admin admin] RBAC: Authorizing `identity:delete_project(project_id=27fe16fa1d8845718830fa26a9dcff7e)` {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 16:13:11.071200 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6791b54-4ff4-4eff-9447-80b09f34ba78 admin admin] RBAC: Authorization granted {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 16:13:11.117045 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77d3e4e6-32fa-43bc-9810-c379d921ee0e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 16:13:11.118481 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77d3e4e6-32fa-43bc-9810-c379d921ee0e service neutron] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 16:13:11.127998 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 1265/2489] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Aug 22 16:13:10 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 176 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 16:13:11.129198 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c1097b-b08d-44ed-9f72-970f0a7501cc None None] REQUEST_METHOD: `POST` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:13:11.129483 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c1097b-b08d-44ed-9f72-970f0a7501cc None None] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:13:11.129754 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c1097b-b08d-44ed-9f72-970f0a7501cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:13:11.150371 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.auth.core [None req-d0c1097b-b08d-44ed-9f72-970f0a7501cc None None] MFA Rules not processed for user `6b2199ae0fb0480aa917ada391f5ec54`. Rule list: `[]` (Enabled: `True`). {{(pid=81768) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 16:13:11.151626 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-d0c1097b-b08d-44ed-9f72-970f0a7501cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 16:13:11.161862 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 1266/2490] 10.222.0.15 () {58 vars in 958 bytes} [Mon Aug 22 16:13:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 16:13:11.165279 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0c1097b-b08d-44ed-9f72-970f0a7501cc None None] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 16:13:11.175718 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-d0c1097b-b08d-44ed-9f72-970f0a7501cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 16:13:11.185666 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0c1097b-b08d-44ed-9f72-970f0a7501cc None None] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 16:13:11.190548 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.notifications [None req-a6791b54-4ff4-4eff-9447-80b09f34ba78 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '27fe16fa1d8845718830fa26a9dcff7e', 'request_id': 'req-a6791b54-4ff4-4eff-9447-80b09f34ba78'} {{(pid=81767) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 16:13:11.196776 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.notifications [None req-a6791b54-4ff4-4eff-9447-80b09f34ba78 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '27fe16fa1d8845718830fa26a9dcff7e', 'request_id': 'req-a6791b54-4ff4-4eff-9447-80b09f34ba78'} {{(pid=81767) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 16:13:11.259323 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-819ef008-9b0e-4c9d-b80f-28cfd8b5d694 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a98b1e4431940ab8d1fc95df2858a6c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 16:13:11.260030 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-819ef008-9b0e-4c9d-b80f-28cfd8b5d694 admin admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:13:11.260312 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-819ef008-9b0e-4c9d-b80f-28cfd8b5d694 admin admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:13:11.260582 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-819ef008-9b0e-4c9d-b80f-28cfd8b5d694 admin admin] PATH_INFO: `/v3/roles` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:13:11.261862 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-819ef008-9b0e-4c9d-b80f-28cfd8b5d694 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 16:13:11.263807 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-819ef008-9b0e-4c9d-b80f-28cfd8b5d694 admin admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 16:13:11.300710 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 1267/2491] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Aug 22 16:13:11 2022] GET /identity/v3/roles => generated 1420 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 22 16:13:11.300710 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-819ef008-9b0e-4c9d-b80f-28cfd8b5d694 admin admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 16:13:11.300710 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-819ef008-9b0e-4c9d-b80f-28cfd8b5d694 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 16:13:11.306395 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.notifications [None req-a6791b54-4ff4-4eff-9447-80b09f34ba78 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '27fe16fa1d8845718830fa26a9dcff7e', 'request_id': 'req-a6791b54-4ff4-4eff-9447-80b09f34ba78'} {{(pid=81767) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 16:13:11.332995 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.notifications [None req-a6791b54-4ff4-4eff-9447-80b09f34ba78 admin admin] The token cache is being invalidate because project 27fe16fa1d8845718830fa26a9dcff7e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81767) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 22 16:13:11.333557 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.notifications [None req-a6791b54-4ff4-4eff-9447-80b09f34ba78 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81767) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 16:13:11.334587 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.notifications [None req-a6791b54-4ff4-4eff-9447-80b09f34ba78 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81767) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 16:13:11.336856 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 1225/2492] 10.222.0.15 () {58 vars in 1233 bytes} [Mon Aug 22 16:13:10 2022] DELETE /identity/v3/projects/27fe16fa1d8845718830fa26a9dcff7e => generated 0 bytes in 387 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 16:13:11.337982 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6791b54-4ff4-4eff-9447-80b09f34ba78 admin admin] Authenticating user token {{(pid=81767) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 16:13:11.339511 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.fernet_utils [None req-a6791b54-4ff4-4eff-9447-80b09f34ba78 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81767) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 16:13:11.355906 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6791b54-4ff4-4eff-9447-80b09f34ba78 admin admin] Validating token access rules against request {{(pid=81767) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 16:13:11.364986 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfa89a3c-e029-459f-9777-4b420b88f736 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a98b1e4431940ab8d1fc95df2858a6c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81767) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 16:13:11.365969 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfa89a3c-e029-459f-9777-4b420b88f736 admin admin] REQUEST_METHOD: `PUT` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:13:11.366344 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfa89a3c-e029-459f-9777-4b420b88f736 admin admin] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:13:11.366690 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfa89a3c-e029-459f-9777-4b420b88f736 admin admin] PATH_INFO: `/v3/projects/dfe1b430e4694baa861828803211f324/users/7be15e5dbc3b4419b4ada8e7f075d6a9/roles/e6fc1ee3fb974125ba9b4fe55b12b45e` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:13:11.385147 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfa89a3c-e029-459f-9777-4b420b88f736 admin admin] RBAC: Authorizing `identity:create_grant(project_id=dfe1b430e4694baa861828803211f324, user_id=7be15e5dbc3b4419b4ada8e7f075d6a9, role_id=e6fc1ee3fb974125ba9b4fe55b12b45e)` {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 16:13:11.388087 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfa89a3c-e029-459f-9777-4b420b88f736 admin admin] RBAC: Authorization granted {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 16:13:11.391913 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG dbcounter [-] [81767] Writing DB stats keystone:DELETE=8,keystone:SELECT=90,keystone:INSERT=2 {{(pid=81767) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 16:13:11.406480 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-819ef008-9b0e-4c9d-b80f-28cfd8b5d694 admin admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 16:13:11.410900 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e56f3eb-ae30-4950-87c4-6a5bad36dfd0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '234687adad594f1aa45775a565ffbdc7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4ee0413f9ef4d4fb1f95a660babe090', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 16:13:11.411542 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e56f3eb-ae30-4950-87c4-6a5bad36dfd0 service neutron] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:13:11.411818 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e56f3eb-ae30-4950-87c4-6a5bad36dfd0 service neutron] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:13:11.412217 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e56f3eb-ae30-4950-87c4-6a5bad36dfd0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:13:11.414803 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-1e56f3eb-ae30-4950-87c4-6a5bad36dfd0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 16:13:11.429074 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e56f3eb-ae30-4950-87c4-6a5bad36dfd0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 16:13:11.431370 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e56f3eb-ae30-4950-87c4-6a5bad36dfd0 service neutron] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 16:13:11.443443 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 1268/2493] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Aug 22 16:13:11 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 146 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 16:13:11.444837 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7553b9b4-c280-44a9-af1f-f2bf3f7696b1 None None] REQUEST_METHOD: `POST` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:13:11.445120 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7553b9b4-c280-44a9-af1f-f2bf3f7696b1 None None] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:13:11.445394 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7553b9b4-c280-44a9-af1f-f2bf3f7696b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:13:11.471624 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.auth.core [None req-7553b9b4-c280-44a9-af1f-f2bf3f7696b1 None None] MFA Rules not processed for user `6b2199ae0fb0480aa917ada391f5ec54`. Rule list: `[]` (Enabled: `True`). {{(pid=81768) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 16:13:11.472871 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-7553b9b4-c280-44a9-af1f-f2bf3f7696b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 16:13:11.483036 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 1269/2494] 10.222.0.15 () {58 vars in 958 bytes} [Mon Aug 22 16:13:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 16:13:11.491531 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 1226/2495] 10.222.0.15 () {60 vars in 1483 bytes} [Mon Aug 22 16:13:11 2022] PUT /identity/v3/projects/dfe1b430e4694baa861828803211f324/users/7be15e5dbc3b4419b4ada8e7f075d6a9/roles/e6fc1ee3fb974125ba9b4fe55b12b45e => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 16:13:11.494503 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b342f77a-97fb-4c77-8778-e47af712c776 None None] REQUEST_METHOD: `POST` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:13:11.494806 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b342f77a-97fb-4c77-8778-e47af712c776 None None] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:13:11.498751 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b342f77a-97fb-4c77-8778-e47af712c776 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:13:11.500680 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fec473f-40f5-45ed-beea-3ed26c115506 None None] REQUEST_METHOD: `POST` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:13:11.500971 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fec473f-40f5-45ed-beea-3ed26c115506 None None] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:13:11.501243 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fec473f-40f5-45ed-beea-3ed26c115506 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:13:11.540639 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.auth.core [None req-9fec473f-40f5-45ed-beea-3ed26c115506 None None] MFA Rules not processed for user `6b2199ae0fb0480aa917ada391f5ec54`. Rule list: `[]` (Enabled: `True`). {{(pid=81767) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 16:13:11.540746 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.auth.core [None req-b342f77a-97fb-4c77-8778-e47af712c776 None None] MFA Rules not processed for user `6b2199ae0fb0480aa917ada391f5ec54`. Rule list: `[]` (Enabled: `True`). {{(pid=81768) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 16:13:11.540746 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-b342f77a-97fb-4c77-8778-e47af712c776 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 16:13:11.540907 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.fernet_utils [None req-9fec473f-40f5-45ed-beea-3ed26c115506 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81767) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 16:13:11.621072 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=100 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 16:13:11.642193 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 1270/2496] 10.222.0.15 () {58 vars in 958 bytes} [Mon Aug 22 16:13:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 16:13:11.643169 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b342f77a-97fb-4c77-8778-e47af712c776 None None] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 16:13:11.645111 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-b342f77a-97fb-4c77-8778-e47af712c776 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 16:13:11.645903 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 1227/2497] 10.222.0.15 () {58 vars in 958 bytes} [Mon Aug 22 16:13:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 16:13:11.646323 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fec473f-40f5-45ed-beea-3ed26c115506 None None] Authenticating user token {{(pid=81767) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 16:13:11.659579 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fec473f-40f5-45ed-beea-3ed26c115506 None None] Validating token access rules against request {{(pid=81767) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 16:13:11.665615 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b342f77a-97fb-4c77-8778-e47af712c776 None None] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 16:13:11.666280 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a19e86fa-f66a-449f-bf5f-6256f2f30f99 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a98b1e4431940ab8d1fc95df2858a6c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81767) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 16:13:11.667003 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a19e86fa-f66a-449f-bf5f-6256f2f30f99 admin admin] REQUEST_METHOD: `GET` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:13:11.667423 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a19e86fa-f66a-449f-bf5f-6256f2f30f99 admin admin] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:13:11.668269 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a19e86fa-f66a-449f-bf5f-6256f2f30f99 admin admin] PATH_INFO: `/v3/domains` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:13:11.668564 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a19e86fa-f66a-449f-bf5f-6256f2f30f99 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81767) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 16:13:11.668963 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a19e86fa-f66a-449f-bf5f-6256f2f30f99 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 16:13:11.671016 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a19e86fa-f66a-449f-bf5f-6256f2f30f99 admin admin] RBAC: Authorization granted {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 16:13:11.674232 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1a1e3a2-fb73-49d8-865f-996c057bee1e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a98b1e4431940ab8d1fc95df2858a6c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 16:13:11.675095 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1a1e3a2-fb73-49d8-865f-996c057bee1e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:13:11.675757 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1a1e3a2-fb73-49d8-865f-996c057bee1e admin admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:13:11.676213 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1a1e3a2-fb73-49d8-865f-996c057bee1e admin admin] PATH_INFO: `/v3/projects/ef2d9591bc134ba18519a0e19ede6253` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:13:11.678788 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1a1e3a2-fb73-49d8-865f-996c057bee1e admin admin] RBAC: Authorizing `identity:delete_project(project_id=ef2d9591bc134ba18519a0e19ede6253)` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 16:13:11.681323 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1a1e3a2-fb73-49d8-865f-996c057bee1e admin admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 16:13:11.699339 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 1228/2498] 10.222.0.15 () {58 vars in 1153 bytes} [Mon Aug 22 16:13:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 16:13:11.699339 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a19e86fa-f66a-449f-bf5f-6256f2f30f99 admin admin] Authenticating user token {{(pid=81767) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 16:13:11.699339 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.fernet_utils [None req-a19e86fa-f66a-449f-bf5f-6256f2f30f99 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81767) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 16:13:11.757524 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a19e86fa-f66a-449f-bf5f-6256f2f30f99 admin admin] Validating token access rules against request {{(pid=81767) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 16:13:11.761784 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe210c2d-ff8f-41dc-9687-f281040b2f94 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '234687adad594f1aa45775a565ffbdc7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4ee0413f9ef4d4fb1f95a660babe090', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81767) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 16:13:11.762386 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe210c2d-ff8f-41dc-9687-f281040b2f94 service neutron] REQUEST_METHOD: `GET` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:13:11.762686 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe210c2d-ff8f-41dc-9687-f281040b2f94 service neutron] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:13:11.762946 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe210c2d-ff8f-41dc-9687-f281040b2f94 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:13:11.772753 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe210c2d-ff8f-41dc-9687-f281040b2f94 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 16:13:11.774234 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe210c2d-ff8f-41dc-9687-f281040b2f94 service neutron] RBAC: Authorization granted {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 16:13:11.784642 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 1229/2499] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Aug 22 16:13:11 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 16:13:11.785409 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe210c2d-ff8f-41dc-9687-f281040b2f94 service neutron] Authenticating user token {{(pid=81767) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 16:13:11.794932 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe210c2d-ff8f-41dc-9687-f281040b2f94 service neutron] Validating token access rules against request {{(pid=81767) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 16:13:11.817446 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2627226b-b49b-48b6-9a8c-84272c15cc03 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a98b1e4431940ab8d1fc95df2858a6c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81767) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 16:13:11.818300 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2627226b-b49b-48b6-9a8c-84272c15cc03 admin admin] REQUEST_METHOD: `POST` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:13:11.818427 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2627226b-b49b-48b6-9a8c-84272c15cc03 admin admin] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:13:11.818629 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2627226b-b49b-48b6-9a8c-84272c15cc03 admin admin] PATH_INFO: `/v3/projects` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:13:11.819457 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2627226b-b49b-48b6-9a8c-84272c15cc03 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 16:13:11.823869 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2627226b-b49b-48b6-9a8c-84272c15cc03 admin admin] RBAC: Authorization granted {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 16:13:11.844685 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-b1a1e3a2-fb73-49d8-865f-996c057bee1e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ef2d9591bc134ba18519a0e19ede6253', 'request_id': 'req-b1a1e3a2-fb73-49d8-865f-996c057bee1e'} {{(pid=81768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 16:13:11.853967 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-b1a1e3a2-fb73-49d8-865f-996c057bee1e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ef2d9591bc134ba18519a0e19ede6253', 'request_id': 'req-b1a1e3a2-fb73-49d8-865f-996c057bee1e'} {{(pid=81768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 16:13:11.945949 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 1230/2500] 10.222.0.15 () {60 vars in 1153 bytes} [Mon Aug 22 16:13:11 2022] POST /identity/v3/projects => generated 383 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 22 16:13:11.946723 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2627226b-b49b-48b6-9a8c-84272c15cc03 admin admin] Authenticating user token {{(pid=81767) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 16:13:11.950701 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.fernet_utils [None req-2627226b-b49b-48b6-9a8c-84272c15cc03 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81767) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 16:13:11.969076 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-b1a1e3a2-fb73-49d8-865f-996c057bee1e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ef2d9591bc134ba18519a0e19ede6253', 'request_id': 'req-b1a1e3a2-fb73-49d8-865f-996c057bee1e'} {{(pid=81768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 16:13:11.983760 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-b1a1e3a2-fb73-49d8-865f-996c057bee1e admin admin] The token cache is being invalidate because project ef2d9591bc134ba18519a0e19ede6253 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81768) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 22 16:13:11.984396 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-b1a1e3a2-fb73-49d8-865f-996c057bee1e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 16:13:11.985847 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-b1a1e3a2-fb73-49d8-865f-996c057bee1e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 16:13:11.991916 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 1271/2501] 10.222.0.15 () {58 vars in 1233 bytes} [Mon Aug 22 16:13:11 2022] DELETE /identity/v3/projects/ef2d9591bc134ba18519a0e19ede6253 => generated 0 bytes in 345 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 16:13:11.992889 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1a1e3a2-fb73-49d8-865f-996c057bee1e admin admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 16:13:11.995507 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-b1a1e3a2-fb73-49d8-865f-996c057bee1e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 16:13:12.056624 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2627226b-b49b-48b6-9a8c-84272c15cc03 admin admin] Validating token access rules against request {{(pid=81767) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 16:13:12.061838 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92a576b4-a6c3-49e3-9ada-7ea64eee6784 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a98b1e4431940ab8d1fc95df2858a6c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81767) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 16:13:12.062480 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a576b4-a6c3-49e3-9ada-7ea64eee6784 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:13:12.063616 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a576b4-a6c3-49e3-9ada-7ea64eee6784 admin admin] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:13:12.063890 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a576b4-a6c3-49e3-9ada-7ea64eee6784 admin admin] PATH_INFO: `/v3/projects/9752eece61444130b0b869263cc92305` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:13:12.065070 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92a576b4-a6c3-49e3-9ada-7ea64eee6784 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9752eece61444130b0b869263cc92305)` {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 16:13:12.067016 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92a576b4-a6c3-49e3-9ada-7ea64eee6784 admin admin] RBAC: Authorization granted {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 16:13:12.097525 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1a1e3a2-fb73-49d8-865f-996c057bee1e admin admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 16:13:12.102104 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-659a8054-d1db-4182-9b2d-741f1e8bbef8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a98b1e4431940ab8d1fc95df2858a6c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 16:13:12.102770 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-659a8054-d1db-4182-9b2d-741f1e8bbef8 admin admin] REQUEST_METHOD: `POST` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:13:12.103226 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-659a8054-d1db-4182-9b2d-741f1e8bbef8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:13:12.103542 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-659a8054-d1db-4182-9b2d-741f1e8bbef8 admin admin] PATH_INFO: `/v3/users` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:13:12.104184 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-659a8054-d1db-4182-9b2d-741f1e8bbef8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 16:13:12.106135 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-659a8054-d1db-4182-9b2d-741f1e8bbef8 admin admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 16:13:12.224093 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.notifications [None req-92a576b4-a6c3-49e3-9ada-7ea64eee6784 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9752eece61444130b0b869263cc92305', 'request_id': 'req-92a576b4-a6c3-49e3-9ada-7ea64eee6784'} {{(pid=81767) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 16:13:12.229430 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.notifications [None req-92a576b4-a6c3-49e3-9ada-7ea64eee6784 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9752eece61444130b0b869263cc92305', 'request_id': 'req-92a576b4-a6c3-49e3-9ada-7ea64eee6784'} {{(pid=81767) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 16:13:12.254290 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 1272/2502] 10.222.0.15 () {60 vars in 1144 bytes} [Mon Aug 22 16:13:11 2022] POST /identity/v3/users => generated 292 bytes in 262 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 22 16:13:12.255573 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1500f2c3-b31a-453d-a7be-a22c920bbfb6 None None] REQUEST_METHOD: `POST` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:13:12.255856 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1500f2c3-b31a-453d-a7be-a22c920bbfb6 None None] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:13:12.256150 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1500f2c3-b31a-453d-a7be-a22c920bbfb6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:13:12.276117 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.auth.core [None req-1500f2c3-b31a-453d-a7be-a22c920bbfb6 None None] MFA Rules not processed for user `6b2199ae0fb0480aa917ada391f5ec54`. Rule list: `[]` (Enabled: `True`). {{(pid=81768) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 16:13:12.277476 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-1500f2c3-b31a-453d-a7be-a22c920bbfb6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 16:13:12.300580 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 16:13:12.326324 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.notifications [None req-92a576b4-a6c3-49e3-9ada-7ea64eee6784 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9752eece61444130b0b869263cc92305', 'request_id': 'req-92a576b4-a6c3-49e3-9ada-7ea64eee6784'} {{(pid=81767) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 16:13:12.342259 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.notifications [None req-92a576b4-a6c3-49e3-9ada-7ea64eee6784 admin admin] The token cache is being invalidate because project 9752eece61444130b0b869263cc92305 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81767) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 22 16:13:12.342565 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.notifications [None req-92a576b4-a6c3-49e3-9ada-7ea64eee6784 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81767) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 16:13:12.344139 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.notifications [None req-92a576b4-a6c3-49e3-9ada-7ea64eee6784 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81767) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 16:13:12.349468 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 1231/2503] 10.222.0.15 () {58 vars in 1233 bytes} [Mon Aug 22 16:13:11 2022] DELETE /identity/v3/projects/9752eece61444130b0b869263cc92305 => generated 0 bytes in 400 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 16:13:12.350589 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92a576b4-a6c3-49e3-9ada-7ea64eee6784 admin admin] Authenticating user token {{(pid=81767) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 16:13:12.352141 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 1273/2504] 10.222.0.15 () {58 vars in 958 bytes} [Mon Aug 22 16:13:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 16:13:12.352311 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.fernet_utils [None req-92a576b4-a6c3-49e3-9ada-7ea64eee6784 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81767) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 16:13:12.361855 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1500f2c3-b31a-453d-a7be-a22c920bbfb6 None None] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 16:13:12.363362 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-1500f2c3-b31a-453d-a7be-a22c920bbfb6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 16:13:12.376326 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1500f2c3-b31a-453d-a7be-a22c920bbfb6 None None] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 16:13:12.377106 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92a576b4-a6c3-49e3-9ada-7ea64eee6784 admin admin] Validating token access rules against request {{(pid=81767) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 16:13:12.381514 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbddbf1d-e3a9-4e41-a75b-227de3acd7b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a98b1e4431940ab8d1fc95df2858a6c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 16:13:12.382706 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbddbf1d-e3a9-4e41-a75b-227de3acd7b5 admin admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:13:12.382976 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbddbf1d-e3a9-4e41-a75b-227de3acd7b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:13:12.383263 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbddbf1d-e3a9-4e41-a75b-227de3acd7b5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:13:12.383940 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbddbf1d-e3a9-4e41-a75b-227de3acd7b5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 16:13:12.385438 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5480241-0303-4d08-89bf-132213ea6341 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a98b1e4431940ab8d1fc95df2858a6c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81767) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 16:13:12.385604 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbddbf1d-e3a9-4e41-a75b-227de3acd7b5 admin admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 16:13:12.386725 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5480241-0303-4d08-89bf-132213ea6341 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:13:12.386725 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5480241-0303-4d08-89bf-132213ea6341 admin admin] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:13:12.386725 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5480241-0303-4d08-89bf-132213ea6341 admin admin] PATH_INFO: `/v3/projects/d221e15c61bb49189f9380b318fb554e` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:13:12.388045 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5480241-0303-4d08-89bf-132213ea6341 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d221e15c61bb49189f9380b318fb554e)` {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 16:13:12.390076 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5480241-0303-4d08-89bf-132213ea6341 admin admin] RBAC: Authorization granted {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 16:13:12.399469 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 1274/2505] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Aug 22 16:13:12 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 22 16:13:12.400396 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbddbf1d-e3a9-4e41-a75b-227de3acd7b5 admin admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 16:13:12.402210 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-dbddbf1d-e3a9-4e41-a75b-227de3acd7b5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 16:13:12.416259 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbddbf1d-e3a9-4e41-a75b-227de3acd7b5 admin admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 16:13:12.421092 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4b9551e-083a-471f-9ad5-56655d1a90f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a98b1e4431940ab8d1fc95df2858a6c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 16:13:12.421721 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b9551e-083a-471f-9ad5-56655d1a90f5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:13:12.421994 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b9551e-083a-471f-9ad5-56655d1a90f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:13:12.422802 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b9551e-083a-471f-9ad5-56655d1a90f5 admin admin] PATH_INFO: `/v3/users/c1c56586b608403b8a063a222e07995b` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:13:12.424056 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4b9551e-083a-471f-9ad5-56655d1a90f5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c1c56586b608403b8a063a222e07995b)` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 16:13:12.426255 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4b9551e-083a-471f-9ad5-56655d1a90f5 admin admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 16:13:12.587266 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.notifications [None req-c5480241-0303-4d08-89bf-132213ea6341 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd221e15c61bb49189f9380b318fb554e', 'request_id': 'req-c5480241-0303-4d08-89bf-132213ea6341'} {{(pid=81767) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 16:13:12.593397 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.notifications [None req-c5480241-0303-4d08-89bf-132213ea6341 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd221e15c61bb49189f9380b318fb554e', 'request_id': 'req-c5480241-0303-4d08-89bf-132213ea6341'} {{(pid=81767) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 16:13:12.602014 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-f4b9551e-083a-471f-9ad5-56655d1a90f5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c1c56586b608403b8a063a222e07995b'} {{(pid=81768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 16:13:12.602949 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-f4b9551e-083a-471f-9ad5-56655d1a90f5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c1c56586b608403b8a063a222e07995b'} {{(pid=81768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 16:13:12.603719 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-f4b9551e-083a-471f-9ad5-56655d1a90f5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c1c56586b608403b8a063a222e07995b'} {{(pid=81768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 16:13:12.612190 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-f4b9551e-083a-471f-9ad5-56655d1a90f5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c1c56586b608403b8a063a222e07995b'} {{(pid=81768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 16:13:12.677567 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.notifications [None req-c5480241-0303-4d08-89bf-132213ea6341 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd221e15c61bb49189f9380b318fb554e', 'request_id': 'req-c5480241-0303-4d08-89bf-132213ea6341'} {{(pid=81767) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 16:13:12.690500 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.notifications [None req-c5480241-0303-4d08-89bf-132213ea6341 admin admin] The token cache is being invalidate because project d221e15c61bb49189f9380b318fb554e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81767) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 22 16:13:12.690846 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.notifications [None req-c5480241-0303-4d08-89bf-132213ea6341 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81767) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 16:13:12.691557 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.notifications [None req-c5480241-0303-4d08-89bf-132213ea6341 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81767) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 16:13:12.693326 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 1232/2506] 10.222.0.15 () {58 vars in 1233 bytes} [Mon Aug 22 16:13:12 2022] DELETE /identity/v3/projects/d221e15c61bb49189f9380b318fb554e => generated 0 bytes in 343 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 16:13:12.694426 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ec7725-821d-40a8-ae97-ef9f443e3853 None None] REQUEST_METHOD: `POST` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:13:12.694735 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ec7725-821d-40a8-ae97-ef9f443e3853 None None] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:13:12.695083 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ec7725-821d-40a8-ae97-ef9f443e3853 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:13:12.717150 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.auth.core [None req-97ec7725-821d-40a8-ae97-ef9f443e3853 None None] MFA Rules not processed for user `6b2199ae0fb0480aa917ada391f5ec54`. Rule list: `[]` (Enabled: `True`). {{(pid=81767) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 16:13:12.718279 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.fernet_utils [None req-97ec7725-821d-40a8-ae97-ef9f443e3853 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81767) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 16:13:12.760240 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-f4b9551e-083a-471f-9ad5-56655d1a90f5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c1c56586b608403b8a063a222e07995b'} {{(pid=81768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 16:13:12.781314 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 1275/2507] 10.222.0.15 () {58 vars in 1224 bytes} [Mon Aug 22 16:13:12 2022] DELETE /identity/v3/users/c1c56586b608403b8a063a222e07995b => generated 0 bytes in 381 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 16:13:12.782929 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 1233/2508] 10.222.0.15 () {58 vars in 958 bytes} [Mon Aug 22 16:13:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 16:13:12.783053 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4b9551e-083a-471f-9ad5-56655d1a90f5 admin admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 16:13:12.784316 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1769566-607a-4516-b55d-f666d16d8d48 None None] REQUEST_METHOD: `POST` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:13:12.784838 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1769566-607a-4516-b55d-f666d16d8d48 None None] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:13:12.785534 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1769566-607a-4516-b55d-f666d16d8d48 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:13:12.788144 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-f4b9551e-083a-471f-9ad5-56655d1a90f5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 16:13:12.810264 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.auth.core [None req-e1769566-607a-4516-b55d-f666d16d8d48 None None] MFA Rules not processed for user `6b2199ae0fb0480aa917ada391f5ec54`. Rule list: `[]` (Enabled: `True`). {{(pid=81767) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 16:13:12.811921 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.fernet_utils [None req-e1769566-607a-4516-b55d-f666d16d8d48 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81767) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 16:13:12.866773 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4b9551e-083a-471f-9ad5-56655d1a90f5 admin admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 16:13:12.877102 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1fb1053-f878-47a5-9301-e3303f5e76c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a98b1e4431940ab8d1fc95df2858a6c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 16:13:12.877859 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1fb1053-f878-47a5-9301-e3303f5e76c2 admin admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:13:12.878424 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1fb1053-f878-47a5-9301-e3303f5e76c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:13:12.878829 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1fb1053-f878-47a5-9301-e3303f5e76c2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:13:12.879632 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1fb1053-f878-47a5-9301-e3303f5e76c2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 16:13:12.881654 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1fb1053-f878-47a5-9301-e3303f5e76c2 admin admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 16:13:12.895020 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 1276/2509] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Aug 22 16:13:12 2022] GET /identity/v3/roles => generated 1420 bytes in 113 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 22 16:13:12.896237 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1fb1053-f878-47a5-9301-e3303f5e76c2 admin admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 16:13:12.902761 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 1234/2510] 10.222.0.15 () {58 vars in 958 bytes} [Mon Aug 22 16:13:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 16:13:12.905183 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1769566-607a-4516-b55d-f666d16d8d48 None None] Authenticating user token {{(pid=81767) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 16:13:12.907913 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.fernet_utils [None req-e1769566-607a-4516-b55d-f666d16d8d48 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81767) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 16:13:12.908788 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1fb1053-f878-47a5-9301-e3303f5e76c2 admin admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 16:13:12.914832 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30cb0841-e1ee-426b-932f-f238f3a7ccef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a98b1e4431940ab8d1fc95df2858a6c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 16:13:12.916199 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30cb0841-e1ee-426b-932f-f238f3a7ccef admin admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:13:12.916552 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30cb0841-e1ee-426b-932f-f238f3a7ccef admin admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:13:12.916890 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30cb0841-e1ee-426b-932f-f238f3a7ccef admin admin] PATH_INFO: `/v3/domains` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:13:12.917725 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30cb0841-e1ee-426b-932f-f238f3a7ccef admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81768) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 16:13:12.918259 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30cb0841-e1ee-426b-932f-f238f3a7ccef admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 16:13:12.921397 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30cb0841-e1ee-426b-932f-f238f3a7ccef admin admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 16:13:12.922625 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1769566-607a-4516-b55d-f666d16d8d48 None None] Validating token access rules against request {{(pid=81767) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 16:13:12.928709 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ba614c3-0fa5-49a9-a6d5-108871022bd0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a98b1e4431940ab8d1fc95df2858a6c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81767) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 16:13:12.939315 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ba614c3-0fa5-49a9-a6d5-108871022bd0 admin admin] REQUEST_METHOD: `PUT` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:13:12.939315 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ba614c3-0fa5-49a9-a6d5-108871022bd0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:13:12.939315 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ba614c3-0fa5-49a9-a6d5-108871022bd0 admin admin] PATH_INFO: `/v3/projects/956d4eb12bce4648bb34bb81bbd8f098/users/26d4cd03ad1e4be081210b68e2c6d0d8/roles/e6fc1ee3fb974125ba9b4fe55b12b45e` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:13:12.947761 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 1277/2511] 10.222.0.15 () {58 vars in 1153 bytes} [Mon Aug 22 16:13:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 16:13:12.947761 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b8ed4b8-afd5-4e36-838d-b83e64b6f366 None None] REQUEST_METHOD: `POST` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:13:12.947761 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b8ed4b8-afd5-4e36-838d-b83e64b6f366 None None] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:13:12.947761 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b8ed4b8-afd5-4e36-838d-b83e64b6f366 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:13:12.958635 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ba614c3-0fa5-49a9-a6d5-108871022bd0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=956d4eb12bce4648bb34bb81bbd8f098, user_id=26d4cd03ad1e4be081210b68e2c6d0d8, role_id=e6fc1ee3fb974125ba9b4fe55b12b45e)` {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 16:13:12.961830 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ba614c3-0fa5-49a9-a6d5-108871022bd0 admin admin] RBAC: Authorization granted {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 16:13:13.005621 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.auth.core [None req-4b8ed4b8-afd5-4e36-838d-b83e64b6f366 None None] MFA Rules not processed for user `6b2199ae0fb0480aa917ada391f5ec54`. Rule list: `[]` (Enabled: `True`). {{(pid=81768) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 16:13:13.007305 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-4b8ed4b8-afd5-4e36-838d-b83e64b6f366 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 16:13:13.019639 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 1278/2512] 10.222.0.15 () {58 vars in 958 bytes} [Mon Aug 22 16:13:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 72 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 16:13:13.019639 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b8ed4b8-afd5-4e36-838d-b83e64b6f366 None None] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 16:13:13.032874 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b8ed4b8-afd5-4e36-838d-b83e64b6f366 None None] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 16:13:13.054554 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 1235/2513] 10.222.0.15 () {60 vars in 1483 bytes} [Mon Aug 22 16:13:12 2022] PUT /identity/v3/projects/956d4eb12bce4648bb34bb81bbd8f098/users/26d4cd03ad1e4be081210b68e2c6d0d8/roles/e6fc1ee3fb974125ba9b4fe55b12b45e => generated 0 bytes in 147 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 16:13:13.054554 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ba614c3-0fa5-49a9-a6d5-108871022bd0 admin admin] Authenticating user token {{(pid=81767) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 16:13:13.160515 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d07373e-fa0c-4172-9f20-c068596b76c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a98b1e4431940ab8d1fc95df2858a6c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 16:13:13.161195 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d07373e-fa0c-4172-9f20-c068596b76c0 admin admin] REQUEST_METHOD: `POST` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:13:13.161569 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d07373e-fa0c-4172-9f20-c068596b76c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:13:13.161904 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d07373e-fa0c-4172-9f20-c068596b76c0 admin admin] PATH_INFO: `/v3/projects` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:13:13.162637 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d07373e-fa0c-4172-9f20-c068596b76c0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 16:13:13.167735 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d07373e-fa0c-4172-9f20-c068596b76c0 admin admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 16:13:13.174812 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ba614c3-0fa5-49a9-a6d5-108871022bd0 admin admin] Validating token access rules against request {{(pid=81767) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 16:13:13.181480 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a146bd98-afd1-4f38-803f-e2e6f7e549da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a98b1e4431940ab8d1fc95df2858a6c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81767) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 16:13:13.183094 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a146bd98-afd1-4f38-803f-e2e6f7e549da admin admin] REQUEST_METHOD: `GET` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:13:13.183759 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a146bd98-afd1-4f38-803f-e2e6f7e549da admin admin] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:13:13.184226 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a146bd98-afd1-4f38-803f-e2e6f7e549da admin admin] PATH_INFO: `/v3/domains` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:13:13.185263 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a146bd98-afd1-4f38-803f-e2e6f7e549da admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81767) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 16:13:13.186118 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a146bd98-afd1-4f38-803f-e2e6f7e549da admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 16:13:13.188897 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a146bd98-afd1-4f38-803f-e2e6f7e549da admin admin] RBAC: Authorization granted {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 16:13:13.203249 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 1236/2514] 10.222.0.15 () {58 vars in 1153 bytes} [Mon Aug 22 16:13:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 16:13:13.204720 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b85e44-ae60-4a06-9a14-5ff51a7f5c4e None None] REQUEST_METHOD: `POST` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:13:13.205074 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b85e44-ae60-4a06-9a14-5ff51a7f5c4e None None] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:13:13.205427 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b85e44-ae60-4a06-9a14-5ff51a7f5c4e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:13:13.236922 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.auth.core [None req-47b85e44-ae60-4a06-9a14-5ff51a7f5c4e None None] MFA Rules not processed for user `26d4cd03ad1e4be081210b68e2c6d0d8`. Rule list: `[]` (Enabled: `True`). {{(pid=81767) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 16:13:13.238368 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.fernet_utils [None req-47b85e44-ae60-4a06-9a14-5ff51a7f5c4e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81767) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 16:13:13.272110 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 1279/2515] 10.222.0.15 () {60 vars in 1153 bytes} [Mon Aug 22 16:13:13 2022] POST /identity/v3/projects => generated 399 bytes in 255 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 22 16:13:13.274190 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d07373e-fa0c-4172-9f20-c068596b76c0 admin admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 16:13:13.314461 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 1237/2516] 10.222.0.15 () {58 vars in 958 bytes} [Mon Aug 22 16:13:13 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 16:13:13.315418 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47b85e44-ae60-4a06-9a14-5ff51a7f5c4e None None] Authenticating user token {{(pid=81767) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 16:13:13.329670 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47b85e44-ae60-4a06-9a14-5ff51a7f5c4e None None] Validating token access rules against request {{(pid=81767) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 16:13:13.355522 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d07373e-fa0c-4172-9f20-c068596b76c0 admin admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 16:13:13.362107 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aef44fe2-1e56-4c85-9b58-4dcfd15c0308 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a98b1e4431940ab8d1fc95df2858a6c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 16:13:13.364006 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aef44fe2-1e56-4c85-9b58-4dcfd15c0308 admin admin] REQUEST_METHOD: `POST` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:13:13.364293 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aef44fe2-1e56-4c85-9b58-4dcfd15c0308 admin admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:13:13.364575 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aef44fe2-1e56-4c85-9b58-4dcfd15c0308 admin admin] PATH_INFO: `/v3/projects` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:13:13.365249 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aef44fe2-1e56-4c85-9b58-4dcfd15c0308 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 16:13:13.367514 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aef44fe2-1e56-4c85-9b58-4dcfd15c0308 admin admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 16:13:13.389484 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG dbcounter [-] [81767] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81767) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 16:13:13.416061 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6809b70-e224-4fb6-a988-d723e653ae38 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a98b1e4431940ab8d1fc95df2858a6c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81767) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 16:13:13.416687 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6809b70-e224-4fb6-a988-d723e653ae38 admin admin] REQUEST_METHOD: `POST` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:13:13.416969 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6809b70-e224-4fb6-a988-d723e653ae38 admin admin] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:13:13.417240 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6809b70-e224-4fb6-a988-d723e653ae38 admin admin] PATH_INFO: `/v3/users` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:13:13.417963 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6809b70-e224-4fb6-a988-d723e653ae38 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 16:13:13.420145 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6809b70-e224-4fb6-a988-d723e653ae38 admin admin] RBAC: Authorization granted {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 16:13:13.532401 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 1238/2517] 10.222.0.15 () {60 vars in 1144 bytes} [Mon Aug 22 16:13:13 2022] POST /identity/v3/users => generated 300 bytes in 217 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 22 16:13:13.533102 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6809b70-e224-4fb6-a988-d723e653ae38 admin admin] Authenticating user token {{(pid=81767) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 16:13:13.536545 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 1280/2518] 10.222.0.15 () {60 vars in 1153 bytes} [Mon Aug 22 16:13:13 2022] POST /identity/v3/projects => generated 389 bytes in 263 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 22 16:13:13.537533 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aef44fe2-1e56-4c85-9b58-4dcfd15c0308 admin admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 16:13:13.541005 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-aef44fe2-1e56-4c85-9b58-4dcfd15c0308 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 16:13:13.549739 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6809b70-e224-4fb6-a988-d723e653ae38 admin admin] Validating token access rules against request {{(pid=81767) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 16:13:13.619669 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83d9e284-fb08-4e00-bc4f-0633a60df892 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a98b1e4431940ab8d1fc95df2858a6c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81767) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 16:13:13.620282 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83d9e284-fb08-4e00-bc4f-0633a60df892 admin admin] REQUEST_METHOD: `POST` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:13:13.620556 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83d9e284-fb08-4e00-bc4f-0633a60df892 admin admin] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:13:13.620824 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83d9e284-fb08-4e00-bc4f-0633a60df892 admin admin] PATH_INFO: `/v3/projects` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:13:13.621587 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83d9e284-fb08-4e00-bc4f-0633a60df892 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 16:13:13.623518 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83d9e284-fb08-4e00-bc4f-0633a60df892 admin admin] RBAC: Authorization granted {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 16:13:13.633909 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aef44fe2-1e56-4c85-9b58-4dcfd15c0308 admin admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 16:13:13.644193 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d11a2760-ccd9-4c07-bac2-919f5324fc93 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a98b1e4431940ab8d1fc95df2858a6c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 16:13:13.644920 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d11a2760-ccd9-4c07-bac2-919f5324fc93 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:13:13.645694 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d11a2760-ccd9-4c07-bac2-919f5324fc93 admin admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:13:13.645966 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d11a2760-ccd9-4c07-bac2-919f5324fc93 admin admin] PATH_INFO: `/v3/projects/6d12cfa870464db1be34b222751fd7c4` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:13:13.647179 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d11a2760-ccd9-4c07-bac2-919f5324fc93 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6d12cfa870464db1be34b222751fd7c4)` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 16:13:13.649316 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d11a2760-ccd9-4c07-bac2-919f5324fc93 admin admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 16:13:13.755213 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 1239/2519] 10.222.0.15 () {60 vars in 1153 bytes} [Mon Aug 22 16:13:13 2022] POST /identity/v3/projects => generated 385 bytes in 220 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 22 16:13:13.755213 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83d9e284-fb08-4e00-bc4f-0633a60df892 admin admin] Authenticating user token {{(pid=81767) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 16:13:13.799471 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-d11a2760-ccd9-4c07-bac2-919f5324fc93 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6d12cfa870464db1be34b222751fd7c4', 'request_id': 'req-d11a2760-ccd9-4c07-bac2-919f5324fc93'} {{(pid=81768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 16:13:13.806284 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-d11a2760-ccd9-4c07-bac2-919f5324fc93 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6d12cfa870464db1be34b222751fd7c4', 'request_id': 'req-d11a2760-ccd9-4c07-bac2-919f5324fc93'} {{(pid=81768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 16:13:13.833818 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83d9e284-fb08-4e00-bc4f-0633a60df892 admin admin] Validating token access rules against request {{(pid=81767) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 16:13:13.874882 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-d11a2760-ccd9-4c07-bac2-919f5324fc93 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6d12cfa870464db1be34b222751fd7c4', 'request_id': 'req-d11a2760-ccd9-4c07-bac2-919f5324fc93'} {{(pid=81768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 16:13:13.897079 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-d11a2760-ccd9-4c07-bac2-919f5324fc93 admin admin] The token cache is being invalidate because project 6d12cfa870464db1be34b222751fd7c4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81768) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 22 16:13:13.897448 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-d11a2760-ccd9-4c07-bac2-919f5324fc93 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 16:13:13.903657 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-d11a2760-ccd9-4c07-bac2-919f5324fc93 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 16:13:13.903657 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 1281/2520] 10.222.0.15 () {58 vars in 1233 bytes} [Mon Aug 22 16:13:13 2022] DELETE /identity/v3/projects/6d12cfa870464db1be34b222751fd7c4 => generated 0 bytes in 363 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 16:13:13.903657 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d11a2760-ccd9-4c07-bac2-919f5324fc93 admin admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 16:13:13.903657 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-d11a2760-ccd9-4c07-bac2-919f5324fc93 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 16:13:13.905735 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38c76d61-6ede-4bbf-9875-88b04b607ba4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a98b1e4431940ab8d1fc95df2858a6c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81767) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 16:13:13.906356 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38c76d61-6ede-4bbf-9875-88b04b607ba4 admin admin] REQUEST_METHOD: `GET` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:13:13.906666 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38c76d61-6ede-4bbf-9875-88b04b607ba4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:13:13.906941 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38c76d61-6ede-4bbf-9875-88b04b607ba4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:13:13.907624 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38c76d61-6ede-4bbf-9875-88b04b607ba4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 16:13:13.909221 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38c76d61-6ede-4bbf-9875-88b04b607ba4 admin admin] RBAC: Authorization granted {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 16:13:13.921398 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d11a2760-ccd9-4c07-bac2-919f5324fc93 admin admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 16:13:13.927651 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 1240/2521] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Aug 22 16:13:13 2022] GET /identity/v3/roles => generated 1420 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 22 16:13:13.927651 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1a08873-12f7-4c94-ab5f-21663619084e None None] REQUEST_METHOD: `POST` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:13:13.927651 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1a08873-12f7-4c94-ab5f-21663619084e None None] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:13:13.927651 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1a08873-12f7-4c94-ab5f-21663619084e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:13:13.934852 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9083efc2-329d-447c-9aee-e06be92ce4cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a98b1e4431940ab8d1fc95df2858a6c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 16:13:13.935518 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9083efc2-329d-447c-9aee-e06be92ce4cf admin admin] REQUEST_METHOD: `POST` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:13:13.935799 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9083efc2-329d-447c-9aee-e06be92ce4cf admin admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:13:13.936342 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9083efc2-329d-447c-9aee-e06be92ce4cf admin admin] PATH_INFO: `/v3/users` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:13:13.936700 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9083efc2-329d-447c-9aee-e06be92ce4cf admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 16:13:13.939739 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9083efc2-329d-447c-9aee-e06be92ce4cf admin admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 16:13:13.972458 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.auth.core [None req-e1a08873-12f7-4c94-ab5f-21663619084e None None] MFA Rules not processed for user `7be15e5dbc3b4419b4ada8e7f075d6a9`. Rule list: `[]` (Enabled: `True`). {{(pid=81767) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 16:13:13.973691 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.fernet_utils [None req-e1a08873-12f7-4c94-ab5f-21663619084e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81767) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 16:13:14.039685 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 1241/2522] 10.222.0.15 () {58 vars in 958 bytes} [Mon Aug 22 16:13:13 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 16:13:14.040667 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1a08873-12f7-4c94-ab5f-21663619084e None None] Authenticating user token {{(pid=81767) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 16:13:14.042154 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.fernet_utils [None req-e1a08873-12f7-4c94-ab5f-21663619084e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81767) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 16:13:14.057244 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 1282/2523] 10.222.0.15 () {60 vars in 1144 bytes} [Mon Aug 22 16:13:13 2022] POST /identity/v3/users => generated 295 bytes in 154 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 22 16:13:14.057244 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95654844-d7b3-4d18-ace4-3b1f585b4156 None None] REQUEST_METHOD: `POST` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:13:14.057244 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95654844-d7b3-4d18-ace4-3b1f585b4156 None None] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:13:14.057244 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95654844-d7b3-4d18-ace4-3b1f585b4156 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:13:14.066084 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1a08873-12f7-4c94-ab5f-21663619084e None None] Validating token access rules against request {{(pid=81767) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 16:13:14.072904 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-081302b8-14e9-4c61-b8b1-4ae6744c719f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a98b1e4431940ab8d1fc95df2858a6c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81767) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 16:13:14.073556 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-081302b8-14e9-4c61-b8b1-4ae6744c719f admin admin] REQUEST_METHOD: `POST` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:13:14.074293 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-081302b8-14e9-4c61-b8b1-4ae6744c719f admin admin] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:13:14.074585 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-081302b8-14e9-4c61-b8b1-4ae6744c719f admin admin] PATH_INFO: `/v3/users` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:13:14.075216 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-081302b8-14e9-4c61-b8b1-4ae6744c719f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 16:13:14.077225 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-081302b8-14e9-4c61-b8b1-4ae6744c719f admin admin] RBAC: Authorization granted {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 16:13:14.078994 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.auth.core [None req-95654844-d7b3-4d18-ace4-3b1f585b4156 None None] MFA Rules not processed for user `6b2199ae0fb0480aa917ada391f5ec54`. Rule list: `[]` (Enabled: `True`). {{(pid=81768) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 16:13:14.080411 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-95654844-d7b3-4d18-ace4-3b1f585b4156 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 16:13:14.089778 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 1283/2524] 10.222.0.15 () {58 vars in 958 bytes} [Mon Aug 22 16:13:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 16:13:14.090641 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95654844-d7b3-4d18-ace4-3b1f585b4156 None None] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 16:13:14.091935 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-95654844-d7b3-4d18-ace4-3b1f585b4156 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 16:13:14.108266 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95654844-d7b3-4d18-ace4-3b1f585b4156 None None] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 16:13:14.114455 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8449ade9-6a90-4385-abad-5efce1fbf054 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a98b1e4431940ab8d1fc95df2858a6c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 16:13:14.115463 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8449ade9-6a90-4385-abad-5efce1fbf054 admin admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:13:14.115803 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8449ade9-6a90-4385-abad-5efce1fbf054 admin admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:13:14.116498 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8449ade9-6a90-4385-abad-5efce1fbf054 admin admin] PATH_INFO: `/v3/roles` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:13:14.117235 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8449ade9-6a90-4385-abad-5efce1fbf054 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 16:13:14.118894 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8449ade9-6a90-4385-abad-5efce1fbf054 admin admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 16:13:14.130072 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 1284/2525] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Aug 22 16:13:14 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 22 16:13:14.130922 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8449ade9-6a90-4385-abad-5efce1fbf054 admin admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 16:13:14.132670 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-8449ade9-6a90-4385-abad-5efce1fbf054 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 16:13:14.189351 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 1242/2526] 10.222.0.15 () {60 vars in 1144 bytes} [Mon Aug 22 16:13:14 2022] POST /identity/v3/users => generated 293 bytes in 146 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 22 16:13:14.189351 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-081302b8-14e9-4c61-b8b1-4ae6744c719f admin admin] Authenticating user token {{(pid=81767) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 16:13:14.207773 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8449ade9-6a90-4385-abad-5efce1fbf054 admin admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 16:13:14.209830 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-081302b8-14e9-4c61-b8b1-4ae6744c719f admin admin] Validating token access rules against request {{(pid=81767) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 16:13:14.213599 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0f4d5d4-4427-4205-8f13-4c7239a0e733 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79706d480f864998b9f8594be407ceab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4ee0413f9ef4d4fb1f95a660babe090', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 16:13:14.214664 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0f4d5d4-4427-4205-8f13-4c7239a0e733 service cinder] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:13:14.215006 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0f4d5d4-4427-4205-8f13-4c7239a0e733 service cinder] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:13:14.215488 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0f4d5d4-4427-4205-8f13-4c7239a0e733 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:13:14.217221 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97e54f57-5922-4678-a008-904d4a96bb0c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a98b1e4431940ab8d1fc95df2858a6c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81767) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 16:13:14.217848 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97e54f57-5922-4678-a008-904d4a96bb0c admin admin] REQUEST_METHOD: `GET` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:13:14.218164 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97e54f57-5922-4678-a008-904d4a96bb0c admin admin] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:13:14.218603 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97e54f57-5922-4678-a008-904d4a96bb0c admin admin] PATH_INFO: `/v3/roles` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:13:14.220018 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97e54f57-5922-4678-a008-904d4a96bb0c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 16:13:14.222027 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97e54f57-5922-4678-a008-904d4a96bb0c admin admin] RBAC: Authorization granted {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 16:13:14.226349 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0f4d5d4-4427-4205-8f13-4c7239a0e733 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 16:13:14.229381 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0f4d5d4-4427-4205-8f13-4c7239a0e733 service cinder] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 16:13:14.233551 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 1243/2527] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Aug 22 16:13:14 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 22 16:13:14.235737 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b808fce4-ec11-4c91-ab4e-8dfd32dc09b8 None None] REQUEST_METHOD: `POST` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:13:14.236015 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b808fce4-ec11-4c91-ab4e-8dfd32dc09b8 None None] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:13:14.236277 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b808fce4-ec11-4c91-ab4e-8dfd32dc09b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:13:14.243363 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 1285/2528] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Aug 22 16:13:14 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 113 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 16:13:14.244190 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0f4d5d4-4427-4205-8f13-4c7239a0e733 service cinder] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 16:13:14.259225 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0f4d5d4-4427-4205-8f13-4c7239a0e733 service cinder] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 16:13:14.260076 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.auth.core [None req-b808fce4-ec11-4c91-ab4e-8dfd32dc09b8 None None] MFA Rules not processed for user `6b2199ae0fb0480aa917ada391f5ec54`. Rule list: `[]` (Enabled: `True`). {{(pid=81767) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 16:13:14.261532 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.fernet_utils [None req-b808fce4-ec11-4c91-ab4e-8dfd32dc09b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81767) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 16:13:14.264263 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b257e5c-179d-43d1-a890-09967994e821 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a98b1e4431940ab8d1fc95df2858a6c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 16:13:14.265463 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b257e5c-179d-43d1-a890-09967994e821 admin admin] REQUEST_METHOD: `PUT` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:13:14.265463 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b257e5c-179d-43d1-a890-09967994e821 admin admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:13:14.265670 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b257e5c-179d-43d1-a890-09967994e821 admin admin] PATH_INFO: `/v3/projects/9a7fbb564d4e4c2c8fd1fd4b02a6c029/users/e6f322c5805645fd973754061f4eb7be/roles/e6fc1ee3fb974125ba9b4fe55b12b45e` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:13:14.273952 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 1244/2529] 10.222.0.15 () {58 vars in 958 bytes} [Mon Aug 22 16:13:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 16:13:14.273952 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b808fce4-ec11-4c91-ab4e-8dfd32dc09b8 None None] Authenticating user token {{(pid=81767) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 16:13:14.282757 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b808fce4-ec11-4c91-ab4e-8dfd32dc09b8 None None] Validating token access rules against request {{(pid=81767) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 16:13:14.284409 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b257e5c-179d-43d1-a890-09967994e821 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9a7fbb564d4e4c2c8fd1fd4b02a6c029, user_id=e6f322c5805645fd973754061f4eb7be, role_id=e6fc1ee3fb974125ba9b4fe55b12b45e)` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 16:13:14.287475 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b257e5c-179d-43d1-a890-09967994e821 admin admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 16:13:14.288222 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc8a2662-6e10-4852-9932-f15288b4aaea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a98b1e4431940ab8d1fc95df2858a6c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81767) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 16:13:14.288876 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc8a2662-6e10-4852-9932-f15288b4aaea admin admin] REQUEST_METHOD: `GET` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:13:14.289199 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc8a2662-6e10-4852-9932-f15288b4aaea admin admin] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:13:14.289552 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc8a2662-6e10-4852-9932-f15288b4aaea admin admin] PATH_INFO: `/v3/roles` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:13:14.292660 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc8a2662-6e10-4852-9932-f15288b4aaea admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 16:13:14.294984 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc8a2662-6e10-4852-9932-f15288b4aaea admin admin] RBAC: Authorization granted {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 16:13:14.308746 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 1245/2530] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Aug 22 16:13:14 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 22 16:13:14.310062 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc8a2662-6e10-4852-9932-f15288b4aaea admin admin] Authenticating user token {{(pid=81767) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 16:13:14.320091 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc8a2662-6e10-4852-9932-f15288b4aaea admin admin] Validating token access rules against request {{(pid=81767) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 16:13:14.325141 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0b3afaa-f782-45ea-adc6-541425214ba7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a98b1e4431940ab8d1fc95df2858a6c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81767) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 16:13:14.325849 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0b3afaa-f782-45ea-adc6-541425214ba7 admin admin] REQUEST_METHOD: `GET` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:13:14.326314 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0b3afaa-f782-45ea-adc6-541425214ba7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:13:14.326695 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0b3afaa-f782-45ea-adc6-541425214ba7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:13:14.327512 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0b3afaa-f782-45ea-adc6-541425214ba7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 16:13:14.329433 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0b3afaa-f782-45ea-adc6-541425214ba7 admin admin] RBAC: Authorization granted {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 16:13:14.345295 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 1246/2531] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Aug 22 16:13:14 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 22 16:13:14.345295 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0b3afaa-f782-45ea-adc6-541425214ba7 admin admin] Authenticating user token {{(pid=81767) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 16:13:14.357393 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0b3afaa-f782-45ea-adc6-541425214ba7 admin admin] Validating token access rules against request {{(pid=81767) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 16:13:14.372146 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 1286/2532] 10.222.0.15 () {60 vars in 1483 bytes} [Mon Aug 22 16:13:14 2022] PUT /identity/v3/projects/9a7fbb564d4e4c2c8fd1fd4b02a6c029/users/e6f322c5805645fd973754061f4eb7be/roles/e6fc1ee3fb974125ba9b4fe55b12b45e => generated 0 bytes in 128 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 16:13:14.373945 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b257e5c-179d-43d1-a890-09967994e821 admin admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 16:13:14.406107 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 16:13:14.447448 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ced1c6a-4fcb-42e2-acf3-0d850df613fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a98b1e4431940ab8d1fc95df2858a6c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81767) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 16:13:14.448343 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ced1c6a-4fcb-42e2-acf3-0d850df613fd admin admin] REQUEST_METHOD: `GET` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:13:14.448635 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ced1c6a-4fcb-42e2-acf3-0d850df613fd admin admin] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:13:14.448887 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ced1c6a-4fcb-42e2-acf3-0d850df613fd admin admin] PATH_INFO: `/v3/domains` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:13:14.449739 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ced1c6a-4fcb-42e2-acf3-0d850df613fd admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81767) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 16:13:14.450715 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ced1c6a-4fcb-42e2-acf3-0d850df613fd admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 16:13:14.452838 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ced1c6a-4fcb-42e2-acf3-0d850df613fd admin admin] RBAC: Authorization granted {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 16:13:14.468632 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b257e5c-179d-43d1-a890-09967994e821 admin admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 16:13:14.470008 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 1247/2533] 10.222.0.15 () {58 vars in 1153 bytes} [Mon Aug 22 16:13:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 128 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 16:13:14.470956 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ced1c6a-4fcb-42e2-acf3-0d850df613fd admin admin] Authenticating user token {{(pid=81767) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 16:13:14.476405 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e82e9eb2-6b18-41eb-bead-395f4c61a5af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a98b1e4431940ab8d1fc95df2858a6c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 16:13:14.477748 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e82e9eb2-6b18-41eb-bead-395f4c61a5af admin admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:13:14.478026 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e82e9eb2-6b18-41eb-bead-395f4c61a5af admin admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:13:14.478347 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e82e9eb2-6b18-41eb-bead-395f4c61a5af admin admin] PATH_INFO: `/v3/roles` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:13:14.479965 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e82e9eb2-6b18-41eb-bead-395f4c61a5af admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 16:13:14.481895 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ced1c6a-4fcb-42e2-acf3-0d850df613fd admin admin] Validating token access rules against request {{(pid=81767) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 16:13:14.484445 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e82e9eb2-6b18-41eb-bead-395f4c61a5af admin admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 16:13:14.489789 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5662859b-c531-4111-80f0-6d3dc7be46fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a98b1e4431940ab8d1fc95df2858a6c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81767) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 16:13:14.490331 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5662859b-c531-4111-80f0-6d3dc7be46fd admin admin] REQUEST_METHOD: `PUT` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:13:14.490331 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5662859b-c531-4111-80f0-6d3dc7be46fd admin admin] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:13:14.490612 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5662859b-c531-4111-80f0-6d3dc7be46fd admin admin] PATH_INFO: `/v3/projects/e8b902f6d01f4580ac10ddca4ccd7ed4/users/3052a42ed8bb4342b0460d94332f6a6e/roles/e6fc1ee3fb974125ba9b4fe55b12b45e` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:13:14.500095 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 1287/2534] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Aug 22 16:13:14 2022] GET /identity/v3/roles => generated 1420 bytes in 124 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 22 16:13:14.500095 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4f4ddcf-6fee-4535-926d-2382bc4b3648 None None] REQUEST_METHOD: `POST` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:13:14.500095 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4f4ddcf-6fee-4535-926d-2382bc4b3648 None None] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:13:14.500095 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4f4ddcf-6fee-4535-926d-2382bc4b3648 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:13:14.513459 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5662859b-c531-4111-80f0-6d3dc7be46fd admin admin] RBAC: Authorizing `identity:create_grant(project_id=e8b902f6d01f4580ac10ddca4ccd7ed4, user_id=3052a42ed8bb4342b0460d94332f6a6e, role_id=e6fc1ee3fb974125ba9b4fe55b12b45e)` {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 16:13:14.516699 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5662859b-c531-4111-80f0-6d3dc7be46fd admin admin] RBAC: Authorization granted {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 16:13:14.528432 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.auth.core [None req-a4f4ddcf-6fee-4535-926d-2382bc4b3648 None None] MFA Rules not processed for user `e6f322c5805645fd973754061f4eb7be`. Rule list: `[]` (Enabled: `True`). {{(pid=81768) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 16:13:14.529620 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-a4f4ddcf-6fee-4535-926d-2382bc4b3648 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 16:13:14.557520 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG dbcounter [-] [81767] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81767) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 16:13:14.596213 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 1288/2535] 10.222.0.15 () {58 vars in 958 bytes} [Mon Aug 22 16:13:14 2022] POST /identity/v3/auth/tokens => generated 3276 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 16:13:14.597136 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4f4ddcf-6fee-4535-926d-2382bc4b3648 None None] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 16:13:14.607742 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4f4ddcf-6fee-4535-926d-2382bc4b3648 None None] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 16:13:14.612663 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4b8b16a-7052-4487-a2b2-5f57874a861d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a98b1e4431940ab8d1fc95df2858a6c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 16:13:14.613253 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4b8b16a-7052-4487-a2b2-5f57874a861d admin admin] REQUEST_METHOD: `POST` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:13:14.613581 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4b8b16a-7052-4487-a2b2-5f57874a861d admin admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:13:14.614003 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4b8b16a-7052-4487-a2b2-5f57874a861d admin admin] PATH_INFO: `/v3/projects` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:13:14.614646 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4b8b16a-7052-4487-a2b2-5f57874a861d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 16:13:14.629991 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4b8b16a-7052-4487-a2b2-5f57874a861d admin admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 16:13:14.630145 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 1248/2536] 10.222.0.15 () {60 vars in 1483 bytes} [Mon Aug 22 16:13:14 2022] PUT /identity/v3/projects/e8b902f6d01f4580ac10ddca4ccd7ed4/users/3052a42ed8bb4342b0460d94332f6a6e/roles/e6fc1ee3fb974125ba9b4fe55b12b45e => generated 0 bytes in 148 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 16:13:14.630145 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d27ab7d8-4da5-4fb0-92fd-17e115ded761 None None] REQUEST_METHOD: `POST` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:13:14.630145 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d27ab7d8-4da5-4fb0-92fd-17e115ded761 None None] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:13:14.630145 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d27ab7d8-4da5-4fb0-92fd-17e115ded761 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:13:14.641849 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.auth.core [None req-d27ab7d8-4da5-4fb0-92fd-17e115ded761 None None] MFA Rules not processed for user `058e76354dce412cb78b2875c88f2791`. Rule list: `[]` (Enabled: `True`). {{(pid=81767) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 16:13:14.643099 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.fernet_utils [None req-d27ab7d8-4da5-4fb0-92fd-17e115ded761 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81767) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 16:13:14.756558 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 1289/2537] 10.222.0.15 () {60 vars in 1153 bytes} [Mon Aug 22 16:13:14 2022] POST /identity/v3/projects => generated 407 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 22 16:13:14.757743 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4b8b16a-7052-4487-a2b2-5f57874a861d admin admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 16:13:14.766646 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 1249/2538] 10.222.0.15 () {58 vars in 1009 bytes} [Mon Aug 22 16:13:14 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 16:13:14.767903 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db105cf1-dd92-4c79-b2c8-87c330d9d039 None None] REQUEST_METHOD: `POST` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:13:14.768192 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db105cf1-dd92-4c79-b2c8-87c330d9d039 None None] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:13:14.768470 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db105cf1-dd92-4c79-b2c8-87c330d9d039 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:13:14.805661 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.auth.core [None req-db105cf1-dd92-4c79-b2c8-87c330d9d039 None None] MFA Rules not processed for user `3052a42ed8bb4342b0460d94332f6a6e`. Rule list: `[]` (Enabled: `True`). {{(pid=81767) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 16:13:14.806885 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.fernet_utils [None req-db105cf1-dd92-4c79-b2c8-87c330d9d039 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81767) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 16:13:14.833658 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4b8b16a-7052-4487-a2b2-5f57874a861d admin admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 16:13:14.840509 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-778746ca-4ccd-4962-9b3f-65f3c1e1ba66 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a98b1e4431940ab8d1fc95df2858a6c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 16:13:14.841216 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-778746ca-4ccd-4962-9b3f-65f3c1e1ba66 admin admin] REQUEST_METHOD: `PUT` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:13:14.841499 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-778746ca-4ccd-4962-9b3f-65f3c1e1ba66 admin admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:13:14.842905 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-778746ca-4ccd-4962-9b3f-65f3c1e1ba66 admin admin] PATH_INFO: `/v3/projects/61d3bc83c029431da6368e57f1bd4da3/users/04fdae1d631b45729443e921ea40c74d/roles/e6fc1ee3fb974125ba9b4fe55b12b45e` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:13:14.859625 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-778746ca-4ccd-4962-9b3f-65f3c1e1ba66 admin admin] RBAC: Authorizing `identity:create_grant(project_id=61d3bc83c029431da6368e57f1bd4da3, user_id=04fdae1d631b45729443e921ea40c74d, role_id=e6fc1ee3fb974125ba9b4fe55b12b45e)` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 16:13:14.862341 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-778746ca-4ccd-4962-9b3f-65f3c1e1ba66 admin admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 16:13:14.876958 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 1250/2539] 10.222.0.15 () {58 vars in 958 bytes} [Mon Aug 22 16:13:14 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 16:13:14.877917 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db105cf1-dd92-4c79-b2c8-87c330d9d039 None None] Authenticating user token {{(pid=81767) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 16:13:14.879211 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.fernet_utils [None req-db105cf1-dd92-4c79-b2c8-87c330d9d039 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81767) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 16:13:14.935592 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db105cf1-dd92-4c79-b2c8-87c330d9d039 None None] Validating token access rules against request {{(pid=81767) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 16:13:14.939964 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6129bbd3-34c9-4878-9f50-e0246b4eaada service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79706d480f864998b9f8594be407ceab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4ee0413f9ef4d4fb1f95a660babe090', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81767) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 16:13:14.940694 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6129bbd3-34c9-4878-9f50-e0246b4eaada service cinder] REQUEST_METHOD: `GET` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:13:14.940862 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6129bbd3-34c9-4878-9f50-e0246b4eaada service cinder] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:13:14.941135 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6129bbd3-34c9-4878-9f50-e0246b4eaada service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:13:14.969110 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 1290/2540] 10.222.0.15 () {60 vars in 1483 bytes} [Mon Aug 22 16:13:14 2022] PUT /identity/v3/projects/61d3bc83c029431da6368e57f1bd4da3/users/04fdae1d631b45729443e921ea40c74d/roles/e6fc1ee3fb974125ba9b4fe55b12b45e => generated 0 bytes in 212 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 16:13:14.970354 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-778746ca-4ccd-4962-9b3f-65f3c1e1ba66 admin admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 16:13:15.006831 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6129bbd3-34c9-4878-9f50-e0246b4eaada service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 16:13:15.008197 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6129bbd3-34c9-4878-9f50-e0246b4eaada service cinder] RBAC: Authorization granted {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 16:13:15.017915 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 1251/2541] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Aug 22 16:13:14 2022] GET /identity/v3/auth/tokens => generated 3276 bytes in 140 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 16:13:15.018910 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6129bbd3-34c9-4878-9f50-e0246b4eaada service cinder] Authenticating user token {{(pid=81767) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 16:13:15.020303 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.fernet_utils [None req-6129bbd3-34c9-4878-9f50-e0246b4eaada service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81767) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 16:13:15.049239 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-778746ca-4ccd-4962-9b3f-65f3c1e1ba66 admin admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 16:13:15.056416 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a7bdda8-b390-4493-bc04-1bcb76cf5745 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a98b1e4431940ab8d1fc95df2858a6c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 16:13:15.057298 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a7bdda8-b390-4493-bc04-1bcb76cf5745 admin admin] REQUEST_METHOD: `POST` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:13:15.057686 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a7bdda8-b390-4493-bc04-1bcb76cf5745 admin admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:13:15.058083 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a7bdda8-b390-4493-bc04-1bcb76cf5745 admin admin] PATH_INFO: `/v3/users` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:13:15.058834 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a7bdda8-b390-4493-bc04-1bcb76cf5745 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 16:13:15.072174 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a7bdda8-b390-4493-bc04-1bcb76cf5745 admin admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 16:13:15.111707 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6129bbd3-34c9-4878-9f50-e0246b4eaada service cinder] Validating token access rules against request {{(pid=81767) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 16:13:15.170233 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 1291/2542] 10.222.0.15 () {60 vars in 1144 bytes} [Mon Aug 22 16:13:14 2022] POST /identity/v3/users => generated 304 bytes in 200 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 22 16:13:15.171910 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deb5cbe3-dc8c-4dd4-a7c3-ab1d365f973c None None] REQUEST_METHOD: `POST` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:13:15.172389 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deb5cbe3-dc8c-4dd4-a7c3-ab1d365f973c None None] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:13:15.172973 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deb5cbe3-dc8c-4dd4-a7c3-ab1d365f973c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:13:15.180063 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7826f45d-b495-4d1b-95e0-30e45cc649dd service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a09d128dee9438283633e51f4834643', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4ee0413f9ef4d4fb1f95a660babe090', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81767) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 16:13:15.180687 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7826f45d-b495-4d1b-95e0-30e45cc649dd service swift] REQUEST_METHOD: `GET` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:13:15.180971 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7826f45d-b495-4d1b-95e0-30e45cc649dd service swift] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:13:15.181247 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7826f45d-b495-4d1b-95e0-30e45cc649dd service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:13:15.216729 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.auth.core [None req-deb5cbe3-dc8c-4dd4-a7c3-ab1d365f973c None None] MFA Rules not processed for user `04fdae1d631b45729443e921ea40c74d`. Rule list: `[]` (Enabled: `True`). {{(pid=81768) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 16:13:15.218464 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-deb5cbe3-dc8c-4dd4-a7c3-ab1d365f973c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 16:13:15.268725 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7826f45d-b495-4d1b-95e0-30e45cc649dd service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 16:13:15.268725 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7826f45d-b495-4d1b-95e0-30e45cc649dd service swift] RBAC: Authorization granted {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 16:13:15.276349 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 1252/2543] 10.222.0.15 () {60 vars in 1384 bytes} [Mon Aug 22 16:13:15 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 258 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 22 16:13:15.277359 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7826f45d-b495-4d1b-95e0-30e45cc649dd service swift] Authenticating user token {{(pid=81767) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 16:13:15.371364 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7826f45d-b495-4d1b-95e0-30e45cc649dd service swift] Validating token access rules against request {{(pid=81767) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 16:13:15.375623 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2233bf2f-8b96-4b53-bf48-a5a22038e9fe service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79706d480f864998b9f8594be407ceab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4ee0413f9ef4d4fb1f95a660babe090', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81767) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 16:13:15.376384 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2233bf2f-8b96-4b53-bf48-a5a22038e9fe service cinder] REQUEST_METHOD: `GET` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:13:15.382402 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2233bf2f-8b96-4b53-bf48-a5a22038e9fe service cinder] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:13:15.382402 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2233bf2f-8b96-4b53-bf48-a5a22038e9fe service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:13:15.382818 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 1292/2544] 10.222.0.15 () {58 vars in 958 bytes} [Mon Aug 22 16:13:15 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 206 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 16:13:15.382818 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deb5cbe3-dc8c-4dd4-a7c3-ab1d365f973c None None] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 16:13:15.390354 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deb5cbe3-dc8c-4dd4-a7c3-ab1d365f973c None None] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 16:13:15.408780 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bda777d-22c9-435f-a0c3-7f3aeddfdd54 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a98b1e4431940ab8d1fc95df2858a6c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 16:13:15.409300 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bda777d-22c9-435f-a0c3-7f3aeddfdd54 admin admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:13:15.409598 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bda777d-22c9-435f-a0c3-7f3aeddfdd54 admin admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:13:15.409870 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bda777d-22c9-435f-a0c3-7f3aeddfdd54 admin admin] PATH_INFO: `/v3/roles` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:13:15.410966 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bda777d-22c9-435f-a0c3-7f3aeddfdd54 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 16:13:15.413183 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bda777d-22c9-435f-a0c3-7f3aeddfdd54 admin admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 16:13:15.435354 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 1293/2545] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Aug 22 16:13:15 2022] GET /identity/v3/roles => generated 1420 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 22 16:13:15.439611 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bda777d-22c9-435f-a0c3-7f3aeddfdd54 admin admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 16:13:15.455262 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bda777d-22c9-435f-a0c3-7f3aeddfdd54 admin admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 16:13:15.467884 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6e03c4c-3d5b-4ed5-bfa2-6d5ab111b2a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a98b1e4431940ab8d1fc95df2858a6c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 16:13:15.468863 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6e03c4c-3d5b-4ed5-bfa2-6d5ab111b2a5 admin admin] REQUEST_METHOD: `POST` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:13:15.469770 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6e03c4c-3d5b-4ed5-bfa2-6d5ab111b2a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:13:15.470324 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6e03c4c-3d5b-4ed5-bfa2-6d5ab111b2a5 admin admin] PATH_INFO: `/v3/projects` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:13:15.471123 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6e03c4c-3d5b-4ed5-bfa2-6d5ab111b2a5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 16:13:15.473543 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6e03c4c-3d5b-4ed5-bfa2-6d5ab111b2a5 admin admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 16:13:15.500522 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2233bf2f-8b96-4b53-bf48-a5a22038e9fe service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 16:13:15.502074 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2233bf2f-8b96-4b53-bf48-a5a22038e9fe service cinder] RBAC: Authorization granted {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 16:13:15.525985 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 1253/2546] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Aug 22 16:13:15 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 249 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 16:13:15.527091 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2233bf2f-8b96-4b53-bf48-a5a22038e9fe service cinder] Authenticating user token {{(pid=81767) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 16:13:15.545592 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2233bf2f-8b96-4b53-bf48-a5a22038e9fe service cinder] Validating token access rules against request {{(pid=81767) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 16:13:15.548024 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72b4f278-d1d4-4891-a996-af17631f56c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a98b1e4431940ab8d1fc95df2858a6c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81767) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 16:13:15.548539 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b4f278-d1d4-4891-a996-af17631f56c0 admin admin] REQUEST_METHOD: `GET` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:13:15.548818 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b4f278-d1d4-4891-a996-af17631f56c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:13:15.549086 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b4f278-d1d4-4891-a996-af17631f56c0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:13:15.549867 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72b4f278-d1d4-4891-a996-af17631f56c0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 16:13:15.551687 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72b4f278-d1d4-4891-a996-af17631f56c0 admin admin] RBAC: Authorization granted {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 16:13:15.562129 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 1254/2547] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Aug 22 16:13:15 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 22 16:13:15.567311 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72b4f278-d1d4-4891-a996-af17631f56c0 admin admin] Authenticating user token {{(pid=81767) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 16:13:15.578080 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72b4f278-d1d4-4891-a996-af17631f56c0 admin admin] Validating token access rules against request {{(pid=81767) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 16:13:15.583711 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc4728fb-07cd-4cc5-9cc8-a5ac65666f1d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a98b1e4431940ab8d1fc95df2858a6c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81767) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 16:13:15.584623 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc4728fb-07cd-4cc5-9cc8-a5ac65666f1d admin admin] REQUEST_METHOD: `PUT` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:13:15.584902 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc4728fb-07cd-4cc5-9cc8-a5ac65666f1d admin admin] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:13:15.585177 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc4728fb-07cd-4cc5-9cc8-a5ac65666f1d admin admin] PATH_INFO: `/v3/projects/e7a1c4960137485fb7f5a3cc7c1cb619/users/f2c372b4c45648c7846e04c721fc36a0/roles/e6fc1ee3fb974125ba9b4fe55b12b45e` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:13:15.601497 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc4728fb-07cd-4cc5-9cc8-a5ac65666f1d admin admin] RBAC: Authorizing `identity:create_grant(project_id=e7a1c4960137485fb7f5a3cc7c1cb619, user_id=f2c372b4c45648c7846e04c721fc36a0, role_id=e6fc1ee3fb974125ba9b4fe55b12b45e)` {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 16:13:15.603011 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 1294/2548] 10.222.0.15 () {60 vars in 1153 bytes} [Mon Aug 22 16:13:15 2022] POST /identity/v3/projects => generated 385 bytes in 167 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 22 16:13:15.604259 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc4728fb-07cd-4cc5-9cc8-a5ac65666f1d admin admin] RBAC: Authorization granted {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 16:13:15.606596 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6e03c4c-3d5b-4ed5-bfa2-6d5ab111b2a5 admin admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 16:13:15.678922 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6e03c4c-3d5b-4ed5-bfa2-6d5ab111b2a5 admin admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 16:13:15.681441 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 1255/2549] 10.222.0.15 () {60 vars in 1483 bytes} [Mon Aug 22 16:13:15 2022] PUT /identity/v3/projects/e7a1c4960137485fb7f5a3cc7c1cb619/users/f2c372b4c45648c7846e04c721fc36a0/roles/e6fc1ee3fb974125ba9b4fe55b12b45e => generated 0 bytes in 115 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 22 16:13:15.694420 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa24197-d282-43ac-854f-90d2437c4183 None None] REQUEST_METHOD: `POST` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:13:15.694733 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa24197-d282-43ac-854f-90d2437c4183 None None] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:13:15.694966 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa24197-d282-43ac-854f-90d2437c4183 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:13:15.727421 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.auth.core [None req-3fa24197-d282-43ac-854f-90d2437c4183 None None] MFA Rules not processed for user `f2c372b4c45648c7846e04c721fc36a0`. Rule list: `[]` (Enabled: `True`). {{(pid=81767) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 16:13:15.728728 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.fernet_utils [None req-3fa24197-d282-43ac-854f-90d2437c4183 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81767) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 16:13:15.750824 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ddb56a1-c227-40c8-80c7-be8286231fcf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a98b1e4431940ab8d1fc95df2858a6c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 16:13:15.751481 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ddb56a1-c227-40c8-80c7-be8286231fcf admin admin] REQUEST_METHOD: `POST` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:13:15.751764 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ddb56a1-c227-40c8-80c7-be8286231fcf admin admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:13:15.752034 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ddb56a1-c227-40c8-80c7-be8286231fcf admin admin] PATH_INFO: `/v3/users` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:13:15.752651 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ddb56a1-c227-40c8-80c7-be8286231fcf admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 16:13:15.754877 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ddb56a1-c227-40c8-80c7-be8286231fcf admin admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 16:13:15.802206 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 1256/2550] 10.222.0.15 () {58 vars in 958 bytes} [Mon Aug 22 16:13:15 2022] POST /identity/v3/auth/tokens => generated 3284 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 16:13:15.803697 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fa24197-d282-43ac-854f-90d2437c4183 None None] Authenticating user token {{(pid=81767) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 16:13:15.815708 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fa24197-d282-43ac-854f-90d2437c4183 None None] Validating token access rules against request {{(pid=81767) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 16:13:15.820790 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f912cfea-b6ee-40e2-a5d5-8843612ac51a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a98b1e4431940ab8d1fc95df2858a6c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81767) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 16:13:15.821388 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f912cfea-b6ee-40e2-a5d5-8843612ac51a admin admin] REQUEST_METHOD: `POST` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:13:15.821874 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f912cfea-b6ee-40e2-a5d5-8843612ac51a admin admin] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:13:15.821987 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f912cfea-b6ee-40e2-a5d5-8843612ac51a admin admin] PATH_INFO: `/v3/projects` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:13:15.822551 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f912cfea-b6ee-40e2-a5d5-8843612ac51a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 16:13:15.824514 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f912cfea-b6ee-40e2-a5d5-8843612ac51a admin admin] RBAC: Authorization granted {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 16:13:15.886476 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 1295/2551] 10.222.0.15 () {60 vars in 1144 bytes} [Mon Aug 22 16:13:15 2022] POST /identity/v3/users => generated 299 bytes in 280 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 22 16:13:15.889857 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ddb56a1-c227-40c8-80c7-be8286231fcf admin admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 16:13:15.900755 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ddb56a1-c227-40c8-80c7-be8286231fcf admin admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 16:13:15.905530 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d01ce89-7a07-4985-b490-530a4ab57023 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a98b1e4431940ab8d1fc95df2858a6c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 16:13:15.906153 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d01ce89-7a07-4985-b490-530a4ab57023 admin admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:13:15.906543 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d01ce89-7a07-4985-b490-530a4ab57023 admin admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:13:15.906824 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d01ce89-7a07-4985-b490-530a4ab57023 admin admin] PATH_INFO: `/v3/roles` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:13:15.907497 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d01ce89-7a07-4985-b490-530a4ab57023 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 16:13:15.909103 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d01ce89-7a07-4985-b490-530a4ab57023 admin admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 16:13:15.919343 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 1296/2552] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Aug 22 16:13:15 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 22 16:13:15.923631 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d01ce89-7a07-4985-b490-530a4ab57023 admin admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 16:13:15.935492 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d01ce89-7a07-4985-b490-530a4ab57023 admin admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 16:13:15.947575 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-173f6a45-b463-409d-a54b-f256ca671534 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a98b1e4431940ab8d1fc95df2858a6c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 16:13:15.948245 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-173f6a45-b463-409d-a54b-f256ca671534 admin admin] REQUEST_METHOD: `PUT` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:13:15.948523 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-173f6a45-b463-409d-a54b-f256ca671534 admin admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:13:15.948795 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-173f6a45-b463-409d-a54b-f256ca671534 admin admin] PATH_INFO: `/v3/projects/6b8a2d1450354d698edd83ad16967159/users/02bf5dcc8e1a49ccbf3efd9d0499bea1/roles/9d9848cfd22e4b089db443c350e56670` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:13:15.962452 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 1257/2553] 10.222.0.15 () {60 vars in 1153 bytes} [Mon Aug 22 16:13:15 2022] POST /identity/v3/projects => generated 409 bytes in 156 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 22 16:13:15.963639 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f912cfea-b6ee-40e2-a5d5-8843612ac51a admin admin] Authenticating user token {{(pid=81767) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 16:13:15.970804 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-173f6a45-b463-409d-a54b-f256ca671534 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6b8a2d1450354d698edd83ad16967159, user_id=02bf5dcc8e1a49ccbf3efd9d0499bea1, role_id=9d9848cfd22e4b089db443c350e56670)` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 16:13:15.974443 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-173f6a45-b463-409d-a54b-f256ca671534 admin admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 16:13:16.026316 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f912cfea-b6ee-40e2-a5d5-8843612ac51a admin admin] Validating token access rules against request {{(pid=81767) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 16:13:16.031189 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f708b61a-661b-4558-900b-6552d2cd820a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a98b1e4431940ab8d1fc95df2858a6c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81767) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 16:13:16.031823 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f708b61a-661b-4558-900b-6552d2cd820a admin admin] REQUEST_METHOD: `POST` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:13:16.032234 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f708b61a-661b-4558-900b-6552d2cd820a admin admin] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:13:16.032500 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f708b61a-661b-4558-900b-6552d2cd820a admin admin] PATH_INFO: `/v3/users` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:13:16.033120 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f708b61a-661b-4558-900b-6552d2cd820a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 16:13:16.035024 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f708b61a-661b-4558-900b-6552d2cd820a admin admin] RBAC: Authorization granted {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 16:13:16.110744 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 1297/2554] 10.222.0.15 () {60 vars in 1483 bytes} [Mon Aug 22 16:13:15 2022] PUT /identity/v3/projects/6b8a2d1450354d698edd83ad16967159/users/02bf5dcc8e1a49ccbf3efd9d0499bea1/roles/9d9848cfd22e4b089db443c350e56670 => generated 0 bytes in 187 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 22 16:13:16.115451 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-173f6a45-b463-409d-a54b-f256ca671534 admin admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 16:13:16.175314 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 1258/2555] 10.222.0.15 () {60 vars in 1144 bytes} [Mon Aug 22 16:13:15 2022] POST /identity/v3/users => generated 311 bytes in 212 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 22 16:13:16.179583 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f708b61a-661b-4558-900b-6552d2cd820a admin admin] Authenticating user token {{(pid=81767) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 16:13:16.210114 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f708b61a-661b-4558-900b-6552d2cd820a admin admin] Validating token access rules against request {{(pid=81767) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 16:13:16.210243 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-173f6a45-b463-409d-a54b-f256ca671534 admin admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 16:13:16.212674 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3819dfa2-60b6-4bc6-862b-df8e59e7a4e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a98b1e4431940ab8d1fc95df2858a6c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81767) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 16:13:16.213505 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3819dfa2-60b6-4bc6-862b-df8e59e7a4e7 admin admin] REQUEST_METHOD: `GET` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:13:16.213609 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3819dfa2-60b6-4bc6-862b-df8e59e7a4e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:13:16.213844 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3819dfa2-60b6-4bc6-862b-df8e59e7a4e7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:13:16.214515 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3819dfa2-60b6-4bc6-862b-df8e59e7a4e7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 16:13:16.215146 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fd69c10-6f4d-46d8-90ae-e96bdb2bda62 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a98b1e4431940ab8d1fc95df2858a6c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 16:13:16.215817 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fd69c10-6f4d-46d8-90ae-e96bdb2bda62 admin admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:13:16.216131 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3819dfa2-60b6-4bc6-862b-df8e59e7a4e7 admin admin] RBAC: Authorization granted {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 16:13:16.216246 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fd69c10-6f4d-46d8-90ae-e96bdb2bda62 admin admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:13:16.216674 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fd69c10-6f4d-46d8-90ae-e96bdb2bda62 admin admin] PATH_INFO: `/v3/roles` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:13:16.217334 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fd69c10-6f4d-46d8-90ae-e96bdb2bda62 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 16:13:16.219014 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fd69c10-6f4d-46d8-90ae-e96bdb2bda62 admin admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 16:13:16.229848 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 1298/2556] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Aug 22 16:13:16 2022] GET /identity/v3/roles => generated 1420 bytes in 115 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 22 16:13:16.230301 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 1259/2557] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Aug 22 16:13:16 2022] GET /identity/v3/roles => generated 1420 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 22 16:13:16.240172 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fd69c10-6f4d-46d8-90ae-e96bdb2bda62 admin admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 16:13:16.240423 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3819dfa2-60b6-4bc6-862b-df8e59e7a4e7 admin admin] Authenticating user token {{(pid=81767) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 16:13:16.248306 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3819dfa2-60b6-4bc6-862b-df8e59e7a4e7 admin admin] Validating token access rules against request {{(pid=81767) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 16:13:16.253254 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-556705c1-eae3-42b9-91b6-c0b960ddbf64 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a98b1e4431940ab8d1fc95df2858a6c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81767) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 16:13:16.254049 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-556705c1-eae3-42b9-91b6-c0b960ddbf64 admin admin] REQUEST_METHOD: `PUT` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:13:16.254410 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-556705c1-eae3-42b9-91b6-c0b960ddbf64 admin admin] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:13:16.254784 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-556705c1-eae3-42b9-91b6-c0b960ddbf64 admin admin] PATH_INFO: `/v3/projects/6b8a2d1450354d698edd83ad16967159/users/02bf5dcc8e1a49ccbf3efd9d0499bea1/roles/9d9848cfd22e4b089db443c350e56670` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:13:16.257056 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-556705c1-eae3-42b9-91b6-c0b960ddbf64 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6b8a2d1450354d698edd83ad16967159, user_id=02bf5dcc8e1a49ccbf3efd9d0499bea1, role_id=9d9848cfd22e4b089db443c350e56670)` {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 16:13:16.258662 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fd69c10-6f4d-46d8-90ae-e96bdb2bda62 admin admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 16:13:16.260070 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-556705c1-eae3-42b9-91b6-c0b960ddbf64 admin admin] RBAC: Authorization granted {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 16:13:16.269774 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5eb036e-d348-4ed2-a3d9-c67f61207ed5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a98b1e4431940ab8d1fc95df2858a6c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 16:13:16.270736 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5eb036e-d348-4ed2-a3d9-c67f61207ed5 admin admin] REQUEST_METHOD: `PUT` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:13:16.271496 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5eb036e-d348-4ed2-a3d9-c67f61207ed5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:13:16.271976 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5eb036e-d348-4ed2-a3d9-c67f61207ed5 admin admin] PATH_INFO: `/v3/projects/0cb170a9a5e548448e32393c9d527b90/users/c23424d1cdf8460bb5251646b64bb154/roles/9d9848cfd22e4b089db443c350e56670` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:13:16.285149 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 1260/2558] 10.222.0.15 () {60 vars in 1483 bytes} [Mon Aug 22 16:13:16 2022] PUT /identity/v3/projects/6b8a2d1450354d698edd83ad16967159/users/02bf5dcc8e1a49ccbf3efd9d0499bea1/roles/9d9848cfd22e4b089db443c350e56670 => generated 0 bytes in 41 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 22 16:13:16.288885 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3144c49-4f41-489d-9bbf-547070c41a17 None None] REQUEST_METHOD: `POST` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:13:16.289315 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3144c49-4f41-489d-9bbf-547070c41a17 None None] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:13:16.291702 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3144c49-4f41-489d-9bbf-547070c41a17 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:13:16.304689 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5eb036e-d348-4ed2-a3d9-c67f61207ed5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0cb170a9a5e548448e32393c9d527b90, user_id=c23424d1cdf8460bb5251646b64bb154, role_id=9d9848cfd22e4b089db443c350e56670)` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 16:13:16.307669 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5eb036e-d348-4ed2-a3d9-c67f61207ed5 admin admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 16:13:16.329210 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 16:13:16.345866 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.auth.core [None req-a3144c49-4f41-489d-9bbf-547070c41a17 None None] MFA Rules not processed for user `02bf5dcc8e1a49ccbf3efd9d0499bea1`. Rule list: `[]` (Enabled: `True`). {{(pid=81767) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 16:13:16.347224 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.fernet_utils [None req-a3144c49-4f41-489d-9bbf-547070c41a17 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81767) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 16:13:16.402642 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 1299/2559] 10.222.0.15 () {60 vars in 1483 bytes} [Mon Aug 22 16:13:16 2022] PUT /identity/v3/projects/0cb170a9a5e548448e32393c9d527b90/users/c23424d1cdf8460bb5251646b64bb154/roles/9d9848cfd22e4b089db443c350e56670 => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 16:13:16.403537 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5eb036e-d348-4ed2-a3d9-c67f61207ed5 admin admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 16:13:16.464751 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 1261/2560] 10.222.0.15 () {58 vars in 958 bytes} [Mon Aug 22 16:13:16 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 22 16:13:16.473052 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3144c49-4f41-489d-9bbf-547070c41a17 None None] Authenticating user token {{(pid=81767) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 16:13:16.517354 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5eb036e-d348-4ed2-a3d9-c67f61207ed5 admin admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 16:13:16.524727 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3ada1ee-218f-4b10-88e9-d2279407bebc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a98b1e4431940ab8d1fc95df2858a6c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 16:13:16.525353 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3ada1ee-218f-4b10-88e9-d2279407bebc admin admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:13:16.525634 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3ada1ee-218f-4b10-88e9-d2279407bebc admin admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:13:16.525903 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3ada1ee-218f-4b10-88e9-d2279407bebc admin admin] PATH_INFO: `/v3/roles` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:13:16.526599 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3ada1ee-218f-4b10-88e9-d2279407bebc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 16:13:16.528332 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3ada1ee-218f-4b10-88e9-d2279407bebc admin admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 16:13:16.541920 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 1300/2561] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Aug 22 16:13:16 2022] GET /identity/v3/roles => generated 1420 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 22 16:13:16.546248 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3ada1ee-218f-4b10-88e9-d2279407bebc admin admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 16:13:16.550292 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3144c49-4f41-489d-9bbf-547070c41a17 None None] Validating token access rules against request {{(pid=81767) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 16:13:16.569614 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3ada1ee-218f-4b10-88e9-d2279407bebc admin admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 16:13:16.576837 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d834eb5-b80b-4e22-b4a5-8d5cd3d1b1ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a98b1e4431940ab8d1fc95df2858a6c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 16:13:16.577511 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d834eb5-b80b-4e22-b4a5-8d5cd3d1b1ce admin admin] REQUEST_METHOD: `PUT` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:13:16.577845 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d834eb5-b80b-4e22-b4a5-8d5cd3d1b1ce admin admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:13:16.578509 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d834eb5-b80b-4e22-b4a5-8d5cd3d1b1ce admin admin] PATH_INFO: `/v3/projects/0cb170a9a5e548448e32393c9d527b90/users/c23424d1cdf8460bb5251646b64bb154/roles/9d9848cfd22e4b089db443c350e56670` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:13:16.585729 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d834eb5-b80b-4e22-b4a5-8d5cd3d1b1ce admin admin] RBAC: Authorizing `identity:create_grant(project_id=0cb170a9a5e548448e32393c9d527b90, user_id=c23424d1cdf8460bb5251646b64bb154, role_id=9d9848cfd22e4b089db443c350e56670)` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 16:13:16.585729 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d834eb5-b80b-4e22-b4a5-8d5cd3d1b1ce admin admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 16:13:16.603553 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 1301/2562] 10.222.0.15 () {60 vars in 1483 bytes} [Mon Aug 22 16:13:16 2022] PUT /identity/v3/projects/0cb170a9a5e548448e32393c9d527b90/users/c23424d1cdf8460bb5251646b64bb154/roles/9d9848cfd22e4b089db443c350e56670 => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 16:13:16.610879 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG dbcounter [-] [81767] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81767) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 16:13:16.614393 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87106fc6-ad2c-43b6-afe8-80a8928e5f66 None None] REQUEST_METHOD: `POST` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:13:16.614674 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87106fc6-ad2c-43b6-afe8-80a8928e5f66 None None] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:13:16.614950 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87106fc6-ad2c-43b6-afe8-80a8928e5f66 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:13:16.632489 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21316db9-7a8c-41fc-aed4-25f26db1d861 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79706d480f864998b9f8594be407ceab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4ee0413f9ef4d4fb1f95a660babe090', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81767) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 16:13:16.633310 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21316db9-7a8c-41fc-aed4-25f26db1d861 service cinder] REQUEST_METHOD: `GET` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:13:16.633454 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21316db9-7a8c-41fc-aed4-25f26db1d861 service cinder] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:13:16.633707 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21316db9-7a8c-41fc-aed4-25f26db1d861 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:13:16.635927 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.fernet_utils [None req-21316db9-7a8c-41fc-aed4-25f26db1d861 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81767) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 16:13:16.649979 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.auth.core [None req-87106fc6-ad2c-43b6-afe8-80a8928e5f66 None None] MFA Rules not processed for user `c23424d1cdf8460bb5251646b64bb154`. Rule list: `[]` (Enabled: `True`). {{(pid=81768) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 16:13:16.651719 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-87106fc6-ad2c-43b6-afe8-80a8928e5f66 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 16:13:16.711992 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21316db9-7a8c-41fc-aed4-25f26db1d861 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 16:13:16.713416 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21316db9-7a8c-41fc-aed4-25f26db1d861 service cinder] RBAC: Authorization granted {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 16:13:16.723497 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 1262/2563] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Aug 22 16:13:16 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 251 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 22 16:13:16.729070 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 1302/2564] 10.222.0.15 () {58 vars in 958 bytes} [Mon Aug 22 16:13:16 2022] POST /identity/v3/auth/tokens => generated 3353 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 22 16:13:16.739987 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21316db9-7a8c-41fc-aed4-25f26db1d861 service cinder] Authenticating user token {{(pid=81767) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 16:13:16.749275 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21316db9-7a8c-41fc-aed4-25f26db1d861 service cinder] Validating token access rules against request {{(pid=81767) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 16:13:16.816985 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06f39e06-5814-430d-9a0b-80f61765b0ca service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79706d480f864998b9f8594be407ceab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4ee0413f9ef4d4fb1f95a660babe090', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81767) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 16:13:16.817601 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f39e06-5814-430d-9a0b-80f61765b0ca service cinder] REQUEST_METHOD: `GET` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:13:16.818019 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f39e06-5814-430d-9a0b-80f61765b0ca service cinder] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:13:16.818185 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f39e06-5814-430d-9a0b-80f61765b0ca service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:13:16.829544 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06f39e06-5814-430d-9a0b-80f61765b0ca service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 16:13:16.831545 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06f39e06-5814-430d-9a0b-80f61765b0ca service cinder] RBAC: Authorization granted {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 16:13:16.852976 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 1263/2565] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Aug 22 16:13:16 2022] GET /identity/v3/auth/tokens => generated 3353 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 16:13:17.280278 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87106fc6-ad2c-43b6-afe8-80a8928e5f66 None None] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 16:13:17.293996 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87106fc6-ad2c-43b6-afe8-80a8928e5f66 None None] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 16:13:17.299131 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64d9484d-c283-4e97-a5cb-96b4ddca13c4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79706d480f864998b9f8594be407ceab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4ee0413f9ef4d4fb1f95a660babe090', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 16:13:17.299834 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d9484d-c283-4e97-a5cb-96b4ddca13c4 service cinder] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:13:17.300115 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d9484d-c283-4e97-a5cb-96b4ddca13c4 service cinder] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:13:17.300381 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d9484d-c283-4e97-a5cb-96b4ddca13c4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:13:17.362933 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64d9484d-c283-4e97-a5cb-96b4ddca13c4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 16:13:17.364464 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64d9484d-c283-4e97-a5cb-96b4ddca13c4 service cinder] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 16:13:17.376780 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 1303/2566] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Aug 22 16:13:17 2022] GET /identity/v3/auth/tokens => generated 3284 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 16:13:17.498400 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=88,keystone:DELETE=7,keystone:INSERT=5 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 16:13:17.784010 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG dbcounter [-] [81767] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81767) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 16:13:17.931669 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06f39e06-5814-430d-9a0b-80f61765b0ca service cinder] Authenticating user token {{(pid=81767) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 16:13:17.933201 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.fernet_utils [None req-06f39e06-5814-430d-9a0b-80f61765b0ca service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81767) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 16:13:18.006075 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06f39e06-5814-430d-9a0b-80f61765b0ca service cinder] Validating token access rules against request {{(pid=81767) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 16:13:18.010935 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b734184-20e0-4058-9173-42b0b90d4ee8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a98b1e4431940ab8d1fc95df2858a6c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81767) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 16:13:18.011603 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b734184-20e0-4058-9173-42b0b90d4ee8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:13:18.011885 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b734184-20e0-4058-9173-42b0b90d4ee8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:13:18.012160 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b734184-20e0-4058-9173-42b0b90d4ee8 admin admin] PATH_INFO: `/v3/users/01bb7d8843624abcbeba077d9ccdb6a6` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:13:18.013410 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b734184-20e0-4058-9173-42b0b90d4ee8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=01bb7d8843624abcbeba077d9ccdb6a6)` {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 16:13:18.015334 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b734184-20e0-4058-9173-42b0b90d4ee8 admin admin] RBAC: Authorization granted {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 16:13:18.115109 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d60d7d24-4798-4fc2-9d5f-7aeb34fe14f7 None None] REQUEST_METHOD: `POST` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:13:18.115680 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d60d7d24-4798-4fc2-9d5f-7aeb34fe14f7 None None] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:13:18.116317 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d60d7d24-4798-4fc2-9d5f-7aeb34fe14f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:13:18.139915 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.auth.core [None req-d60d7d24-4798-4fc2-9d5f-7aeb34fe14f7 None None] MFA Rules not processed for user `7be15e5dbc3b4419b4ada8e7f075d6a9`. Rule list: `[]` (Enabled: `True`). {{(pid=81768) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 16:13:18.141688 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-d60d7d24-4798-4fc2-9d5f-7aeb34fe14f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 16:13:18.206727 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.notifications [None req-9b734184-20e0-4058-9173-42b0b90d4ee8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '01bb7d8843624abcbeba077d9ccdb6a6'} {{(pid=81767) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 16:13:18.207142 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 1304/2567] 10.222.0.15 () {58 vars in 958 bytes} [Mon Aug 22 16:13:18 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 22 16:13:18.207761 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.notifications [None req-9b734184-20e0-4058-9173-42b0b90d4ee8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '01bb7d8843624abcbeba077d9ccdb6a6'} {{(pid=81767) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 16:13:18.303991 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.notifications [None req-9b734184-20e0-4058-9173-42b0b90d4ee8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '01bb7d8843624abcbeba077d9ccdb6a6'} {{(pid=81767) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 16:13:18.316203 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.notifications [None req-9b734184-20e0-4058-9173-42b0b90d4ee8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '01bb7d8843624abcbeba077d9ccdb6a6'} {{(pid=81767) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 16:13:18.332270 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.notifications [None req-9b734184-20e0-4058-9173-42b0b90d4ee8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '01bb7d8843624abcbeba077d9ccdb6a6'} {{(pid=81767) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 16:13:18.340257 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 1264/2568] 10.222.0.15 () {58 vars in 1224 bytes} [Mon Aug 22 16:13:17 2022] DELETE /identity/v3/users/01bb7d8843624abcbeba077d9ccdb6a6 => generated 0 bytes in 409 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 16:13:18.489662 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e84dc3d7-5d2b-4f6b-95d8-c78415563750 None None] REQUEST_METHOD: `POST` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:13:18.490035 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e84dc3d7-5d2b-4f6b-95d8-c78415563750 None None] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:13:18.490316 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e84dc3d7-5d2b-4f6b-95d8-c78415563750 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:13:18.497592 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application [None req-e84dc3d7-5d2b-4f6b-95d8-c78415563750 None None] Expecting to find domain in project. The server could not comply with the request since it is either malformed or otherwise incorrect. The client is assumed to be in error.: keystone.exception.ValidationError: Expecting to find domain in project. The server could not comply with the request since it is either malformed or otherwise incorrect. The client is assumed to be in error. Aug 22 16:13:18.497592 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 16:13:18.497592 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 16:13:18.497592 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 16:13:18.497592 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 16:13:18.497592 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 16:13:18.497592 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 16:13:18.497592 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 16:13:18.497592 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 16:13:18.497592 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 16:13:18.497592 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 16:13:18.497592 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 16:13:18.497592 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/server/flask/common.py", line 1064, in wrapper Aug 22 16:13:18.499125 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return f(*args, **kwargs) Aug 22 16:13:18.499125 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/auth.py", line 315, in post Aug 22 16:13:18.499125 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application token = authentication.authenticate_for_token(auth_data) Aug 22 16:13:18.499125 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/_shared/authentication.py", line 182, in authenticate_for_token Aug 22 16:13:18.499125 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application auth_info = core.AuthInfo.create(auth=auth) Aug 22 16:13:18.499125 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/auth/core.py", line 126, in create Aug 22 16:13:18.499125 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application auth_info._validate_and_normalize_auth_data(scope_only) Aug 22 16:13:18.499125 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/auth/core.py", line 340, in _validate_and_normalize_auth_data Aug 22 16:13:18.499125 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application self._validate_and_normalize_scope_data() Aug 22 16:13:18.499125 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/auth/core.py", line 290, in _validate_and_normalize_scope_data Aug 22 16:13:18.499125 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application project_ref = self._lookup_project(self.auth['scope']['project']) Aug 22 16:13:18.499125 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/auth/core.py", line 194, in _lookup_project Aug 22 16:13:18.499125 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application raise exception.ValidationError(attribute='domain', Aug 22 16:13:18.499125 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application keystone.exception.ValidationError: Expecting to find domain in project. The server could not comply with the request since it is either malformed or otherwise incorrect. The client is assumed to be in error. Aug 22 16:13:18.499125 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application Aug 22 16:13:18.501079 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 1305/2569] 10.222.0.15 () {58 vars in 1009 bytes} [Mon Aug 22 16:13:18 2022] POST /identity/v3/auth/tokens => generated 230 bytes in 10 msecs (HTTP/1.1 400) 5 headers in 186 bytes (1 switches on core 0) Aug 22 16:13:18.686589 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b734184-20e0-4058-9173-42b0b90d4ee8 admin admin] Authenticating user token {{(pid=81767) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 16:13:18.688236 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.fernet_utils [None req-9b734184-20e0-4058-9173-42b0b90d4ee8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81767) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 16:13:18.764824 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b734184-20e0-4058-9173-42b0b90d4ee8 admin admin] Validating token access rules against request {{(pid=81767) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 16:13:18.769468 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-427b3752-8327-4b63-94df-98c8fae0e5ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a98b1e4431940ab8d1fc95df2858a6c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81767) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 16:13:18.770011 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-427b3752-8327-4b63-94df-98c8fae0e5ca admin admin] REQUEST_METHOD: `DELETE` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:13:18.770290 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-427b3752-8327-4b63-94df-98c8fae0e5ca admin admin] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:13:18.770556 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-427b3752-8327-4b63-94df-98c8fae0e5ca admin admin] PATH_INFO: `/v3/projects/6c8f21f91d4441a39dccb10831ccf011` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:13:18.771729 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-427b3752-8327-4b63-94df-98c8fae0e5ca admin admin] RBAC: Authorizing `identity:delete_project(project_id=6c8f21f91d4441a39dccb10831ccf011)` {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 16:13:18.773614 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-427b3752-8327-4b63-94df-98c8fae0e5ca admin admin] RBAC: Authorization granted {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 16:13:18.946948 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.notifications [None req-427b3752-8327-4b63-94df-98c8fae0e5ca admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6c8f21f91d4441a39dccb10831ccf011', 'request_id': 'req-427b3752-8327-4b63-94df-98c8fae0e5ca'} {{(pid=81767) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 16:13:18.952340 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.notifications [None req-427b3752-8327-4b63-94df-98c8fae0e5ca admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6c8f21f91d4441a39dccb10831ccf011', 'request_id': 'req-427b3752-8327-4b63-94df-98c8fae0e5ca'} {{(pid=81767) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 16:13:19.039745 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.notifications [None req-427b3752-8327-4b63-94df-98c8fae0e5ca admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6c8f21f91d4441a39dccb10831ccf011', 'request_id': 'req-427b3752-8327-4b63-94df-98c8fae0e5ca'} {{(pid=81767) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 16:13:19.051644 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.notifications [None req-427b3752-8327-4b63-94df-98c8fae0e5ca admin admin] The token cache is being invalidate because project 6c8f21f91d4441a39dccb10831ccf011 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81767) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 22 16:13:19.052038 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.notifications [None req-427b3752-8327-4b63-94df-98c8fae0e5ca admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81767) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 16:13:19.052791 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.notifications [None req-427b3752-8327-4b63-94df-98c8fae0e5ca admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81767) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 16:13:19.054283 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 1265/2570] 10.222.0.15 () {58 vars in 1233 bytes} [Mon Aug 22 16:13:18 2022] DELETE /identity/v3/projects/6c8f21f91d4441a39dccb10831ccf011 => generated 0 bytes in 369 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 16:13:19.058344 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e84dc3d7-5d2b-4f6b-95d8-c78415563750 None None] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 16:13:19.060765 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG dbcounter [-] [81767] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81767) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 16:13:19.060917 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-e84dc3d7-5d2b-4f6b-95d8-c78415563750 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 16:13:19.140508 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e84dc3d7-5d2b-4f6b-95d8-c78415563750 None None] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 16:13:19.145676 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-399bcc1f-59a7-4a9c-b7c4-0931380f6582 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a98b1e4431940ab8d1fc95df2858a6c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 16:13:19.147093 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-399bcc1f-59a7-4a9c-b7c4-0931380f6582 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:13:19.147646 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-399bcc1f-59a7-4a9c-b7c4-0931380f6582 admin admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:13:19.148048 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-399bcc1f-59a7-4a9c-b7c4-0931380f6582 admin admin] PATH_INFO: `/v3/users/a165e8e56b4f4cdb8f6eb4831bf0feeb` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:13:19.152072 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-399bcc1f-59a7-4a9c-b7c4-0931380f6582 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a165e8e56b4f4cdb8f6eb4831bf0feeb)` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 16:13:19.154172 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-399bcc1f-59a7-4a9c-b7c4-0931380f6582 admin admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 16:13:19.371825 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-399bcc1f-59a7-4a9c-b7c4-0931380f6582 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a165e8e56b4f4cdb8f6eb4831bf0feeb'} {{(pid=81768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 16:13:19.372658 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-399bcc1f-59a7-4a9c-b7c4-0931380f6582 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a165e8e56b4f4cdb8f6eb4831bf0feeb'} {{(pid=81768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 16:13:19.373440 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-399bcc1f-59a7-4a9c-b7c4-0931380f6582 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a165e8e56b4f4cdb8f6eb4831bf0feeb'} {{(pid=81768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 16:13:19.381825 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-399bcc1f-59a7-4a9c-b7c4-0931380f6582 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a165e8e56b4f4cdb8f6eb4831bf0feeb'} {{(pid=81768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 16:13:19.451898 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-399bcc1f-59a7-4a9c-b7c4-0931380f6582 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a165e8e56b4f4cdb8f6eb4831bf0feeb'} {{(pid=81768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 16:13:19.460938 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 16:13:19.474624 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 1306/2571] 10.222.0.15 () {58 vars in 1224 bytes} [Mon Aug 22 16:13:19 2022] DELETE /identity/v3/users/a165e8e56b4f4cdb8f6eb4831bf0feeb => generated 0 bytes in 417 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 16:13:19.480993 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-427b3752-8327-4b63-94df-98c8fae0e5ca admin admin] Authenticating user token {{(pid=81767) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 16:13:19.482490 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.fernet_utils [None req-427b3752-8327-4b63-94df-98c8fae0e5ca admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81767) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 16:13:19.556575 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-427b3752-8327-4b63-94df-98c8fae0e5ca admin admin] Validating token access rules against request {{(pid=81767) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 16:13:19.621391 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c790fad-39bc-47c7-bb57-cd11c6001530 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79706d480f864998b9f8594be407ceab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4ee0413f9ef4d4fb1f95a660babe090', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81767) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 16:13:19.622304 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c790fad-39bc-47c7-bb57-cd11c6001530 service cinder] REQUEST_METHOD: `GET` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:13:19.622744 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c790fad-39bc-47c7-bb57-cd11c6001530 service cinder] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:13:19.627564 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c790fad-39bc-47c7-bb57-cd11c6001530 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:13:19.627564 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.fernet_utils [None req-7c790fad-39bc-47c7-bb57-cd11c6001530 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81767) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 16:13:19.686387 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c790fad-39bc-47c7-bb57-cd11c6001530 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 16:13:19.688440 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c790fad-39bc-47c7-bb57-cd11c6001530 service cinder] RBAC: Authorization granted {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 16:13:19.697521 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 1266/2572] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Aug 22 16:13:19 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 217 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 16:13:19.773455 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-399bcc1f-59a7-4a9c-b7c4-0931380f6582 admin admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 16:13:19.776640 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-399bcc1f-59a7-4a9c-b7c4-0931380f6582 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 16:13:19.854539 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-399bcc1f-59a7-4a9c-b7c4-0931380f6582 admin admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 16:13:19.860048 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-566d0bb8-beb7-46aa-a50f-edcd2fe67239 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a98b1e4431940ab8d1fc95df2858a6c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 16:13:19.860683 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-566d0bb8-beb7-46aa-a50f-edcd2fe67239 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:13:19.860984 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-566d0bb8-beb7-46aa-a50f-edcd2fe67239 admin admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:13:19.861255 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-566d0bb8-beb7-46aa-a50f-edcd2fe67239 admin admin] PATH_INFO: `/v3/projects/e807ae4c0f2d4c67a3a7da815459c063` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:13:19.862383 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-566d0bb8-beb7-46aa-a50f-edcd2fe67239 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e807ae4c0f2d4c67a3a7da815459c063)` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 16:13:19.864528 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-566d0bb8-beb7-46aa-a50f-edcd2fe67239 admin admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 16:13:19.985801 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-566d0bb8-beb7-46aa-a50f-edcd2fe67239 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e807ae4c0f2d4c67a3a7da815459c063', 'request_id': 'req-566d0bb8-beb7-46aa-a50f-edcd2fe67239'} {{(pid=81768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 16:13:19.991862 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-566d0bb8-beb7-46aa-a50f-edcd2fe67239 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e807ae4c0f2d4c67a3a7da815459c063', 'request_id': 'req-566d0bb8-beb7-46aa-a50f-edcd2fe67239'} {{(pid=81768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 16:13:20.080442 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-566d0bb8-beb7-46aa-a50f-edcd2fe67239 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e807ae4c0f2d4c67a3a7da815459c063', 'request_id': 'req-566d0bb8-beb7-46aa-a50f-edcd2fe67239'} {{(pid=81768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 16:13:20.088625 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG dbcounter [-] [81767] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81767) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 16:13:20.093572 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-566d0bb8-beb7-46aa-a50f-edcd2fe67239 admin admin] The token cache is being invalidate because project e807ae4c0f2d4c67a3a7da815459c063 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81768) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 22 16:13:20.094278 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-566d0bb8-beb7-46aa-a50f-edcd2fe67239 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 16:13:20.094946 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-566d0bb8-beb7-46aa-a50f-edcd2fe67239 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 16:13:20.096484 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 1307/2573] 10.222.0.15 () {58 vars in 1233 bytes} [Mon Aug 22 16:13:19 2022] DELETE /identity/v3/projects/e807ae4c0f2d4c67a3a7da815459c063 => generated 0 bytes in 324 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 16:13:20.101699 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6822943f-2057-45a7-abca-ced37688f18d None None] REQUEST_METHOD: `POST` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:13:20.102105 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6822943f-2057-45a7-abca-ced37688f18d None None] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:13:20.102450 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6822943f-2057-45a7-abca-ced37688f18d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:13:20.123754 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.auth.core [None req-6822943f-2057-45a7-abca-ced37688f18d None None] MFA Rules not processed for user `6b2199ae0fb0480aa917ada391f5ec54`. Rule list: `[]` (Enabled: `True`). {{(pid=81767) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 16:13:20.125154 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.fernet_utils [None req-6822943f-2057-45a7-abca-ced37688f18d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81767) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 16:13:20.191825 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 1267/2574] 10.222.0.15 () {58 vars in 958 bytes} [Mon Aug 22 16:13:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 22 16:13:20.205892 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c597e751-0b93-46a3-9311-509f2efa58aa None None] REQUEST_METHOD: `POST` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:13:20.206205 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c597e751-0b93-46a3-9311-509f2efa58aa None None] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:13:20.206425 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c597e751-0b93-46a3-9311-509f2efa58aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:13:20.229920 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.auth.core [None req-c597e751-0b93-46a3-9311-509f2efa58aa None None] MFA Rules not processed for user `6b2199ae0fb0480aa917ada391f5ec54`. Rule list: `[]` (Enabled: `True`). {{(pid=81768) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 16:13:20.231116 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-c597e751-0b93-46a3-9311-509f2efa58aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 16:13:20.243838 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 1308/2575] 10.222.0.15 () {58 vars in 958 bytes} [Mon Aug 22 16:13:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 16:13:20.245487 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6822943f-2057-45a7-abca-ced37688f18d None None] Authenticating user token {{(pid=81767) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 16:13:20.256736 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6822943f-2057-45a7-abca-ced37688f18d None None] Validating token access rules against request {{(pid=81767) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 16:13:20.263674 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b8aab33-59a2-4115-abd0-6073084bca58 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a98b1e4431940ab8d1fc95df2858a6c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81767) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 16:13:20.264833 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b8aab33-59a2-4115-abd0-6073084bca58 admin admin] REQUEST_METHOD: `GET` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:13:20.265413 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b8aab33-59a2-4115-abd0-6073084bca58 admin admin] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:13:20.265982 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b8aab33-59a2-4115-abd0-6073084bca58 admin admin] PATH_INFO: `/v3/domains` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:13:20.267244 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b8aab33-59a2-4115-abd0-6073084bca58 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81767) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 16:13:20.267903 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b8aab33-59a2-4115-abd0-6073084bca58 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 16:13:20.271095 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b8aab33-59a2-4115-abd0-6073084bca58 admin admin] RBAC: Authorization granted {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 16:13:20.294730 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 1268/2576] 10.222.0.15 () {58 vars in 1153 bytes} [Mon Aug 22 16:13:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 16:13:20.301011 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c597e751-0b93-46a3-9311-509f2efa58aa None None] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 16:13:20.318162 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c597e751-0b93-46a3-9311-509f2efa58aa None None] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 16:13:20.325603 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5629f6f0-5494-43ee-850d-d7cd95267a21 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a98b1e4431940ab8d1fc95df2858a6c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 16:13:20.326746 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5629f6f0-5494-43ee-850d-d7cd95267a21 admin admin] REQUEST_METHOD: `POST` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:13:20.327204 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5629f6f0-5494-43ee-850d-d7cd95267a21 admin admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:13:20.327679 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5629f6f0-5494-43ee-850d-d7cd95267a21 admin admin] PATH_INFO: `/v3/projects` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:13:20.328773 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5629f6f0-5494-43ee-850d-d7cd95267a21 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 16:13:20.333072 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5629f6f0-5494-43ee-850d-d7cd95267a21 admin admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 16:13:20.371558 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 16:13:20.463971 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 1309/2577] 10.222.0.15 () {60 vars in 1153 bytes} [Mon Aug 22 16:13:20 2022] POST /identity/v3/projects => generated 371 bytes in 164 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 22 16:13:20.470347 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b8aab33-59a2-4115-abd0-6073084bca58 admin admin] Authenticating user token {{(pid=81767) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 16:13:20.481618 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b8aab33-59a2-4115-abd0-6073084bca58 admin admin] Validating token access rules against request {{(pid=81767) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 16:13:20.552253 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-407a1e82-f893-4401-8313-d2b32622b179 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a98b1e4431940ab8d1fc95df2858a6c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81767) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 16:13:20.552253 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-407a1e82-f893-4401-8313-d2b32622b179 admin admin] REQUEST_METHOD: `POST` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:13:20.552253 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-407a1e82-f893-4401-8313-d2b32622b179 admin admin] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:13:20.552253 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-407a1e82-f893-4401-8313-d2b32622b179 admin admin] PATH_INFO: `/v3/users` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:13:20.552900 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-407a1e82-f893-4401-8313-d2b32622b179 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 16:13:20.552900 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-407a1e82-f893-4401-8313-d2b32622b179 admin admin] RBAC: Authorization granted {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 16:13:20.654901 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 1269/2578] 10.222.0.15 () {60 vars in 1144 bytes} [Mon Aug 22 16:13:20 2022] POST /identity/v3/users => generated 286 bytes in 187 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 22 16:13:20.661425 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5629f6f0-5494-43ee-850d-d7cd95267a21 admin admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 16:13:20.678551 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5629f6f0-5494-43ee-850d-d7cd95267a21 admin admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 16:13:20.683426 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30dd4281-cdd1-460d-96cb-e2f3350d7c2b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a98b1e4431940ab8d1fc95df2858a6c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 16:13:20.684053 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30dd4281-cdd1-460d-96cb-e2f3350d7c2b admin admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:13:20.684323 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30dd4281-cdd1-460d-96cb-e2f3350d7c2b admin admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:13:20.686101 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30dd4281-cdd1-460d-96cb-e2f3350d7c2b admin admin] PATH_INFO: `/v3/roles` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:13:20.686101 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30dd4281-cdd1-460d-96cb-e2f3350d7c2b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 16:13:20.687046 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30dd4281-cdd1-460d-96cb-e2f3350d7c2b admin admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 16:13:20.697256 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 1310/2579] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Aug 22 16:13:20 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 22 16:13:20.701664 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-407a1e82-f893-4401-8313-d2b32622b179 admin admin] Authenticating user token {{(pid=81767) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 16:13:20.713560 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-407a1e82-f893-4401-8313-d2b32622b179 admin admin] Validating token access rules against request {{(pid=81767) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 16:13:20.718608 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51c21c4b-6954-405a-a4f2-996593ec3ecb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a98b1e4431940ab8d1fc95df2858a6c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81767) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 16:13:20.719896 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51c21c4b-6954-405a-a4f2-996593ec3ecb admin admin] REQUEST_METHOD: `GET` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:13:20.720107 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51c21c4b-6954-405a-a4f2-996593ec3ecb admin admin] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:13:20.720374 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51c21c4b-6954-405a-a4f2-996593ec3ecb admin admin] PATH_INFO: `/v3/roles` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:13:20.721090 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51c21c4b-6954-405a-a4f2-996593ec3ecb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 16:13:20.722669 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51c21c4b-6954-405a-a4f2-996593ec3ecb admin admin] RBAC: Authorization granted {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 16:13:20.735546 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 1270/2580] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Aug 22 16:13:20 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 22 16:13:20.739555 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30dd4281-cdd1-460d-96cb-e2f3350d7c2b admin admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 16:13:20.750489 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30dd4281-cdd1-460d-96cb-e2f3350d7c2b admin admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 16:13:20.755482 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38c830b7-f41e-4c8b-86ae-76373f160ee8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a98b1e4431940ab8d1fc95df2858a6c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 16:13:20.756174 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38c830b7-f41e-4c8b-86ae-76373f160ee8 admin admin] REQUEST_METHOD: `PUT` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:13:20.756513 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38c830b7-f41e-4c8b-86ae-76373f160ee8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:13:20.756788 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38c830b7-f41e-4c8b-86ae-76373f160ee8 admin admin] PATH_INFO: `/v3/projects/2399c81627ad4305bcea2f29aaac5d4e/users/a257f47de3544e2d8bd1e7aa55bf9495/roles/e6fc1ee3fb974125ba9b4fe55b12b45e` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:13:20.762909 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG dbcounter [-] [81767] Writing DB stats keystone:SELECT=100 {{(pid=81767) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 16:13:20.774593 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38c830b7-f41e-4c8b-86ae-76373f160ee8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2399c81627ad4305bcea2f29aaac5d4e, user_id=a257f47de3544e2d8bd1e7aa55bf9495, role_id=e6fc1ee3fb974125ba9b4fe55b12b45e)` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 16:13:20.777636 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38c830b7-f41e-4c8b-86ae-76373f160ee8 admin admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 16:13:20.876932 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 1311/2581] 10.222.0.15 () {60 vars in 1483 bytes} [Mon Aug 22 16:13:20 2022] PUT /identity/v3/projects/2399c81627ad4305bcea2f29aaac5d4e/users/a257f47de3544e2d8bd1e7aa55bf9495/roles/e6fc1ee3fb974125ba9b4fe55b12b45e => generated 0 bytes in 136 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 16:13:20.886564 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ef3d0b-5912-421a-9c7a-b850de7d315f None None] REQUEST_METHOD: `POST` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:13:20.886838 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ef3d0b-5912-421a-9c7a-b850de7d315f None None] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:13:20.889422 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ef3d0b-5912-421a-9c7a-b850de7d315f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:13:20.916802 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.auth.core [None req-17ef3d0b-5912-421a-9c7a-b850de7d315f None None] MFA Rules not processed for user `a257f47de3544e2d8bd1e7aa55bf9495`. Rule list: `[]` (Enabled: `True`). {{(pid=81767) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 16:13:20.918259 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.fernet_utils [None req-17ef3d0b-5912-421a-9c7a-b850de7d315f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81767) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 16:13:20.944938 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 16:13:20.970133 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG dbcounter [-] [81767] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81767) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 16:13:20.998181 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 1271/2582] 10.222.0.15 () {58 vars in 958 bytes} [Mon Aug 22 16:13:20 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 22 16:13:21.009832 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38c830b7-f41e-4c8b-86ae-76373f160ee8 admin admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 16:13:21.011808 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-38c830b7-f41e-4c8b-86ae-76373f160ee8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 16:13:21.076229 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38c830b7-f41e-4c8b-86ae-76373f160ee8 admin admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 16:13:21.081043 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85d5ce17-4064-44ab-94d3-a867caebfbdc service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79706d480f864998b9f8594be407ceab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4ee0413f9ef4d4fb1f95a660babe090', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 16:13:21.081657 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d5ce17-4064-44ab-94d3-a867caebfbdc service cinder] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:13:21.081935 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d5ce17-4064-44ab-94d3-a867caebfbdc service cinder] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:13:21.082272 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d5ce17-4064-44ab-94d3-a867caebfbdc service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:13:21.092155 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85d5ce17-4064-44ab-94d3-a867caebfbdc service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 16:13:21.093622 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85d5ce17-4064-44ab-94d3-a867caebfbdc service cinder] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 16:13:21.103395 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 1312/2583] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Aug 22 16:13:21 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 16:13:21.282919 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 16:13:21.346905 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG dbcounter [-] [81767] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81767) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 16:13:21.855316 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 16:13:21.909528 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG dbcounter [-] [81767] Writing DB stats keystone:SELECT=100 {{(pid=81767) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 16:13:21.961488 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17ef3d0b-5912-421a-9c7a-b850de7d315f None None] Authenticating user token {{(pid=81767) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 16:13:21.972803 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17ef3d0b-5912-421a-9c7a-b850de7d315f None None] Validating token access rules against request {{(pid=81767) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 16:13:21.976671 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59fa3ff1-7adb-49ac-8f38-8c71ed5fa2ee service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79706d480f864998b9f8594be407ceab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4ee0413f9ef4d4fb1f95a660babe090', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81767) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 16:13:21.977316 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59fa3ff1-7adb-49ac-8f38-8c71ed5fa2ee service cinder] REQUEST_METHOD: `GET` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:13:21.977684 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59fa3ff1-7adb-49ac-8f38-8c71ed5fa2ee service cinder] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:13:21.977885 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59fa3ff1-7adb-49ac-8f38-8c71ed5fa2ee service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:13:21.979933 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.fernet_utils [None req-59fa3ff1-7adb-49ac-8f38-8c71ed5fa2ee service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81767) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 16:13:22.062987 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59fa3ff1-7adb-49ac-8f38-8c71ed5fa2ee service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 16:13:22.064903 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59fa3ff1-7adb-49ac-8f38-8c71ed5fa2ee service cinder] RBAC: Authorization granted {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 16:13:22.075077 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 1272/2584] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Aug 22 16:13:21 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 16:13:22.095863 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG dbcounter [-] [81767] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=81767) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 16:13:22.383000 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG dbcounter [-] [81767] Writing DB stats keystone:SELECT=100 {{(pid=81767) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 16:13:32.331550 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=68,keystone:DELETE=6,keystone:INSERT=3 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 16:13:32.412010 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG dbcounter [-] [81767] Writing DB stats keystone:SELECT=92,keystone:INSERT=3 {{(pid=81767) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 16:13:48.503785 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85d5ce17-4064-44ab-94d3-a867caebfbdc service cinder] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 16:13:48.505370 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-85d5ce17-4064-44ab-94d3-a867caebfbdc service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 16:13:48.580731 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85d5ce17-4064-44ab-94d3-a867caebfbdc service cinder] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 16:13:48.586126 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1cc5404-072f-4b3c-9a2c-0c2179e7fee5 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '08159bf8edda4246bb463f7b11b0894b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4ee0413f9ef4d4fb1f95a660babe090', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 16:13:48.586728 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1cc5404-072f-4b3c-9a2c-0c2179e7fee5 service placement] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:13:48.587007 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1cc5404-072f-4b3c-9a2c-0c2179e7fee5 service placement] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:13:48.587301 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1cc5404-072f-4b3c-9a2c-0c2179e7fee5 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:13:48.589153 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-c1cc5404-072f-4b3c-9a2c-0c2179e7fee5 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 16:13:48.601082 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1cc5404-072f-4b3c-9a2c-0c2179e7fee5 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 16:13:48.602714 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1cc5404-072f-4b3c-9a2c-0c2179e7fee5 service placement] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 16:13:48.613014 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 1313/2585] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Aug 22 16:13:48 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 16:13:56.636587 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d712c14-c8a2-4a24-86ff-3f90895a3cb4 None None] REQUEST_METHOD: `POST` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:13:56.637202 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d712c14-c8a2-4a24-86ff-3f90895a3cb4 None None] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:13:56.637202 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d712c14-c8a2-4a24-86ff-3f90895a3cb4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:13:56.658362 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.auth.core [None req-4d712c14-c8a2-4a24-86ff-3f90895a3cb4 None None] MFA Rules not processed for user `7be15e5dbc3b4419b4ada8e7f075d6a9`. Rule list: `[]` (Enabled: `True`). {{(pid=81767) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 16:13:56.659621 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.fernet_utils [None req-4d712c14-c8a2-4a24-86ff-3f90895a3cb4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81767) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 16:13:56.723622 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 1273/2586] 10.222.0.15 () {58 vars in 958 bytes} [Mon Aug 22 16:13:56 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 22 16:13:56.898466 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1cc5404-072f-4b3c-9a2c-0c2179e7fee5 service placement] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 16:13:56.900129 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-c1cc5404-072f-4b3c-9a2c-0c2179e7fee5 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 16:13:56.990390 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4dd910f-2a8f-4e5a-b455-45b6b5b3e55a None None] REQUEST_METHOD: `POST` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:13:56.990854 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4dd910f-2a8f-4e5a-b455-45b6b5b3e55a None None] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:13:56.991234 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4dd910f-2a8f-4e5a-b455-45b6b5b3e55a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:13:57.003126 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application [None req-e4dd910f-2a8f-4e5a-b455-45b6b5b3e55a None None] Expecting to find domain in project. The server could not comply with the request since it is either malformed or otherwise incorrect. The client is assumed to be in error.: keystone.exception.ValidationError: Expecting to find domain in project. The server could not comply with the request since it is either malformed or otherwise incorrect. The client is assumed to be in error. Aug 22 16:13:57.003126 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 16:13:57.003126 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 16:13:57.003126 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 16:13:57.003126 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 16:13:57.003126 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 16:13:57.003126 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 16:13:57.003126 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 16:13:57.003126 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 16:13:57.003126 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 16:13:57.003126 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 16:13:57.003126 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 16:13:57.003126 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/server/flask/common.py", line 1064, in wrapper Aug 22 16:13:57.014293 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return f(*args, **kwargs) Aug 22 16:13:57.014293 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/auth.py", line 315, in post Aug 22 16:13:57.014293 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application token = authentication.authenticate_for_token(auth_data) Aug 22 16:13:57.014293 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/_shared/authentication.py", line 182, in authenticate_for_token Aug 22 16:13:57.014293 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application auth_info = core.AuthInfo.create(auth=auth) Aug 22 16:13:57.014293 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/auth/core.py", line 126, in create Aug 22 16:13:57.014293 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application auth_info._validate_and_normalize_auth_data(scope_only) Aug 22 16:13:57.014293 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/auth/core.py", line 340, in _validate_and_normalize_auth_data Aug 22 16:13:57.014293 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application self._validate_and_normalize_scope_data() Aug 22 16:13:57.014293 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/auth/core.py", line 290, in _validate_and_normalize_scope_data Aug 22 16:13:57.014293 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application project_ref = self._lookup_project(self.auth['scope']['project']) Aug 22 16:13:57.014293 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/auth/core.py", line 194, in _lookup_project Aug 22 16:13:57.014293 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application raise exception.ValidationError(attribute='domain', Aug 22 16:13:57.014293 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application keystone.exception.ValidationError: Expecting to find domain in project. The server could not comply with the request since it is either malformed or otherwise incorrect. The client is assumed to be in error. Aug 22 16:13:57.014293 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application Aug 22 16:13:57.017260 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 1274/2587] 10.222.0.15 () {58 vars in 1009 bytes} [Mon Aug 22 16:13:56 2022] POST /identity/v3/auth/tokens => generated 230 bytes in 16 msecs (HTTP/1.1 400) 5 headers in 186 bytes (1 switches on core 0) Aug 22 16:13:57.023063 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1cc5404-072f-4b3c-9a2c-0c2179e7fee5 service placement] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 16:13:57.042016 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58ff7afc-c531-440e-9b0c-4d03ac8b0bf8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a98b1e4431940ab8d1fc95df2858a6c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 16:13:57.042016 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58ff7afc-c531-440e-9b0c-4d03ac8b0bf8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:13:57.042219 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58ff7afc-c531-440e-9b0c-4d03ac8b0bf8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:13:57.042473 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58ff7afc-c531-440e-9b0c-4d03ac8b0bf8 admin admin] PATH_INFO: `/v3/users/e6f322c5805645fd973754061f4eb7be` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:13:57.043875 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58ff7afc-c531-440e-9b0c-4d03ac8b0bf8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e6f322c5805645fd973754061f4eb7be)` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 16:13:57.045853 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58ff7afc-c531-440e-9b0c-4d03ac8b0bf8 admin admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 16:13:57.265702 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-58ff7afc-c531-440e-9b0c-4d03ac8b0bf8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e6f322c5805645fd973754061f4eb7be'} {{(pid=81768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 16:13:57.266646 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-58ff7afc-c531-440e-9b0c-4d03ac8b0bf8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e6f322c5805645fd973754061f4eb7be'} {{(pid=81768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 16:13:57.267386 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-58ff7afc-c531-440e-9b0c-4d03ac8b0bf8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e6f322c5805645fd973754061f4eb7be'} {{(pid=81768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 16:13:57.278189 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-58ff7afc-c531-440e-9b0c-4d03ac8b0bf8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e6f322c5805645fd973754061f4eb7be'} {{(pid=81768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 16:13:57.356832 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-58ff7afc-c531-440e-9b0c-4d03ac8b0bf8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e6f322c5805645fd973754061f4eb7be'} {{(pid=81768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 16:13:57.378090 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 1314/2588] 10.222.0.15 () {58 vars in 1224 bytes} [Mon Aug 22 16:13:56 2022] DELETE /identity/v3/users/e6f322c5805645fd973754061f4eb7be => generated 0 bytes in 480 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 16:13:57.382649 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4281a343-1597-4ccf-b712-b1b2dbb60634 None None] REQUEST_METHOD: `POST` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:13:57.382925 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4281a343-1597-4ccf-b712-b1b2dbb60634 None None] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:13:57.383344 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4281a343-1597-4ccf-b712-b1b2dbb60634 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:13:57.404586 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.auth.core [None req-4281a343-1597-4ccf-b712-b1b2dbb60634 None None] MFA Rules not processed for user `6b2199ae0fb0480aa917ada391f5ec54`. Rule list: `[]` (Enabled: `True`). {{(pid=81767) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 16:13:57.405907 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.fernet_utils [None req-4281a343-1597-4ccf-b712-b1b2dbb60634 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81767) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 16:13:57.475569 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 1275/2589] 10.222.0.15 () {58 vars in 958 bytes} [Mon Aug 22 16:13:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 16:13:57.485974 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58ff7afc-c531-440e-9b0c-4d03ac8b0bf8 admin admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 16:13:57.487467 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-58ff7afc-c531-440e-9b0c-4d03ac8b0bf8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 16:13:57.498596 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 16:13:57.552429 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58ff7afc-c531-440e-9b0c-4d03ac8b0bf8 admin admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 16:13:57.560760 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ae5e12b-85a6-44e8-93ef-e18ee78c4f66 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '234687adad594f1aa45775a565ffbdc7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4ee0413f9ef4d4fb1f95a660babe090', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 16:13:57.560760 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ae5e12b-85a6-44e8-93ef-e18ee78c4f66 service neutron] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:13:57.560760 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ae5e12b-85a6-44e8-93ef-e18ee78c4f66 service neutron] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:13:57.560760 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ae5e12b-85a6-44e8-93ef-e18ee78c4f66 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:13:57.572101 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ae5e12b-85a6-44e8-93ef-e18ee78c4f66 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 16:13:57.573602 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ae5e12b-85a6-44e8-93ef-e18ee78c4f66 service neutron] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 16:13:57.590734 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 1315/2590] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Aug 22 16:13:57 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 16:13:58.134423 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4281a343-1597-4ccf-b712-b1b2dbb60634 None None] Authenticating user token {{(pid=81767) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 16:13:58.135917 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.fernet_utils [None req-4281a343-1597-4ccf-b712-b1b2dbb60634 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81767) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 16:13:58.161561 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4281a343-1597-4ccf-b712-b1b2dbb60634 None None] Validating token access rules against request {{(pid=81767) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 16:13:58.161561 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9751011a-4d11-4171-b0b8-60f8363af964 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a98b1e4431940ab8d1fc95df2858a6c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81767) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 16:13:58.162372 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9751011a-4d11-4171-b0b8-60f8363af964 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:13:58.162704 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9751011a-4d11-4171-b0b8-60f8363af964 admin admin] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:13:58.162979 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9751011a-4d11-4171-b0b8-60f8363af964 admin admin] PATH_INFO: `/v3/projects/9a7fbb564d4e4c2c8fd1fd4b02a6c029` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:13:58.164209 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9751011a-4d11-4171-b0b8-60f8363af964 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9a7fbb564d4e4c2c8fd1fd4b02a6c029)` {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 16:13:58.166162 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9751011a-4d11-4171-b0b8-60f8363af964 admin admin] RBAC: Authorization granted {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 16:13:58.282643 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.notifications [None req-9751011a-4d11-4171-b0b8-60f8363af964 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9a7fbb564d4e4c2c8fd1fd4b02a6c029', 'request_id': 'req-9751011a-4d11-4171-b0b8-60f8363af964'} {{(pid=81767) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 16:13:58.288819 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.notifications [None req-9751011a-4d11-4171-b0b8-60f8363af964 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9a7fbb564d4e4c2c8fd1fd4b02a6c029', 'request_id': 'req-9751011a-4d11-4171-b0b8-60f8363af964'} {{(pid=81767) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 16:13:58.337540 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ae5e12b-85a6-44e8-93ef-e18ee78c4f66 service neutron] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 16:13:58.339052 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-1ae5e12b-85a6-44e8-93ef-e18ee78c4f66 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 16:13:58.352284 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ae5e12b-85a6-44e8-93ef-e18ee78c4f66 service neutron] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 16:13:58.376867 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.notifications [None req-9751011a-4d11-4171-b0b8-60f8363af964 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9a7fbb564d4e4c2c8fd1fd4b02a6c029', 'request_id': 'req-9751011a-4d11-4171-b0b8-60f8363af964'} {{(pid=81767) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 16:13:58.390342 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.notifications [None req-9751011a-4d11-4171-b0b8-60f8363af964 admin admin] The token cache is being invalidate because project 9a7fbb564d4e4c2c8fd1fd4b02a6c029 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81767) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 22 16:13:58.391548 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.notifications [None req-9751011a-4d11-4171-b0b8-60f8363af964 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81767) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 16:13:58.392572 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.notifications [None req-9751011a-4d11-4171-b0b8-60f8363af964 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81767) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 16:13:58.394312 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 1276/2591] 10.222.0.15 () {58 vars in 1233 bytes} [Mon Aug 22 16:13:58 2022] DELETE /identity/v3/projects/9a7fbb564d4e4c2c8fd1fd4b02a6c029 => generated 0 bytes in 261 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 16:13:58.420647 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dda9a1e7-31d6-4741-a2cf-04a595624d90 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a98b1e4431940ab8d1fc95df2858a6c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 16:13:58.421601 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dda9a1e7-31d6-4741-a2cf-04a595624d90 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:13:58.422075 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dda9a1e7-31d6-4741-a2cf-04a595624d90 admin admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:13:58.422703 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dda9a1e7-31d6-4741-a2cf-04a595624d90 admin admin] PATH_INFO: `/v3/users/a257f47de3544e2d8bd1e7aa55bf9495` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:13:58.425016 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dda9a1e7-31d6-4741-a2cf-04a595624d90 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a257f47de3544e2d8bd1e7aa55bf9495)` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 16:13:58.427705 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dda9a1e7-31d6-4741-a2cf-04a595624d90 admin admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 16:13:58.731084 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-dda9a1e7-31d6-4741-a2cf-04a595624d90 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a257f47de3544e2d8bd1e7aa55bf9495'} {{(pid=81768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 16:13:58.732821 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-dda9a1e7-31d6-4741-a2cf-04a595624d90 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a257f47de3544e2d8bd1e7aa55bf9495'} {{(pid=81768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 16:13:58.733676 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-dda9a1e7-31d6-4741-a2cf-04a595624d90 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a257f47de3544e2d8bd1e7aa55bf9495'} {{(pid=81768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 16:13:58.742920 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-dda9a1e7-31d6-4741-a2cf-04a595624d90 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a257f47de3544e2d8bd1e7aa55bf9495'} {{(pid=81768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 16:13:58.836547 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-dda9a1e7-31d6-4741-a2cf-04a595624d90 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a257f47de3544e2d8bd1e7aa55bf9495'} {{(pid=81768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 16:13:58.866895 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 1316/2592] 10.222.0.15 () {58 vars in 1224 bytes} [Mon Aug 22 16:13:58 2022] DELETE /identity/v3/users/a257f47de3544e2d8bd1e7aa55bf9495 => generated 0 bytes in 530 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 22 16:13:58.872912 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c2191f8-0559-405b-8f73-3f5389d76733 None None] REQUEST_METHOD: `POST` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:13:58.873276 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c2191f8-0559-405b-8f73-3f5389d76733 None None] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:13:58.873653 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c2191f8-0559-405b-8f73-3f5389d76733 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:13:58.908419 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.auth.core [None req-3c2191f8-0559-405b-8f73-3f5389d76733 None None] MFA Rules not processed for user `6b2199ae0fb0480aa917ada391f5ec54`. Rule list: `[]` (Enabled: `True`). {{(pid=81767) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 16:13:58.909683 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.fernet_utils [None req-3c2191f8-0559-405b-8f73-3f5389d76733 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81767) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 16:13:58.936713 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG dbcounter [-] [81767] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81767) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 16:13:58.978466 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 1277/2593] 10.222.0.15 () {58 vars in 958 bytes} [Mon Aug 22 16:13:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 22 16:13:58.990969 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dda9a1e7-31d6-4741-a2cf-04a595624d90 admin admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 16:13:58.993719 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-dda9a1e7-31d6-4741-a2cf-04a595624d90 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 16:13:59.038865 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 16:13:59.068884 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dda9a1e7-31d6-4741-a2cf-04a595624d90 admin admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 16:13:59.072668 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-257e8f66-bcb4-4122-8664-f0602f8e5414 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '234687adad594f1aa45775a565ffbdc7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4ee0413f9ef4d4fb1f95a660babe090', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 16:13:59.073279 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-257e8f66-bcb4-4122-8664-f0602f8e5414 service neutron] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:13:59.073596 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-257e8f66-bcb4-4122-8664-f0602f8e5414 service neutron] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:13:59.073836 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-257e8f66-bcb4-4122-8664-f0602f8e5414 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:13:59.084321 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-257e8f66-bcb4-4122-8664-f0602f8e5414 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 16:13:59.086058 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-257e8f66-bcb4-4122-8664-f0602f8e5414 service neutron] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 16:13:59.099386 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 1317/2594] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Aug 22 16:13:58 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 16:13:59.584455 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c2191f8-0559-405b-8f73-3f5389d76733 None None] Authenticating user token {{(pid=81767) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 16:13:59.587746 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.fernet_utils [None req-3c2191f8-0559-405b-8f73-3f5389d76733 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81767) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 16:13:59.624800 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c2191f8-0559-405b-8f73-3f5389d76733 None None] Validating token access rules against request {{(pid=81767) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 16:13:59.630518 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-257e8f66-bcb4-4122-8664-f0602f8e5414 service neutron] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 16:13:59.631975 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-257e8f66-bcb4-4122-8664-f0602f8e5414 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 16:13:59.637439 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd541a3e-6705-40f5-a002-e6bd87bd9534 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a98b1e4431940ab8d1fc95df2858a6c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81767) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 16:13:59.638102 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd541a3e-6705-40f5-a002-e6bd87bd9534 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:13:59.638437 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd541a3e-6705-40f5-a002-e6bd87bd9534 admin admin] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:13:59.638666 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd541a3e-6705-40f5-a002-e6bd87bd9534 admin admin] PATH_INFO: `/v3/projects/2399c81627ad4305bcea2f29aaac5d4e` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:13:59.639993 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd541a3e-6705-40f5-a002-e6bd87bd9534 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2399c81627ad4305bcea2f29aaac5d4e)` {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 16:13:59.642531 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd541a3e-6705-40f5-a002-e6bd87bd9534 admin admin] RBAC: Authorization granted {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 16:13:59.647864 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-257e8f66-bcb4-4122-8664-f0602f8e5414 service neutron] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 16:13:59.654167 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-842e507e-b8ff-4c59-a694-2f58d5569897 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a98b1e4431940ab8d1fc95df2858a6c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 16:13:59.654729 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-842e507e-b8ff-4c59-a694-2f58d5569897 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:13:59.655045 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-842e507e-b8ff-4c59-a694-2f58d5569897 admin admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:13:59.655331 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-842e507e-b8ff-4c59-a694-2f58d5569897 admin admin] PATH_INFO: `/v3/users/f2c372b4c45648c7846e04c721fc36a0` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:13:59.656547 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-842e507e-b8ff-4c59-a694-2f58d5569897 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f2c372b4c45648c7846e04c721fc36a0)` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 16:13:59.659981 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-842e507e-b8ff-4c59-a694-2f58d5569897 admin admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 16:13:59.920629 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-842e507e-b8ff-4c59-a694-2f58d5569897 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f2c372b4c45648c7846e04c721fc36a0'} {{(pid=81768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 16:13:59.921194 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-842e507e-b8ff-4c59-a694-2f58d5569897 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f2c372b4c45648c7846e04c721fc36a0'} {{(pid=81768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 16:13:59.921800 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-842e507e-b8ff-4c59-a694-2f58d5569897 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f2c372b4c45648c7846e04c721fc36a0'} {{(pid=81768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 16:13:59.924085 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.notifications [None req-bd541a3e-6705-40f5-a002-e6bd87bd9534 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2399c81627ad4305bcea2f29aaac5d4e', 'request_id': 'req-bd541a3e-6705-40f5-a002-e6bd87bd9534'} {{(pid=81767) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 16:13:59.932411 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-842e507e-b8ff-4c59-a694-2f58d5569897 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f2c372b4c45648c7846e04c721fc36a0'} {{(pid=81768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 16:13:59.936311 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.notifications [None req-bd541a3e-6705-40f5-a002-e6bd87bd9534 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2399c81627ad4305bcea2f29aaac5d4e', 'request_id': 'req-bd541a3e-6705-40f5-a002-e6bd87bd9534'} {{(pid=81767) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 16:14:00.026392 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-842e507e-b8ff-4c59-a694-2f58d5569897 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f2c372b4c45648c7846e04c721fc36a0'} {{(pid=81768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 16:14:00.046058 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 1318/2595] 10.222.0.15 () {58 vars in 1224 bytes} [Mon Aug 22 16:13:59 2022] DELETE /identity/v3/users/f2c372b4c45648c7846e04c721fc36a0 => generated 0 bytes in 416 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 22 16:14:00.050519 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4625e1ef-67e7-47cf-a38a-494ed11d47e1 None None] REQUEST_METHOD: `POST` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:14:00.058949 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4625e1ef-67e7-47cf-a38a-494ed11d47e1 None None] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:14:00.058949 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4625e1ef-67e7-47cf-a38a-494ed11d47e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:14:00.072273 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.auth.core [None req-4625e1ef-67e7-47cf-a38a-494ed11d47e1 None None] MFA Rules not processed for user `6b2199ae0fb0480aa917ada391f5ec54`. Rule list: `[]` (Enabled: `True`). {{(pid=81768) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 16:14:00.073519 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-4625e1ef-67e7-47cf-a38a-494ed11d47e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 16:14:00.145384 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.notifications [None req-bd541a3e-6705-40f5-a002-e6bd87bd9534 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2399c81627ad4305bcea2f29aaac5d4e', 'request_id': 'req-bd541a3e-6705-40f5-a002-e6bd87bd9534'} {{(pid=81767) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 16:14:00.150677 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 1319/2596] 10.222.0.15 () {58 vars in 958 bytes} [Mon Aug 22 16:14:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 22 16:14:00.162261 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.notifications [None req-bd541a3e-6705-40f5-a002-e6bd87bd9534 admin admin] The token cache is being invalidate because project 2399c81627ad4305bcea2f29aaac5d4e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81767) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 22 16:14:00.163042 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.notifications [None req-bd541a3e-6705-40f5-a002-e6bd87bd9534 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81767) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 16:14:00.163996 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4625e1ef-67e7-47cf-a38a-494ed11d47e1 None None] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 16:14:00.164340 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.notifications [None req-bd541a3e-6705-40f5-a002-e6bd87bd9534 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81767) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 16:14:00.166903 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 1278/2597] 10.222.0.15 () {58 vars in 1233 bytes} [Mon Aug 22 16:13:59 2022] DELETE /identity/v3/projects/2399c81627ad4305bcea2f29aaac5d4e => generated 0 bytes in 585 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 16:14:00.167739 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-4625e1ef-67e7-47cf-a38a-494ed11d47e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 16:14:00.175488 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2040159-8ef5-44dc-8d66-91f001a40d02 None None] REQUEST_METHOD: `POST` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:14:00.176179 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2040159-8ef5-44dc-8d66-91f001a40d02 None None] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:14:00.176840 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2040159-8ef5-44dc-8d66-91f001a40d02 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:14:00.212465 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.auth.core [None req-d2040159-8ef5-44dc-8d66-91f001a40d02 None None] MFA Rules not processed for user `6b2199ae0fb0480aa917ada391f5ec54`. Rule list: `[]` (Enabled: `True`). {{(pid=81767) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 16:14:00.214062 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.fernet_utils [None req-d2040159-8ef5-44dc-8d66-91f001a40d02 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81767) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 16:14:00.229502 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 1279/2598] 10.222.0.15 () {58 vars in 958 bytes} [Mon Aug 22 16:14:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 57 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 16:14:00.239609 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb41caaf-01d0-4e8b-bece-8430f54a47c9 None None] REQUEST_METHOD: `POST` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:14:00.240619 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb41caaf-01d0-4e8b-bece-8430f54a47c9 None None] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:14:00.240993 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb41caaf-01d0-4e8b-bece-8430f54a47c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:14:00.254328 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 16:14:00.262296 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.auth.core [None req-bb41caaf-01d0-4e8b-bece-8430f54a47c9 None None] MFA Rules not processed for user `6b2199ae0fb0480aa917ada391f5ec54`. Rule list: `[]` (Enabled: `True`). {{(pid=81767) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 16:14:00.263641 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.fernet_utils [None req-bb41caaf-01d0-4e8b-bece-8430f54a47c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81767) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 16:14:00.264166 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4625e1ef-67e7-47cf-a38a-494ed11d47e1 None None] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 16:14:00.268232 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4321829-4d99-4a95-97f5-35d155c1a60f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '234687adad594f1aa45775a565ffbdc7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4ee0413f9ef4d4fb1f95a660babe090', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 16:14:00.268870 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4321829-4d99-4a95-97f5-35d155c1a60f service neutron] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:14:00.269285 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4321829-4d99-4a95-97f5-35d155c1a60f service neutron] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:14:00.269468 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4321829-4d99-4a95-97f5-35d155c1a60f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:14:00.271298 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-a4321829-4d99-4a95-97f5-35d155c1a60f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 16:14:00.275595 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 1280/2599] 10.222.0.15 () {58 vars in 958 bytes} [Mon Aug 22 16:14:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 16:14:00.276736 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb41caaf-01d0-4e8b-bece-8430f54a47c9 None None] Authenticating user token {{(pid=81767) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 16:14:00.286822 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb41caaf-01d0-4e8b-bece-8430f54a47c9 None None] Validating token access rules against request {{(pid=81767) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 16:14:00.289689 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4321829-4d99-4a95-97f5-35d155c1a60f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 16:14:00.291364 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4321829-4d99-4a95-97f5-35d155c1a60f service neutron] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 16:14:00.292403 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc6cd8fb-7076-44c1-a325-f9c8e1e6ca22 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a98b1e4431940ab8d1fc95df2858a6c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81767) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 16:14:00.293010 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc6cd8fb-7076-44c1-a325-f9c8e1e6ca22 admin admin] REQUEST_METHOD: `GET` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:14:00.293457 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc6cd8fb-7076-44c1-a325-f9c8e1e6ca22 admin admin] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:14:00.293725 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc6cd8fb-7076-44c1-a325-f9c8e1e6ca22 admin admin] PATH_INFO: `/v3/domains` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:14:00.294380 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc6cd8fb-7076-44c1-a325-f9c8e1e6ca22 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81767) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 16:14:00.294681 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc6cd8fb-7076-44c1-a325-f9c8e1e6ca22 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 16:14:00.297623 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc6cd8fb-7076-44c1-a325-f9c8e1e6ca22 admin admin] RBAC: Authorization granted {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 16:14:00.307007 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 1320/2600] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Aug 22 16:14:00 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 144 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 16:14:00.312303 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 1281/2601] 10.222.0.15 () {58 vars in 1153 bytes} [Mon Aug 22 16:14:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 22 16:14:00.317313 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4321829-4d99-4a95-97f5-35d155c1a60f service neutron] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 16:14:00.327771 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4321829-4d99-4a95-97f5-35d155c1a60f service neutron] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 16:14:00.334230 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a40685b-4393-4fbd-bd5e-cd837e17f7c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a98b1e4431940ab8d1fc95df2858a6c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 16:14:00.335112 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a40685b-4393-4fbd-bd5e-cd837e17f7c8 admin admin] REQUEST_METHOD: `POST` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:14:00.335425 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a40685b-4393-4fbd-bd5e-cd837e17f7c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:14:00.335698 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a40685b-4393-4fbd-bd5e-cd837e17f7c8 admin admin] PATH_INFO: `/v3/projects` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:14:00.336681 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a40685b-4393-4fbd-bd5e-cd837e17f7c8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 16:14:00.338879 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a40685b-4393-4fbd-bd5e-cd837e17f7c8 admin admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 16:14:00.446312 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 1321/2602] 10.222.0.15 () {60 vars in 1153 bytes} [Mon Aug 22 16:14:00 2022] POST /identity/v3/projects => generated 377 bytes in 129 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 22 16:14:00.450589 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc6cd8fb-7076-44c1-a325-f9c8e1e6ca22 admin admin] Authenticating user token {{(pid=81767) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 16:14:00.461924 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc6cd8fb-7076-44c1-a325-f9c8e1e6ca22 admin admin] Validating token access rules against request {{(pid=81767) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 16:14:00.527250 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3df2e48-b0d1-4659-b7a5-cc03f0421ff0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a98b1e4431940ab8d1fc95df2858a6c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81767) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 16:14:00.527940 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3df2e48-b0d1-4659-b7a5-cc03f0421ff0 admin admin] REQUEST_METHOD: `POST` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:14:00.528218 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3df2e48-b0d1-4659-b7a5-cc03f0421ff0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:14:00.528505 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3df2e48-b0d1-4659-b7a5-cc03f0421ff0 admin admin] PATH_INFO: `/v3/users` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:14:00.529199 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3df2e48-b0d1-4659-b7a5-cc03f0421ff0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 16:14:00.531223 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3df2e48-b0d1-4659-b7a5-cc03f0421ff0 admin admin] RBAC: Authorization granted {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 16:14:00.637732 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 1282/2603] 10.222.0.15 () {60 vars in 1144 bytes} [Mon Aug 22 16:14:00 2022] POST /identity/v3/users => generated 289 bytes in 188 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 22 16:14:00.642791 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a40685b-4393-4fbd-bd5e-cd837e17f7c8 admin admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 16:14:00.655386 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a40685b-4393-4fbd-bd5e-cd837e17f7c8 admin admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 16:14:00.661602 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9c59848-cbac-4dc3-b12e-274e74da6824 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a98b1e4431940ab8d1fc95df2858a6c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 16:14:00.662238 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9c59848-cbac-4dc3-b12e-274e74da6824 admin admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:14:00.662518 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9c59848-cbac-4dc3-b12e-274e74da6824 admin admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:14:00.662789 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9c59848-cbac-4dc3-b12e-274e74da6824 admin admin] PATH_INFO: `/v3/roles` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:14:00.663513 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9c59848-cbac-4dc3-b12e-274e74da6824 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 16:14:00.665341 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9c59848-cbac-4dc3-b12e-274e74da6824 admin admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 16:14:00.679251 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 1322/2604] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Aug 22 16:14:00 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 22 16:14:00.685929 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3df2e48-b0d1-4659-b7a5-cc03f0421ff0 admin admin] Authenticating user token {{(pid=81767) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 16:14:00.704709 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3df2e48-b0d1-4659-b7a5-cc03f0421ff0 admin admin] Validating token access rules against request {{(pid=81767) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 16:14:00.710383 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-def4220a-baa5-49d0-8b18-b01ba7626923 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a98b1e4431940ab8d1fc95df2858a6c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81767) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 16:14:00.711087 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-def4220a-baa5-49d0-8b18-b01ba7626923 admin admin] REQUEST_METHOD: `GET` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:14:00.711410 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-def4220a-baa5-49d0-8b18-b01ba7626923 admin admin] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:14:00.711686 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-def4220a-baa5-49d0-8b18-b01ba7626923 admin admin] PATH_INFO: `/v3/roles` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:14:00.712393 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-def4220a-baa5-49d0-8b18-b01ba7626923 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 16:14:00.718499 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-def4220a-baa5-49d0-8b18-b01ba7626923 admin admin] RBAC: Authorization granted {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 16:14:00.727458 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 1283/2605] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Aug 22 16:14:00 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 22 16:14:00.732235 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9c59848-cbac-4dc3-b12e-274e74da6824 admin admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 16:14:00.747163 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9c59848-cbac-4dc3-b12e-274e74da6824 admin admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 16:14:00.752515 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a931365d-f9bd-4cc3-9007-86e165414c7d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a98b1e4431940ab8d1fc95df2858a6c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 16:14:00.764300 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a931365d-f9bd-4cc3-9007-86e165414c7d admin admin] REQUEST_METHOD: `PUT` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:14:00.764300 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a931365d-f9bd-4cc3-9007-86e165414c7d admin admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:14:00.764300 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a931365d-f9bd-4cc3-9007-86e165414c7d admin admin] PATH_INFO: `/v3/projects/02aae2708d7b41e7952cfba658be47db/users/4e4eff0c95194682ade3e24aec451cd7/roles/e6fc1ee3fb974125ba9b4fe55b12b45e` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:14:00.772199 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a931365d-f9bd-4cc3-9007-86e165414c7d admin admin] RBAC: Authorizing `identity:create_grant(project_id=02aae2708d7b41e7952cfba658be47db, user_id=4e4eff0c95194682ade3e24aec451cd7, role_id=e6fc1ee3fb974125ba9b4fe55b12b45e)` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 16:14:00.775154 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a931365d-f9bd-4cc3-9007-86e165414c7d admin admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 16:14:00.927580 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-def4220a-baa5-49d0-8b18-b01ba7626923 admin admin] Authenticating user token {{(pid=81767) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 16:14:00.929409 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.fernet_utils [None req-def4220a-baa5-49d0-8b18-b01ba7626923 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81767) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 16:14:00.940534 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG dbcounter [-] [81767] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=81767) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 16:14:00.955916 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-def4220a-baa5-49d0-8b18-b01ba7626923 admin admin] Validating token access rules against request {{(pid=81767) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 16:14:00.967687 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe22ae88-6f7a-461f-ae0b-58ea65cc50ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a98b1e4431940ab8d1fc95df2858a6c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81767) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 16:14:00.967687 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe22ae88-6f7a-461f-ae0b-58ea65cc50ec admin admin] REQUEST_METHOD: `DELETE` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:14:00.967687 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe22ae88-6f7a-461f-ae0b-58ea65cc50ec admin admin] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:14:00.967687 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe22ae88-6f7a-461f-ae0b-58ea65cc50ec admin admin] PATH_INFO: `/v3/projects/e7a1c4960137485fb7f5a3cc7c1cb619` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:14:00.968757 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe22ae88-6f7a-461f-ae0b-58ea65cc50ec admin admin] RBAC: Authorizing `identity:delete_project(project_id=e7a1c4960137485fb7f5a3cc7c1cb619)` {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 16:14:00.971521 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 1323/2606] 10.222.0.15 () {60 vars in 1483 bytes} [Mon Aug 22 16:14:00 2022] PUT /identity/v3/projects/02aae2708d7b41e7952cfba658be47db/users/4e4eff0c95194682ade3e24aec451cd7/roles/e6fc1ee3fb974125ba9b4fe55b12b45e => generated 0 bytes in 240 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 22 16:14:00.971651 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe22ae88-6f7a-461f-ae0b-58ea65cc50ec admin admin] RBAC: Authorization granted {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 16:14:00.972375 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a931365d-f9bd-4cc3-9007-86e165414c7d admin admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 16:14:00.974480 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-a931365d-f9bd-4cc3-9007-86e165414c7d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 16:14:01.044527 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a931365d-f9bd-4cc3-9007-86e165414c7d admin admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 16:14:01.049195 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f093bef-f42f-4d57-a469-5b6060d5dee4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a98b1e4431940ab8d1fc95df2858a6c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 16:14:01.049817 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f093bef-f42f-4d57-a469-5b6060d5dee4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:14:01.050094 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f093bef-f42f-4d57-a469-5b6060d5dee4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:14:01.050366 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f093bef-f42f-4d57-a469-5b6060d5dee4 admin admin] PATH_INFO: `/v3/users/26d4cd03ad1e4be081210b68e2c6d0d8` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:14:01.051551 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f093bef-f42f-4d57-a469-5b6060d5dee4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=26d4cd03ad1e4be081210b68e2c6d0d8)` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 16:14:01.053481 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f093bef-f42f-4d57-a469-5b6060d5dee4 admin admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 16:14:01.086187 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.notifications [None req-fe22ae88-6f7a-461f-ae0b-58ea65cc50ec admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e7a1c4960137485fb7f5a3cc7c1cb619', 'request_id': 'req-fe22ae88-6f7a-461f-ae0b-58ea65cc50ec'} {{(pid=81767) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 16:14:01.091568 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.notifications [None req-fe22ae88-6f7a-461f-ae0b-58ea65cc50ec admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e7a1c4960137485fb7f5a3cc7c1cb619', 'request_id': 'req-fe22ae88-6f7a-461f-ae0b-58ea65cc50ec'} {{(pid=81767) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 16:14:01.224764 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.notifications [None req-fe22ae88-6f7a-461f-ae0b-58ea65cc50ec admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e7a1c4960137485fb7f5a3cc7c1cb619', 'request_id': 'req-fe22ae88-6f7a-461f-ae0b-58ea65cc50ec'} {{(pid=81767) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 16:14:01.237856 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.notifications [None req-fe22ae88-6f7a-461f-ae0b-58ea65cc50ec admin admin] The token cache is being invalidate because project e7a1c4960137485fb7f5a3cc7c1cb619 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81767) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 22 16:14:01.238151 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.notifications [None req-fe22ae88-6f7a-461f-ae0b-58ea65cc50ec admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81767) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 16:14:01.238951 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.notifications [None req-fe22ae88-6f7a-461f-ae0b-58ea65cc50ec admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81767) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 16:14:01.240510 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 1284/2607] 10.222.0.15 () {58 vars in 1233 bytes} [Mon Aug 22 16:14:00 2022] DELETE /identity/v3/projects/e7a1c4960137485fb7f5a3cc7c1cb619 => generated 0 bytes in 314 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 22 16:14:01.241900 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d99c6a63-5e25-4265-b316-a362cfff7030 None None] REQUEST_METHOD: `POST` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:14:01.242202 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d99c6a63-5e25-4265-b316-a362cfff7030 None None] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:14:01.242463 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d99c6a63-5e25-4265-b316-a362cfff7030 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:14:01.261944 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.auth.core [None req-d99c6a63-5e25-4265-b316-a362cfff7030 None None] MFA Rules not processed for user `6b2199ae0fb0480aa917ada391f5ec54`. Rule list: `[]` (Enabled: `True`). {{(pid=81767) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 16:14:01.263125 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.fernet_utils [None req-d99c6a63-5e25-4265-b316-a362cfff7030 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81767) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 16:14:01.281608 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-1f093bef-f42f-4d57-a469-5b6060d5dee4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '26d4cd03ad1e4be081210b68e2c6d0d8'} {{(pid=81768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 16:14:01.282423 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-1f093bef-f42f-4d57-a469-5b6060d5dee4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '26d4cd03ad1e4be081210b68e2c6d0d8'} {{(pid=81768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 16:14:01.284083 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-1f093bef-f42f-4d57-a469-5b6060d5dee4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '26d4cd03ad1e4be081210b68e2c6d0d8'} {{(pid=81768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 16:14:01.292886 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-1f093bef-f42f-4d57-a469-5b6060d5dee4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '26d4cd03ad1e4be081210b68e2c6d0d8'} {{(pid=81768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 16:14:01.333097 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 1285/2608] 10.222.0.15 () {58 vars in 958 bytes} [Mon Aug 22 16:14:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 16:14:01.334056 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d99c6a63-5e25-4265-b316-a362cfff7030 None None] Authenticating user token {{(pid=81767) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 16:14:01.335728 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.fernet_utils [None req-d99c6a63-5e25-4265-b316-a362cfff7030 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81767) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 16:14:01.348673 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d99c6a63-5e25-4265-b316-a362cfff7030 None None] Validating token access rules against request {{(pid=81767) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 16:14:01.353639 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44e3a78d-ef51-4eb9-921e-e70394ba983e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a98b1e4431940ab8d1fc95df2858a6c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81767) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 16:14:01.354317 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44e3a78d-ef51-4eb9-921e-e70394ba983e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:14:01.354587 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44e3a78d-ef51-4eb9-921e-e70394ba983e admin admin] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:14:01.354863 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44e3a78d-ef51-4eb9-921e-e70394ba983e admin admin] PATH_INFO: `/v3/users/3052a42ed8bb4342b0460d94332f6a6e` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:14:01.356846 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44e3a78d-ef51-4eb9-921e-e70394ba983e admin admin] RBAC: Authorizing `identity:delete_user(user_id=3052a42ed8bb4342b0460d94332f6a6e)` {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 16:14:01.358912 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44e3a78d-ef51-4eb9-921e-e70394ba983e admin admin] RBAC: Authorization granted {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 16:14:01.416642 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-1f093bef-f42f-4d57-a469-5b6060d5dee4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '26d4cd03ad1e4be081210b68e2c6d0d8'} {{(pid=81768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 16:14:01.435056 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 1324/2609] 10.222.0.15 () {58 vars in 1224 bytes} [Mon Aug 22 16:14:00 2022] DELETE /identity/v3/users/26d4cd03ad1e4be081210b68e2c6d0d8 => generated 0 bytes in 463 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 16:14:01.435884 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f093bef-f42f-4d57-a469-5b6060d5dee4 admin admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 16:14:01.437244 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-1f093bef-f42f-4d57-a469-5b6060d5dee4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 16:14:01.450993 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=87,keystone:INSERT=3,keystone:DELETE=10 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 16:14:01.506436 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f093bef-f42f-4d57-a469-5b6060d5dee4 admin admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 16:14:01.512651 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98e382b7-057a-4869-8fbc-660a2c5e015d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a98b1e4431940ab8d1fc95df2858a6c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 16:14:01.513966 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e382b7-057a-4869-8fbc-660a2c5e015d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:14:01.514391 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e382b7-057a-4869-8fbc-660a2c5e015d admin admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:14:01.514728 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e382b7-057a-4869-8fbc-660a2c5e015d admin admin] PATH_INFO: `/v3/users/c23424d1cdf8460bb5251646b64bb154` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:14:01.517893 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98e382b7-057a-4869-8fbc-660a2c5e015d admin admin] RBAC: Authorizing `identity:delete_user(user_id=c23424d1cdf8460bb5251646b64bb154)` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 16:14:01.547005 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98e382b7-057a-4869-8fbc-660a2c5e015d admin admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 16:14:01.737094 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.notifications [None req-44e3a78d-ef51-4eb9-921e-e70394ba983e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3052a42ed8bb4342b0460d94332f6a6e'} {{(pid=81767) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 16:14:01.737094 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.notifications [None req-44e3a78d-ef51-4eb9-921e-e70394ba983e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3052a42ed8bb4342b0460d94332f6a6e'} {{(pid=81767) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 16:14:01.804948 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.notifications [None req-44e3a78d-ef51-4eb9-921e-e70394ba983e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3052a42ed8bb4342b0460d94332f6a6e'} {{(pid=81767) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 16:14:01.813853 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.notifications [None req-44e3a78d-ef51-4eb9-921e-e70394ba983e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3052a42ed8bb4342b0460d94332f6a6e'} {{(pid=81767) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 16:14:01.839816 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.notifications [None req-44e3a78d-ef51-4eb9-921e-e70394ba983e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3052a42ed8bb4342b0460d94332f6a6e'} {{(pid=81767) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 16:14:01.850206 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG dbcounter [-] [81767] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=81767) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 16:14:01.856582 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 1286/2610] 10.222.0.15 () {58 vars in 1224 bytes} [Mon Aug 22 16:14:01 2022] DELETE /identity/v3/users/3052a42ed8bb4342b0460d94332f6a6e => generated 0 bytes in 523 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 16:14:01.857686 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44e3a78d-ef51-4eb9-921e-e70394ba983e admin admin] Authenticating user token {{(pid=81767) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 16:14:01.863641 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.fernet_utils [None req-44e3a78d-ef51-4eb9-921e-e70394ba983e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81767) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 16:14:01.873088 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-98e382b7-057a-4869-8fbc-660a2c5e015d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c23424d1cdf8460bb5251646b64bb154'} {{(pid=81768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 16:14:01.874460 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-98e382b7-057a-4869-8fbc-660a2c5e015d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c23424d1cdf8460bb5251646b64bb154'} {{(pid=81768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 16:14:01.875187 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-98e382b7-057a-4869-8fbc-660a2c5e015d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c23424d1cdf8460bb5251646b64bb154'} {{(pid=81768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 16:14:01.884963 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-98e382b7-057a-4869-8fbc-660a2c5e015d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c23424d1cdf8460bb5251646b64bb154'} {{(pid=81768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 16:14:01.928991 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44e3a78d-ef51-4eb9-921e-e70394ba983e admin admin] Validating token access rules against request {{(pid=81767) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 16:14:01.933399 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99274bfd-a147-41e3-804d-8fb512bc64dc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '234687adad594f1aa45775a565ffbdc7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4ee0413f9ef4d4fb1f95a660babe090', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81767) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 16:14:01.934014 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99274bfd-a147-41e3-804d-8fb512bc64dc service neutron] REQUEST_METHOD: `GET` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:14:01.934293 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99274bfd-a147-41e3-804d-8fb512bc64dc service neutron] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:14:01.934624 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99274bfd-a147-41e3-804d-8fb512bc64dc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:14:01.936522 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.fernet_utils [None req-99274bfd-a147-41e3-804d-8fb512bc64dc service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81767) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 16:14:01.971771 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-98e382b7-057a-4869-8fbc-660a2c5e015d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c23424d1cdf8460bb5251646b64bb154'} {{(pid=81768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 16:14:01.989815 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 1325/2611] 10.222.0.15 () {58 vars in 1224 bytes} [Mon Aug 22 16:14:01 2022] DELETE /identity/v3/users/c23424d1cdf8460bb5251646b64bb154 => generated 0 bytes in 554 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 16:14:01.991313 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed0cefbd-1299-4b58-9fcf-2dda5b721314 None None] REQUEST_METHOD: `POST` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:14:01.991696 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed0cefbd-1299-4b58-9fcf-2dda5b721314 None None] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:14:01.992060 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed0cefbd-1299-4b58-9fcf-2dda5b721314 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:14:02.007932 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99274bfd-a147-41e3-804d-8fb512bc64dc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 16:14:02.009402 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99274bfd-a147-41e3-804d-8fb512bc64dc service neutron] RBAC: Authorization granted {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 16:14:02.022592 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 1287/2612] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Aug 22 16:14:01 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 16:14:02.023730 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29480775-b77c-4f29-af02-9c946b259ce7 None None] REQUEST_METHOD: `POST` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:14:02.024026 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29480775-b77c-4f29-af02-9c946b259ce7 None None] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:14:02.024293 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29480775-b77c-4f29-af02-9c946b259ce7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:14:02.034566 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.auth.core [None req-ed0cefbd-1299-4b58-9fcf-2dda5b721314 None None] MFA Rules not processed for user `6b2199ae0fb0480aa917ada391f5ec54`. Rule list: `[]` (Enabled: `True`). {{(pid=81768) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 16:14:02.039498 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-ed0cefbd-1299-4b58-9fcf-2dda5b721314 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 16:14:02.046797 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.auth.core [None req-29480775-b77c-4f29-af02-9c946b259ce7 None None] MFA Rules not processed for user `6b2199ae0fb0480aa917ada391f5ec54`. Rule list: `[]` (Enabled: `True`). {{(pid=81767) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 16:14:02.048128 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.fernet_utils [None req-29480775-b77c-4f29-af02-9c946b259ce7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81767) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 16:14:02.138687 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 1326/2613] 10.222.0.15 () {58 vars in 958 bytes} [Mon Aug 22 16:14:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 16:14:02.139853 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 1288/2614] 10.222.0.15 () {58 vars in 958 bytes} [Mon Aug 22 16:14:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 16:14:02.148862 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed0cefbd-1299-4b58-9fcf-2dda5b721314 None None] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 16:14:02.150387 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-ed0cefbd-1299-4b58-9fcf-2dda5b721314 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 16:14:02.152338 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29480775-b77c-4f29-af02-9c946b259ce7 None None] Authenticating user token {{(pid=81767) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 16:14:02.153713 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.fernet_utils [None req-29480775-b77c-4f29-af02-9c946b259ce7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81767) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 16:14:02.195037 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 16:14:02.222234 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed0cefbd-1299-4b58-9fcf-2dda5b721314 None None] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 16:14:02.226600 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29480775-b77c-4f29-af02-9c946b259ce7 None None] Validating token access rules against request {{(pid=81767) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 16:14:02.226843 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-749c9baf-9fac-4b70-8d3b-6f6fb202d723 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '234687adad594f1aa45775a565ffbdc7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4ee0413f9ef4d4fb1f95a660babe090', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 16:14:02.227478 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-749c9baf-9fac-4b70-8d3b-6f6fb202d723 service neutron] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:14:02.228275 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-749c9baf-9fac-4b70-8d3b-6f6fb202d723 service neutron] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:14:02.228618 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-749c9baf-9fac-4b70-8d3b-6f6fb202d723 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:14:02.231370 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9df1a18b-43cb-4922-80d1-d13ef64451f5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '234687adad594f1aa45775a565ffbdc7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4ee0413f9ef4d4fb1f95a660babe090', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81767) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 16:14:02.231992 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9df1a18b-43cb-4922-80d1-d13ef64451f5 service neutron] REQUEST_METHOD: `GET` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:14:02.232286 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9df1a18b-43cb-4922-80d1-d13ef64451f5 service neutron] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:14:02.232547 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9df1a18b-43cb-4922-80d1-d13ef64451f5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:14:02.245258 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG dbcounter [-] [81767] Writing DB stats keystone:SELECT=100 {{(pid=81767) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 16:14:02.245258 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9df1a18b-43cb-4922-80d1-d13ef64451f5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 16:14:02.246776 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9df1a18b-43cb-4922-80d1-d13ef64451f5 service neutron] RBAC: Authorization granted {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 16:14:02.256106 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-749c9baf-9fac-4b70-8d3b-6f6fb202d723 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 16:14:02.257587 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-749c9baf-9fac-4b70-8d3b-6f6fb202d723 service neutron] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 16:14:02.261305 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 1289/2615] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Aug 22 16:14:02 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 16:14:02.275651 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 1327/2616] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Aug 22 16:14:02 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 127 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 16:14:02.550768 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9df1a18b-43cb-4922-80d1-d13ef64451f5 service neutron] Authenticating user token {{(pid=81767) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 16:14:02.552900 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.fernet_utils [None req-9df1a18b-43cb-4922-80d1-d13ef64451f5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81767) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 16:14:02.574475 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9df1a18b-43cb-4922-80d1-d13ef64451f5 service neutron] Validating token access rules against request {{(pid=81767) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 16:14:02.582407 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acd18a0f-c0cc-4a62-8564-30f983fc06a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a98b1e4431940ab8d1fc95df2858a6c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81767) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 16:14:02.583230 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acd18a0f-c0cc-4a62-8564-30f983fc06a0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:14:02.583607 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acd18a0f-c0cc-4a62-8564-30f983fc06a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:14:02.587828 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acd18a0f-c0cc-4a62-8564-30f983fc06a0 admin admin] PATH_INFO: `/v3/projects/0cb170a9a5e548448e32393c9d527b90` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:14:02.587828 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acd18a0f-c0cc-4a62-8564-30f983fc06a0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0cb170a9a5e548448e32393c9d527b90)` {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 16:14:02.587828 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acd18a0f-c0cc-4a62-8564-30f983fc06a0 admin admin] RBAC: Authorization granted {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 16:14:02.709283 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.notifications [None req-acd18a0f-c0cc-4a62-8564-30f983fc06a0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0cb170a9a5e548448e32393c9d527b90', 'request_id': 'req-acd18a0f-c0cc-4a62-8564-30f983fc06a0'} {{(pid=81767) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 16:14:02.719926 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.notifications [None req-acd18a0f-c0cc-4a62-8564-30f983fc06a0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0cb170a9a5e548448e32393c9d527b90', 'request_id': 'req-acd18a0f-c0cc-4a62-8564-30f983fc06a0'} {{(pid=81767) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 16:14:02.820572 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.notifications [None req-acd18a0f-c0cc-4a62-8564-30f983fc06a0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0cb170a9a5e548448e32393c9d527b90', 'request_id': 'req-acd18a0f-c0cc-4a62-8564-30f983fc06a0'} {{(pid=81767) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 16:14:02.838837 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.notifications [None req-acd18a0f-c0cc-4a62-8564-30f983fc06a0 admin admin] The token cache is being invalidate because project 0cb170a9a5e548448e32393c9d527b90 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81767) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 22 16:14:02.839315 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.notifications [None req-acd18a0f-c0cc-4a62-8564-30f983fc06a0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81767) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 16:14:02.842351 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.notifications [None req-acd18a0f-c0cc-4a62-8564-30f983fc06a0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81767) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 16:14:02.844953 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 1290/2617] 10.222.0.15 () {58 vars in 1233 bytes} [Mon Aug 22 16:14:02 2022] DELETE /identity/v3/projects/0cb170a9a5e548448e32393c9d527b90 => generated 0 bytes in 295 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 16:14:02.850565 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cce0ac4-66b1-4554-88f6-a972679a741b None None] REQUEST_METHOD: `POST` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:14:02.850565 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cce0ac4-66b1-4554-88f6-a972679a741b None None] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:14:02.851169 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cce0ac4-66b1-4554-88f6-a972679a741b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:14:02.902946 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.auth.core [None req-3cce0ac4-66b1-4554-88f6-a972679a741b None None] MFA Rules not processed for user `6b2199ae0fb0480aa917ada391f5ec54`. Rule list: `[]` (Enabled: `True`). {{(pid=81768) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 16:14:02.904876 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-3cce0ac4-66b1-4554-88f6-a972679a741b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 16:14:02.972996 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 1328/2618] 10.222.0.15 () {58 vars in 958 bytes} [Mon Aug 22 16:14:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 16:14:02.981966 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cac8f6a-23a5-488e-b63a-5a43648ed717 None None] REQUEST_METHOD: `POST` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:14:02.982256 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cac8f6a-23a5-488e-b63a-5a43648ed717 None None] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:14:02.982537 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cac8f6a-23a5-488e-b63a-5a43648ed717 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:14:03.004388 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cce0ac4-66b1-4554-88f6-a972679a741b None None] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 16:14:03.006612 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-3cce0ac4-66b1-4554-88f6-a972679a741b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 16:14:03.011067 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.auth.core [None req-1cac8f6a-23a5-488e-b63a-5a43648ed717 None None] MFA Rules not processed for user `6b2199ae0fb0480aa917ada391f5ec54`. Rule list: `[]` (Enabled: `True`). {{(pid=81767) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 16:14:03.012565 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.fernet_utils [None req-1cac8f6a-23a5-488e-b63a-5a43648ed717 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81767) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 16:14:03.026267 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 1291/2619] 10.222.0.15 () {58 vars in 958 bytes} [Mon Aug 22 16:14:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 16:14:03.026267 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cac8f6a-23a5-488e-b63a-5a43648ed717 None None] Authenticating user token {{(pid=81767) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 16:14:03.026522 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.fernet_utils [None req-1cac8f6a-23a5-488e-b63a-5a43648ed717 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81767) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 16:14:03.036618 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cce0ac4-66b1-4554-88f6-a972679a741b None None] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 16:14:03.042268 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cac8f6a-23a5-488e-b63a-5a43648ed717 None None] Validating token access rules against request {{(pid=81767) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 16:14:03.043849 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14a082c4-ddfa-461a-bd6a-fa14f603ee02 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a98b1e4431940ab8d1fc95df2858a6c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 16:14:03.044751 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14a082c4-ddfa-461a-bd6a-fa14f603ee02 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:14:03.046088 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14a082c4-ddfa-461a-bd6a-fa14f603ee02 admin admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:14:03.046728 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14a082c4-ddfa-461a-bd6a-fa14f603ee02 admin admin] PATH_INFO: `/v3/projects/e8b902f6d01f4580ac10ddca4ccd7ed4` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:14:03.048817 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74b1390c-0668-48ac-9ca0-5418c85bbf93 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a98b1e4431940ab8d1fc95df2858a6c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81767) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 16:14:03.049946 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14a082c4-ddfa-461a-bd6a-fa14f603ee02 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e8b902f6d01f4580ac10ddca4ccd7ed4)` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 16:14:03.050062 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74b1390c-0668-48ac-9ca0-5418c85bbf93 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:14:03.050563 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74b1390c-0668-48ac-9ca0-5418c85bbf93 admin admin] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:14:03.051199 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74b1390c-0668-48ac-9ca0-5418c85bbf93 admin admin] PATH_INFO: `/v3/projects/956d4eb12bce4648bb34bb81bbd8f098` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:14:03.053042 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74b1390c-0668-48ac-9ca0-5418c85bbf93 admin admin] RBAC: Authorizing `identity:delete_project(project_id=956d4eb12bce4648bb34bb81bbd8f098)` {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 16:14:03.053963 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14a082c4-ddfa-461a-bd6a-fa14f603ee02 admin admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 16:14:03.055671 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74b1390c-0668-48ac-9ca0-5418c85bbf93 admin admin] RBAC: Authorization granted {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 16:14:03.193643 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-14a082c4-ddfa-461a-bd6a-fa14f603ee02 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e8b902f6d01f4580ac10ddca4ccd7ed4', 'request_id': 'req-14a082c4-ddfa-461a-bd6a-fa14f603ee02'} {{(pid=81768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 16:14:03.200804 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-14a082c4-ddfa-461a-bd6a-fa14f603ee02 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e8b902f6d01f4580ac10ddca4ccd7ed4', 'request_id': 'req-14a082c4-ddfa-461a-bd6a-fa14f603ee02'} {{(pid=81768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 16:14:03.221666 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.notifications [None req-74b1390c-0668-48ac-9ca0-5418c85bbf93 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '956d4eb12bce4648bb34bb81bbd8f098', 'request_id': 'req-74b1390c-0668-48ac-9ca0-5418c85bbf93'} {{(pid=81767) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 16:14:03.228024 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.notifications [None req-74b1390c-0668-48ac-9ca0-5418c85bbf93 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '956d4eb12bce4648bb34bb81bbd8f098', 'request_id': 'req-74b1390c-0668-48ac-9ca0-5418c85bbf93'} {{(pid=81767) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 16:14:03.342386 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-14a082c4-ddfa-461a-bd6a-fa14f603ee02 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e8b902f6d01f4580ac10ddca4ccd7ed4', 'request_id': 'req-14a082c4-ddfa-461a-bd6a-fa14f603ee02'} {{(pid=81768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 16:14:03.365065 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-14a082c4-ddfa-461a-bd6a-fa14f603ee02 admin admin] The token cache is being invalidate because project e8b902f6d01f4580ac10ddca4ccd7ed4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81768) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 22 16:14:03.365808 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-14a082c4-ddfa-461a-bd6a-fa14f603ee02 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 16:14:03.366797 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-14a082c4-ddfa-461a-bd6a-fa14f603ee02 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 16:14:03.368657 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 1329/2620] 10.222.0.15 () {58 vars in 1233 bytes} [Mon Aug 22 16:14:03 2022] DELETE /identity/v3/projects/e8b902f6d01f4580ac10ddca4ccd7ed4 => generated 0 bytes in 365 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 16:14:03.380661 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14a082c4-ddfa-461a-bd6a-fa14f603ee02 admin admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 16:14:03.382479 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-14a082c4-ddfa-461a-bd6a-fa14f603ee02 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 16:14:03.424531 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.notifications [None req-74b1390c-0668-48ac-9ca0-5418c85bbf93 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '956d4eb12bce4648bb34bb81bbd8f098', 'request_id': 'req-74b1390c-0668-48ac-9ca0-5418c85bbf93'} {{(pid=81767) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 16:14:03.445894 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.notifications [None req-74b1390c-0668-48ac-9ca0-5418c85bbf93 admin admin] The token cache is being invalidate because project 956d4eb12bce4648bb34bb81bbd8f098 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81767) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 22 16:14:03.446194 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.notifications [None req-74b1390c-0668-48ac-9ca0-5418c85bbf93 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81767) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 16:14:03.447785 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.notifications [None req-74b1390c-0668-48ac-9ca0-5418c85bbf93 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81767) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 16:14:03.453655 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 1292/2621] 10.222.0.15 () {58 vars in 1233 bytes} [Mon Aug 22 16:14:03 2022] DELETE /identity/v3/projects/956d4eb12bce4648bb34bb81bbd8f098 => generated 0 bytes in 425 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 16:14:03.454853 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74b1390c-0668-48ac-9ca0-5418c85bbf93 admin admin] Authenticating user token {{(pid=81767) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 16:14:03.457695 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.fernet_utils [None req-74b1390c-0668-48ac-9ca0-5418c85bbf93 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81767) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 16:14:03.491488 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 16:14:03.497351 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14a082c4-ddfa-461a-bd6a-fa14f603ee02 admin admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 16:14:03.532323 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG dbcounter [-] [81767] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81767) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 16:14:03.554179 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74b1390c-0668-48ac-9ca0-5418c85bbf93 admin admin] Validating token access rules against request {{(pid=81767) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 16:14:03.560207 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ec7b219-e47e-4c1d-a936-034f0f88416d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a98b1e4431940ab8d1fc95df2858a6c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81767) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 16:14:03.565029 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec7b219-e47e-4c1d-a936-034f0f88416d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:14:03.565029 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec7b219-e47e-4c1d-a936-034f0f88416d admin admin] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:14:03.565029 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec7b219-e47e-4c1d-a936-034f0f88416d admin admin] PATH_INFO: `/v3/users/04fdae1d631b45729443e921ea40c74d` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:14:03.565029 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ec7b219-e47e-4c1d-a936-034f0f88416d admin admin] RBAC: Authorizing `identity:delete_user(user_id=04fdae1d631b45729443e921ea40c74d)` {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 16:14:03.566543 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ec7b219-e47e-4c1d-a936-034f0f88416d admin admin] RBAC: Authorization granted {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 16:14:03.591330 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dffabbc3-e268-460f-8321-5e977eed4ac6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a98b1e4431940ab8d1fc95df2858a6c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 16:14:03.592731 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dffabbc3-e268-460f-8321-5e977eed4ac6 admin admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:14:03.592731 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dffabbc3-e268-460f-8321-5e977eed4ac6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:14:03.592731 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dffabbc3-e268-460f-8321-5e977eed4ac6 admin admin] PATH_INFO: `/v3/domains` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:14:03.593120 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dffabbc3-e268-460f-8321-5e977eed4ac6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81768) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 16:14:03.593424 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dffabbc3-e268-460f-8321-5e977eed4ac6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 16:14:03.595101 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dffabbc3-e268-460f-8321-5e977eed4ac6 admin admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 16:14:03.607252 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 1330/2622] 10.222.0.15 () {58 vars in 1153 bytes} [Mon Aug 22 16:14:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 238 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 16:14:03.611627 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dffabbc3-e268-460f-8321-5e977eed4ac6 admin admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 16:14:03.615712 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-dffabbc3-e268-460f-8321-5e977eed4ac6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 16:14:03.626161 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dffabbc3-e268-460f-8321-5e977eed4ac6 admin admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 16:14:03.632806 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-299a326c-9454-40a7-b378-ce3aebc9d994 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a98b1e4431940ab8d1fc95df2858a6c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 16:14:03.634298 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-299a326c-9454-40a7-b378-ce3aebc9d994 admin admin] REQUEST_METHOD: `POST` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:14:03.634298 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-299a326c-9454-40a7-b378-ce3aebc9d994 admin admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:14:03.634851 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-299a326c-9454-40a7-b378-ce3aebc9d994 admin admin] PATH_INFO: `/v3/projects` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:14:03.635942 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-299a326c-9454-40a7-b378-ce3aebc9d994 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 16:14:03.639744 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-299a326c-9454-40a7-b378-ce3aebc9d994 admin admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 16:14:03.754849 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 1331/2623] 10.222.0.15 () {60 vars in 1153 bytes} [Mon Aug 22 16:14:03 2022] POST /identity/v3/projects => generated 383 bytes in 144 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 22 16:14:03.759166 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-299a326c-9454-40a7-b378-ce3aebc9d994 admin admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 16:14:03.763894 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.notifications [None req-1ec7b219-e47e-4c1d-a936-034f0f88416d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '04fdae1d631b45729443e921ea40c74d'} {{(pid=81767) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 16:14:03.764564 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.notifications [None req-1ec7b219-e47e-4c1d-a936-034f0f88416d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '04fdae1d631b45729443e921ea40c74d'} {{(pid=81767) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 16:14:03.841281 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-299a326c-9454-40a7-b378-ce3aebc9d994 admin admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 16:14:03.849324 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa7d9338-7fec-4c43-a15d-d7a44ae397bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a98b1e4431940ab8d1fc95df2858a6c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 16:14:03.850166 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa7d9338-7fec-4c43-a15d-d7a44ae397bf admin admin] REQUEST_METHOD: `POST` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:14:03.850451 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa7d9338-7fec-4c43-a15d-d7a44ae397bf admin admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:14:03.850777 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa7d9338-7fec-4c43-a15d-d7a44ae397bf admin admin] PATH_INFO: `/v3/users` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:14:03.851444 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa7d9338-7fec-4c43-a15d-d7a44ae397bf admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 16:14:03.853719 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa7d9338-7fec-4c43-a15d-d7a44ae397bf admin admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 16:14:03.907601 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.notifications [None req-1ec7b219-e47e-4c1d-a936-034f0f88416d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '04fdae1d631b45729443e921ea40c74d'} {{(pid=81767) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 16:14:03.915498 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.notifications [None req-1ec7b219-e47e-4c1d-a936-034f0f88416d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '04fdae1d631b45729443e921ea40c74d'} {{(pid=81767) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 16:14:03.932857 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.notifications [None req-1ec7b219-e47e-4c1d-a936-034f0f88416d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '04fdae1d631b45729443e921ea40c74d'} {{(pid=81767) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 16:14:03.940176 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 1293/2624] 10.222.0.15 () {58 vars in 1224 bytes} [Mon Aug 22 16:14:03 2022] DELETE /identity/v3/users/04fdae1d631b45729443e921ea40c74d => generated 0 bytes in 487 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 22 16:14:03.955012 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 1332/2625] 10.222.0.15 () {60 vars in 1144 bytes} [Mon Aug 22 16:14:03 2022] POST /identity/v3/users => generated 292 bytes in 196 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 22 16:14:03.956240 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ec7b219-e47e-4c1d-a936-034f0f88416d admin admin] Authenticating user token {{(pid=81767) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 16:14:03.958042 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.fernet_utils [None req-1ec7b219-e47e-4c1d-a936-034f0f88416d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81767) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 16:14:04.038369 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ec7b219-e47e-4c1d-a936-034f0f88416d admin admin] Validating token access rules against request {{(pid=81767) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 16:14:04.044598 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a83132a4-deb2-4488-bc19-668a2238739c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a98b1e4431940ab8d1fc95df2858a6c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81767) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 16:14:04.050496 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a83132a4-deb2-4488-bc19-668a2238739c admin admin] REQUEST_METHOD: `GET` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:14:04.050496 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a83132a4-deb2-4488-bc19-668a2238739c admin admin] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:14:04.050496 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a83132a4-deb2-4488-bc19-668a2238739c admin admin] PATH_INFO: `/v3/roles` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:14:04.050496 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a83132a4-deb2-4488-bc19-668a2238739c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 16:14:04.050496 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a83132a4-deb2-4488-bc19-668a2238739c admin admin] RBAC: Authorization granted {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 16:14:04.061045 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 1294/2626] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Aug 22 16:14:03 2022] GET /identity/v3/roles => generated 1420 bytes in 105 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 22 16:14:04.066218 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa7d9338-7fec-4c43-a15d-d7a44ae397bf admin admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 16:14:04.078995 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa7d9338-7fec-4c43-a15d-d7a44ae397bf admin admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 16:14:04.084006 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-844a2ee4-a433-40bb-ab43-50e60c2169fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a98b1e4431940ab8d1fc95df2858a6c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 16:14:04.084998 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-844a2ee4-a433-40bb-ab43-50e60c2169fb admin admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:14:04.085648 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-844a2ee4-a433-40bb-ab43-50e60c2169fb admin admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:14:04.086020 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-844a2ee4-a433-40bb-ab43-50e60c2169fb admin admin] PATH_INFO: `/v3/roles` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:14:04.086858 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-844a2ee4-a433-40bb-ab43-50e60c2169fb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 16:14:04.089410 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-844a2ee4-a433-40bb-ab43-50e60c2169fb admin admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 16:14:04.101363 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 1333/2627] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Aug 22 16:14:04 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 22 16:14:04.105704 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a83132a4-deb2-4488-bc19-668a2238739c admin admin] Authenticating user token {{(pid=81767) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 16:14:04.118914 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a83132a4-deb2-4488-bc19-668a2238739c admin admin] Validating token access rules against request {{(pid=81767) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 16:14:04.123821 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94a5c2a6-2fdd-4001-9f42-01afa1797a25 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a98b1e4431940ab8d1fc95df2858a6c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81767) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 16:14:04.124461 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94a5c2a6-2fdd-4001-9f42-01afa1797a25 admin admin] REQUEST_METHOD: `PUT` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:14:04.124743 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94a5c2a6-2fdd-4001-9f42-01afa1797a25 admin admin] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:14:04.125658 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94a5c2a6-2fdd-4001-9f42-01afa1797a25 admin admin] PATH_INFO: `/v3/projects/046d7f0fbf2a45239489310df5423df4/users/b2ccd9491f044187af3936b549473aea/roles/e6fc1ee3fb974125ba9b4fe55b12b45e` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:14:04.143821 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94a5c2a6-2fdd-4001-9f42-01afa1797a25 admin admin] RBAC: Authorizing `identity:create_grant(project_id=046d7f0fbf2a45239489310df5423df4, user_id=b2ccd9491f044187af3936b549473aea, role_id=e6fc1ee3fb974125ba9b4fe55b12b45e)` {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 16:14:04.146667 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94a5c2a6-2fdd-4001-9f42-01afa1797a25 admin admin] RBAC: Authorization granted {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 16:14:04.283291 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 1295/2628] 10.222.0.15 () {60 vars in 1483 bytes} [Mon Aug 22 16:14:04 2022] PUT /identity/v3/projects/046d7f0fbf2a45239489310df5423df4/users/b2ccd9491f044187af3936b549473aea/roles/e6fc1ee3fb974125ba9b4fe55b12b45e => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 16:14:04.293950 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf5c16f-d250-4fcf-8de9-27749aae3a82 None None] REQUEST_METHOD: `POST` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:14:04.294390 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf5c16f-d250-4fcf-8de9-27749aae3a82 None None] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:14:04.295258 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf5c16f-d250-4fcf-8de9-27749aae3a82 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:14:04.327988 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.auth.core [None req-bdf5c16f-d250-4fcf-8de9-27749aae3a82 None None] MFA Rules not processed for user `b2ccd9491f044187af3936b549473aea`. Rule list: `[]` (Enabled: `True`). {{(pid=81768) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 16:14:04.329869 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-bdf5c16f-d250-4fcf-8de9-27749aae3a82 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 16:14:04.419527 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 1334/2629] 10.222.0.15 () {58 vars in 958 bytes} [Mon Aug 22 16:14:04 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 16:14:04.431495 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94a5c2a6-2fdd-4001-9f42-01afa1797a25 admin admin] Authenticating user token {{(pid=81767) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 16:14:04.433286 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.fernet_utils [None req-94a5c2a6-2fdd-4001-9f42-01afa1797a25 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81767) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 16:14:04.499502 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94a5c2a6-2fdd-4001-9f42-01afa1797a25 admin admin] Validating token access rules against request {{(pid=81767) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 16:14:04.503774 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69a60c35-65a9-4011-8178-ef24eaa38304 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79706d480f864998b9f8594be407ceab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4ee0413f9ef4d4fb1f95a660babe090', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81767) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 16:14:04.504401 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a60c35-65a9-4011-8178-ef24eaa38304 service cinder] REQUEST_METHOD: `GET` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:14:04.504703 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a60c35-65a9-4011-8178-ef24eaa38304 service cinder] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:14:04.504943 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a60c35-65a9-4011-8178-ef24eaa38304 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:14:04.514855 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69a60c35-65a9-4011-8178-ef24eaa38304 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 16:14:04.517409 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69a60c35-65a9-4011-8178-ef24eaa38304 service cinder] RBAC: Authorization granted {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 16:14:04.527402 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 1296/2630] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Aug 22 16:14:04 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 16:14:04.552586 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdf5c16f-d250-4fcf-8de9-27749aae3a82 None None] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 16:14:04.622848 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 16:14:04.645246 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG dbcounter [-] [81767] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=81767) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 16:14:04.669477 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdf5c16f-d250-4fcf-8de9-27749aae3a82 None None] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 16:14:04.675679 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fcfce84-8160-485c-bd64-416e8b78e0af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a98b1e4431940ab8d1fc95df2858a6c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 16:14:04.676447 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fcfce84-8160-485c-bd64-416e8b78e0af admin admin] REQUEST_METHOD: `DELETE` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:14:04.676631 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fcfce84-8160-485c-bd64-416e8b78e0af admin admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:14:04.676902 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fcfce84-8160-485c-bd64-416e8b78e0af admin admin] PATH_INFO: `/v3/users/b2ccd9491f044187af3936b549473aea` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:14:04.678813 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fcfce84-8160-485c-bd64-416e8b78e0af admin admin] RBAC: Authorizing `identity:delete_user(user_id=b2ccd9491f044187af3936b549473aea)` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 16:14:04.680807 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fcfce84-8160-485c-bd64-416e8b78e0af admin admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 16:14:04.872190 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69a60c35-65a9-4011-8178-ef24eaa38304 service cinder] Authenticating user token {{(pid=81767) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 16:14:04.873885 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.fernet_utils [None req-69a60c35-65a9-4011-8178-ef24eaa38304 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81767) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 16:14:04.889067 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69a60c35-65a9-4011-8178-ef24eaa38304 service cinder] Validating token access rules against request {{(pid=81767) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 16:14:04.894987 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83478a72-322b-4c07-bbfa-9fab4e2189ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a98b1e4431940ab8d1fc95df2858a6c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81767) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 16:14:04.895671 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83478a72-322b-4c07-bbfa-9fab4e2189ec admin admin] REQUEST_METHOD: `DELETE` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:14:04.895935 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83478a72-322b-4c07-bbfa-9fab4e2189ec admin admin] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:14:04.896207 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83478a72-322b-4c07-bbfa-9fab4e2189ec admin admin] PATH_INFO: `/v3/projects/61d3bc83c029431da6368e57f1bd4da3` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:14:04.897642 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83478a72-322b-4c07-bbfa-9fab4e2189ec admin admin] RBAC: Authorizing `identity:delete_project(project_id=61d3bc83c029431da6368e57f1bd4da3)` {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 16:14:04.899699 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83478a72-322b-4c07-bbfa-9fab4e2189ec admin admin] RBAC: Authorization granted {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 16:14:04.958489 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-8fcfce84-8160-485c-bd64-416e8b78e0af admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b2ccd9491f044187af3936b549473aea'} {{(pid=81768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 16:14:04.960133 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-8fcfce84-8160-485c-bd64-416e8b78e0af admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b2ccd9491f044187af3936b549473aea'} {{(pid=81768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 16:14:04.960755 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-8fcfce84-8160-485c-bd64-416e8b78e0af admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b2ccd9491f044187af3936b549473aea'} {{(pid=81768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 16:14:04.977051 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-8fcfce84-8160-485c-bd64-416e8b78e0af admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b2ccd9491f044187af3936b549473aea'} {{(pid=81768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 16:14:05.042909 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-8fcfce84-8160-485c-bd64-416e8b78e0af admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b2ccd9491f044187af3936b549473aea'} {{(pid=81768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 16:14:05.061596 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.notifications [None req-83478a72-322b-4c07-bbfa-9fab4e2189ec admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '61d3bc83c029431da6368e57f1bd4da3', 'request_id': 'req-83478a72-322b-4c07-bbfa-9fab4e2189ec'} {{(pid=81767) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 16:14:05.067797 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 1335/2631] 10.222.0.15 () {58 vars in 1224 bytes} [Mon Aug 22 16:14:04 2022] DELETE /identity/v3/users/b2ccd9491f044187af3936b549473aea => generated 0 bytes in 516 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 16:14:05.072100 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6693c90-f63c-437a-85c2-ce0a66953a90 None None] REQUEST_METHOD: `POST` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:14:05.072470 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6693c90-f63c-437a-85c2-ce0a66953a90 None None] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:14:05.072996 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6693c90-f63c-437a-85c2-ce0a66953a90 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:14:05.074236 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.notifications [None req-83478a72-322b-4c07-bbfa-9fab4e2189ec admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '61d3bc83c029431da6368e57f1bd4da3', 'request_id': 'req-83478a72-322b-4c07-bbfa-9fab4e2189ec'} {{(pid=81767) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 16:14:05.096985 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.auth.core [None req-e6693c90-f63c-437a-85c2-ce0a66953a90 None None] MFA Rules not processed for user `6b2199ae0fb0480aa917ada391f5ec54`. Rule list: `[]` (Enabled: `True`). {{(pid=81768) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 16:14:05.098124 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-e6693c90-f63c-437a-85c2-ce0a66953a90 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 16:14:05.176470 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 1336/2632] 10.222.0.15 () {58 vars in 958 bytes} [Mon Aug 22 16:14:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 22 16:14:05.182835 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6693c90-f63c-437a-85c2-ce0a66953a90 None None] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 16:14:05.184725 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.notifications [None req-83478a72-322b-4c07-bbfa-9fab4e2189ec admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '61d3bc83c029431da6368e57f1bd4da3', 'request_id': 'req-83478a72-322b-4c07-bbfa-9fab4e2189ec'} {{(pid=81767) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 16:14:05.199058 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-e6693c90-f63c-437a-85c2-ce0a66953a90 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 16:14:05.229922 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.notifications [None req-83478a72-322b-4c07-bbfa-9fab4e2189ec admin admin] The token cache is being invalidate because project 61d3bc83c029431da6368e57f1bd4da3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81767) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 22 16:14:05.232093 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.notifications [None req-83478a72-322b-4c07-bbfa-9fab4e2189ec admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81767) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 16:14:05.234711 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.notifications [None req-83478a72-322b-4c07-bbfa-9fab4e2189ec admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81767) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 16:14:05.238024 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 1297/2633] 10.222.0.15 () {58 vars in 1233 bytes} [Mon Aug 22 16:14:04 2022] DELETE /identity/v3/projects/61d3bc83c029431da6368e57f1bd4da3 => generated 0 bytes in 367 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 22 16:14:05.251332 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83478a72-322b-4c07-bbfa-9fab4e2189ec admin admin] Authenticating user token {{(pid=81767) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 16:14:05.260268 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.fernet_utils [None req-83478a72-322b-4c07-bbfa-9fab4e2189ec admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81767) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 16:14:05.290267 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6693c90-f63c-437a-85c2-ce0a66953a90 None None] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 16:14:05.304183 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e15a7afb-92bd-43b1-b593-93d83805fe03 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '234687adad594f1aa45775a565ffbdc7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4ee0413f9ef4d4fb1f95a660babe090', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 16:14:05.304810 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e15a7afb-92bd-43b1-b593-93d83805fe03 service neutron] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:14:05.305094 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e15a7afb-92bd-43b1-b593-93d83805fe03 service neutron] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:14:05.305365 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e15a7afb-92bd-43b1-b593-93d83805fe03 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:14:05.308134 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-e15a7afb-92bd-43b1-b593-93d83805fe03 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 16:14:05.344721 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e15a7afb-92bd-43b1-b593-93d83805fe03 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 16:14:05.344721 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e15a7afb-92bd-43b1-b593-93d83805fe03 service neutron] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 16:14:05.364234 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 1337/2634] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Aug 22 16:14:05 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 182 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 16:14:05.405856 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83478a72-322b-4c07-bbfa-9fab4e2189ec admin admin] Validating token access rules against request {{(pid=81767) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 16:14:05.412971 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-481c9716-c266-4ef2-bde6-e58362a4f653 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a98b1e4431940ab8d1fc95df2858a6c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81767) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 16:14:05.417926 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-481c9716-c266-4ef2-bde6-e58362a4f653 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:14:05.418247 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-481c9716-c266-4ef2-bde6-e58362a4f653 admin admin] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:14:05.418672 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-481c9716-c266-4ef2-bde6-e58362a4f653 admin admin] PATH_INFO: `/v3/users/02bf5dcc8e1a49ccbf3efd9d0499bea1` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:14:05.420044 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-481c9716-c266-4ef2-bde6-e58362a4f653 admin admin] RBAC: Authorizing `identity:delete_user(user_id=02bf5dcc8e1a49ccbf3efd9d0499bea1)` {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 16:14:05.423013 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-481c9716-c266-4ef2-bde6-e58362a4f653 admin admin] RBAC: Authorization granted {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 16:14:05.454580 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 16:14:05.660264 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.notifications [None req-481c9716-c266-4ef2-bde6-e58362a4f653 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '02bf5dcc8e1a49ccbf3efd9d0499bea1'} {{(pid=81767) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 16:14:05.660962 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.notifications [None req-481c9716-c266-4ef2-bde6-e58362a4f653 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '02bf5dcc8e1a49ccbf3efd9d0499bea1'} {{(pid=81767) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 16:14:05.729048 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.notifications [None req-481c9716-c266-4ef2-bde6-e58362a4f653 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '02bf5dcc8e1a49ccbf3efd9d0499bea1'} {{(pid=81767) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 16:14:05.740176 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.notifications [None req-481c9716-c266-4ef2-bde6-e58362a4f653 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '02bf5dcc8e1a49ccbf3efd9d0499bea1'} {{(pid=81767) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 16:14:05.763078 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.notifications [None req-481c9716-c266-4ef2-bde6-e58362a4f653 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '02bf5dcc8e1a49ccbf3efd9d0499bea1'} {{(pid=81767) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 16:14:05.771340 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 1298/2635] 10.222.0.15 () {58 vars in 1224 bytes} [Mon Aug 22 16:14:05 2022] DELETE /identity/v3/users/02bf5dcc8e1a49ccbf3efd9d0499bea1 => generated 0 bytes in 528 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 16:14:05.997457 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG dbcounter [-] [81767] Writing DB stats keystone:SELECT=82,keystone:DELETE=16,keystone:INSERT=2 {{(pid=81767) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 16:14:06.233899 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e15a7afb-92bd-43b1-b593-93d83805fe03 service neutron] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 16:14:06.236423 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-e15a7afb-92bd-43b1-b593-93d83805fe03 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 16:14:06.264112 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e15a7afb-92bd-43b1-b593-93d83805fe03 service neutron] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 16:14:06.390218 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69174447-5447-4f8b-abea-71b616b7bcba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a98b1e4431940ab8d1fc95df2858a6c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 16:14:06.391738 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69174447-5447-4f8b-abea-71b616b7bcba admin admin] REQUEST_METHOD: `DELETE` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:14:06.392249 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69174447-5447-4f8b-abea-71b616b7bcba admin admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:14:06.392702 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69174447-5447-4f8b-abea-71b616b7bcba admin admin] PATH_INFO: `/v3/projects/046d7f0fbf2a45239489310df5423df4` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:14:06.394374 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69174447-5447-4f8b-abea-71b616b7bcba admin admin] RBAC: Authorizing `identity:delete_project(project_id=046d7f0fbf2a45239489310df5423df4)` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 16:14:06.397543 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69174447-5447-4f8b-abea-71b616b7bcba admin admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 16:14:06.573003 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-481c9716-c266-4ef2-bde6-e58362a4f653 admin admin] Authenticating user token {{(pid=81767) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 16:14:06.573003 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.fernet_utils [None req-481c9716-c266-4ef2-bde6-e58362a4f653 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81767) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 16:14:06.585864 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-481c9716-c266-4ef2-bde6-e58362a4f653 admin admin] Validating token access rules against request {{(pid=81767) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 16:14:06.586972 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-69174447-5447-4f8b-abea-71b616b7bcba admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '046d7f0fbf2a45239489310df5423df4', 'request_id': 'req-69174447-5447-4f8b-abea-71b616b7bcba'} {{(pid=81768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 16:14:06.596247 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-69174447-5447-4f8b-abea-71b616b7bcba admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '046d7f0fbf2a45239489310df5423df4', 'request_id': 'req-69174447-5447-4f8b-abea-71b616b7bcba'} {{(pid=81768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 16:14:06.669425 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dbb06fb-6578-42f2-b2d3-0f1ee533aa2e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a98b1e4431940ab8d1fc95df2858a6c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81767) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 16:14:06.669425 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dbb06fb-6578-42f2-b2d3-0f1ee533aa2e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:14:06.669425 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dbb06fb-6578-42f2-b2d3-0f1ee533aa2e admin admin] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:14:06.669425 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dbb06fb-6578-42f2-b2d3-0f1ee533aa2e admin admin] PATH_INFO: `/v3/projects/6b8a2d1450354d698edd83ad16967159` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:14:06.670100 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dbb06fb-6578-42f2-b2d3-0f1ee533aa2e admin admin] RBAC: Authorizing `identity:delete_project(project_id=6b8a2d1450354d698edd83ad16967159)` {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 16:14:06.670100 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dbb06fb-6578-42f2-b2d3-0f1ee533aa2e admin admin] RBAC: Authorization granted {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 16:14:06.757446 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-69174447-5447-4f8b-abea-71b616b7bcba admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '046d7f0fbf2a45239489310df5423df4', 'request_id': 'req-69174447-5447-4f8b-abea-71b616b7bcba'} {{(pid=81768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 16:14:06.776641 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-69174447-5447-4f8b-abea-71b616b7bcba admin admin] The token cache is being invalidate because project 046d7f0fbf2a45239489310df5423df4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81768) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 22 16:14:06.777316 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-69174447-5447-4f8b-abea-71b616b7bcba admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 16:14:06.778702 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-69174447-5447-4f8b-abea-71b616b7bcba admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 16:14:06.780704 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 1338/2636] 10.222.0.15 () {58 vars in 1233 bytes} [Mon Aug 22 16:14:06 2022] DELETE /identity/v3/projects/046d7f0fbf2a45239489310df5423df4 => generated 0 bytes in 547 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 16:14:06.799520 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ef746d3-0bfa-44e1-b0a8-bcf3f527f341 None None] REQUEST_METHOD: `POST` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:14:06.800156 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ef746d3-0bfa-44e1-b0a8-bcf3f527f341 None None] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:14:06.800607 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ef746d3-0bfa-44e1-b0a8-bcf3f527f341 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:14:06.812893 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.notifications [None req-4dbb06fb-6578-42f2-b2d3-0f1ee533aa2e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6b8a2d1450354d698edd83ad16967159', 'request_id': 'req-4dbb06fb-6578-42f2-b2d3-0f1ee533aa2e'} {{(pid=81767) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 16:14:06.820094 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.notifications [None req-4dbb06fb-6578-42f2-b2d3-0f1ee533aa2e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6b8a2d1450354d698edd83ad16967159', 'request_id': 'req-4dbb06fb-6578-42f2-b2d3-0f1ee533aa2e'} {{(pid=81767) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 16:14:06.837770 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.auth.core [None req-6ef746d3-0bfa-44e1-b0a8-bcf3f527f341 None None] MFA Rules not processed for user `6b2199ae0fb0480aa917ada391f5ec54`. Rule list: `[]` (Enabled: `True`). {{(pid=81768) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 16:14:06.839249 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-6ef746d3-0bfa-44e1-b0a8-bcf3f527f341 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 16:14:06.891235 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 16:14:06.892034 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.notifications [None req-4dbb06fb-6578-42f2-b2d3-0f1ee533aa2e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6b8a2d1450354d698edd83ad16967159', 'request_id': 'req-4dbb06fb-6578-42f2-b2d3-0f1ee533aa2e'} {{(pid=81767) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 16:14:06.912252 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.notifications [None req-4dbb06fb-6578-42f2-b2d3-0f1ee533aa2e admin admin] The token cache is being invalidate because project 6b8a2d1450354d698edd83ad16967159 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81767) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 22 16:14:06.913370 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.notifications [None req-4dbb06fb-6578-42f2-b2d3-0f1ee533aa2e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81767) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 16:14:06.915024 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.notifications [None req-4dbb06fb-6578-42f2-b2d3-0f1ee533aa2e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81767) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 16:14:06.924207 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 1299/2637] 10.222.0.15 () {58 vars in 1233 bytes} [Mon Aug 22 16:14:06 2022] DELETE /identity/v3/projects/6b8a2d1450354d698edd83ad16967159 => generated 0 bytes in 356 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 22 16:14:06.925917 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83330de1-70f8-40d7-b38c-ceb06fba5cad None None] REQUEST_METHOD: `POST` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:14:06.926389 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83330de1-70f8-40d7-b38c-ceb06fba5cad None None] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:14:06.927231 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83330de1-70f8-40d7-b38c-ceb06fba5cad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:14:06.949710 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 1339/2638] 10.222.0.15 () {58 vars in 958 bytes} [Mon Aug 22 16:14:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 16:14:06.953457 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.auth.core [None req-83330de1-70f8-40d7-b38c-ceb06fba5cad None None] MFA Rules not processed for user `6b2199ae0fb0480aa917ada391f5ec54`. Rule list: `[]` (Enabled: `True`). {{(pid=81767) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 16:14:06.955072 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.fernet_utils [None req-83330de1-70f8-40d7-b38c-ceb06fba5cad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81767) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 16:14:06.959597 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-355bd923-a22c-48c0-9545-6bd848862817 None None] REQUEST_METHOD: `POST` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:14:06.960444 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-355bd923-a22c-48c0-9545-6bd848862817 None None] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:14:06.960892 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-355bd923-a22c-48c0-9545-6bd848862817 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:14:06.971793 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 1300/2639] 10.222.0.15 () {58 vars in 958 bytes} [Mon Aug 22 16:14:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 47 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 16:14:06.982040 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88270e41-a507-4d82-b6ce-2e4ab88030b7 None None] REQUEST_METHOD: `POST` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:14:06.982612 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88270e41-a507-4d82-b6ce-2e4ab88030b7 None None] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:14:06.983054 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88270e41-a507-4d82-b6ce-2e4ab88030b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:14:07.009520 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.auth.core [None req-88270e41-a507-4d82-b6ce-2e4ab88030b7 None None] MFA Rules not processed for user `6b2199ae0fb0480aa917ada391f5ec54`. Rule list: `[]` (Enabled: `True`). {{(pid=81767) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 16:14:07.011143 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.fernet_utils [None req-88270e41-a507-4d82-b6ce-2e4ab88030b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81767) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 16:14:07.028678 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 1301/2640] 10.222.0.15 () {58 vars in 958 bytes} [Mon Aug 22 16:14:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 45 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 22 16:14:07.030456 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88270e41-a507-4d82-b6ce-2e4ab88030b7 None None] Authenticating user token {{(pid=81767) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 16:14:07.031392 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.auth.core [None req-355bd923-a22c-48c0-9545-6bd848862817 None None] MFA Rules not processed for user `6b2199ae0fb0480aa917ada391f5ec54`. Rule list: `[]` (Enabled: `True`). {{(pid=81768) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 16:14:07.033125 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-355bd923-a22c-48c0-9545-6bd848862817 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 16:14:07.049283 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88270e41-a507-4d82-b6ce-2e4ab88030b7 None None] Validating token access rules against request {{(pid=81767) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 16:14:07.055208 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6b75bd9-0255-4e8b-9839-75587a00d5b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a98b1e4431940ab8d1fc95df2858a6c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81767) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 16:14:07.056185 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6b75bd9-0255-4e8b-9839-75587a00d5b0 admin admin] REQUEST_METHOD: `GET` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:14:07.056719 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6b75bd9-0255-4e8b-9839-75587a00d5b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:14:07.057172 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6b75bd9-0255-4e8b-9839-75587a00d5b0 admin admin] PATH_INFO: `/v3/domains` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:14:07.058272 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6b75bd9-0255-4e8b-9839-75587a00d5b0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81767) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 16:14:07.058747 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6b75bd9-0255-4e8b-9839-75587a00d5b0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 16:14:07.060858 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6b75bd9-0255-4e8b-9839-75587a00d5b0 admin admin] RBAC: Authorization granted {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 16:14:07.061110 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 1340/2641] 10.222.0.15 () {58 vars in 958 bytes} [Mon Aug 22 16:14:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 22 16:14:07.064309 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-355bd923-a22c-48c0-9545-6bd848862817 None None] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 16:14:07.079196 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 1302/2642] 10.222.0.15 () {58 vars in 1153 bytes} [Mon Aug 22 16:14:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 16:14:07.085780 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-355bd923-a22c-48c0-9545-6bd848862817 None None] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 16:14:07.090934 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6b75bd9-0255-4e8b-9839-75587a00d5b0 admin admin] Authenticating user token {{(pid=81767) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 16:14:07.113300 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92735ff2-5b10-42e6-952a-f011c2f12c5f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a98b1e4431940ab8d1fc95df2858a6c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 16:14:07.121261 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92735ff2-5b10-42e6-952a-f011c2f12c5f admin admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:14:07.121261 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92735ff2-5b10-42e6-952a-f011c2f12c5f admin admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:14:07.121532 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6b75bd9-0255-4e8b-9839-75587a00d5b0 admin admin] Validating token access rules against request {{(pid=81767) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 16:14:07.121639 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92735ff2-5b10-42e6-952a-f011c2f12c5f admin admin] PATH_INFO: `/v3/domains` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:14:07.123716 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92735ff2-5b10-42e6-952a-f011c2f12c5f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81768) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 16:14:07.126329 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8e0cc5e-b0d2-42d4-9c57-03a2b541cfb4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a98b1e4431940ab8d1fc95df2858a6c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81767) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 16:14:07.126827 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92735ff2-5b10-42e6-952a-f011c2f12c5f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 16:14:07.127002 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8e0cc5e-b0d2-42d4-9c57-03a2b541cfb4 admin admin] REQUEST_METHOD: `POST` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:14:07.127304 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8e0cc5e-b0d2-42d4-9c57-03a2b541cfb4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:14:07.127600 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8e0cc5e-b0d2-42d4-9c57-03a2b541cfb4 admin admin] PATH_INFO: `/v3/projects` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:14:07.128249 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8e0cc5e-b0d2-42d4-9c57-03a2b541cfb4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 16:14:07.128524 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92735ff2-5b10-42e6-952a-f011c2f12c5f admin admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 16:14:07.133288 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8e0cc5e-b0d2-42d4-9c57-03a2b541cfb4 admin admin] RBAC: Authorization granted {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 16:14:07.155124 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 1341/2643] 10.222.0.15 () {58 vars in 1153 bytes} [Mon Aug 22 16:14:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 16:14:07.155124 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92735ff2-5b10-42e6-952a-f011c2f12c5f admin admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 16:14:07.160453 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92735ff2-5b10-42e6-952a-f011c2f12c5f admin admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 16:14:07.166388 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdc512dd-24b6-418d-b196-ade3ebb41615 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a98b1e4431940ab8d1fc95df2858a6c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 16:14:07.167092 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdc512dd-24b6-418d-b196-ade3ebb41615 admin admin] REQUEST_METHOD: `POST` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:14:07.168000 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdc512dd-24b6-418d-b196-ade3ebb41615 admin admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:14:07.168000 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdc512dd-24b6-418d-b196-ade3ebb41615 admin admin] PATH_INFO: `/v3/projects` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:14:07.168599 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdc512dd-24b6-418d-b196-ade3ebb41615 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 16:14:07.170643 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdc512dd-24b6-418d-b196-ade3ebb41615 admin admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 16:14:07.261021 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 1303/2644] 10.222.0.15 () {60 vars in 1153 bytes} [Mon Aug 22 16:14:07 2022] POST /identity/v3/projects => generated 383 bytes in 173 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 22 16:14:07.265525 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8e0cc5e-b0d2-42d4-9c57-03a2b541cfb4 admin admin] Authenticating user token {{(pid=81767) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 16:14:07.337706 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 1342/2645] 10.222.0.15 () {60 vars in 1153 bytes} [Mon Aug 22 16:14:07 2022] POST /identity/v3/projects => generated 369 bytes in 187 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 22 16:14:07.338840 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdc512dd-24b6-418d-b196-ade3ebb41615 admin admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 16:14:07.404373 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8e0cc5e-b0d2-42d4-9c57-03a2b541cfb4 admin admin] Validating token access rules against request {{(pid=81767) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 16:14:07.474528 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdc512dd-24b6-418d-b196-ade3ebb41615 admin admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 16:14:07.480949 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97706acb-b07e-4091-bb48-b8f2398bdf9a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a98b1e4431940ab8d1fc95df2858a6c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 16:14:07.481902 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97706acb-b07e-4091-bb48-b8f2398bdf9a admin admin] REQUEST_METHOD: `POST` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:14:07.482350 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97706acb-b07e-4091-bb48-b8f2398bdf9a admin admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:14:07.482773 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97706acb-b07e-4091-bb48-b8f2398bdf9a admin admin] PATH_INFO: `/v3/users` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:14:07.483594 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97706acb-b07e-4091-bb48-b8f2398bdf9a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 16:14:07.484917 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG dbcounter [-] [81767] Writing DB stats keystone:SELECT=90,keystone:DELETE=8,keystone:INSERT=2 {{(pid=81767) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 16:14:07.487293 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97706acb-b07e-4091-bb48-b8f2398bdf9a admin admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 16:14:07.525980 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b4604e6-f89f-42b7-8773-2d01c7080365 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a98b1e4431940ab8d1fc95df2858a6c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81767) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 16:14:07.526722 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b4604e6-f89f-42b7-8773-2d01c7080365 admin admin] REQUEST_METHOD: `POST` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:14:07.527065 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b4604e6-f89f-42b7-8773-2d01c7080365 admin admin] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:14:07.527425 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b4604e6-f89f-42b7-8773-2d01c7080365 admin admin] PATH_INFO: `/v3/users` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:14:07.528130 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b4604e6-f89f-42b7-8773-2d01c7080365 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 16:14:07.534614 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b4604e6-f89f-42b7-8773-2d01c7080365 admin admin] RBAC: Authorization granted {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 16:14:07.623852 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 1343/2646] 10.222.0.15 () {60 vars in 1144 bytes} [Mon Aug 22 16:14:07 2022] POST /identity/v3/users => generated 285 bytes in 285 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 22 16:14:07.627712 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97706acb-b07e-4091-bb48-b8f2398bdf9a admin admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 16:14:07.655214 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 1304/2647] 10.222.0.15 () {60 vars in 1144 bytes} [Mon Aug 22 16:14:07 2022] POST /identity/v3/users => generated 292 bytes in 390 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 22 16:14:07.656291 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b4604e6-f89f-42b7-8773-2d01c7080365 admin admin] Authenticating user token {{(pid=81767) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 16:14:07.668691 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97706acb-b07e-4091-bb48-b8f2398bdf9a admin admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 16:14:07.675446 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b4604e6-f89f-42b7-8773-2d01c7080365 admin admin] Validating token access rules against request {{(pid=81767) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 16:14:07.679851 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b364c6df-6489-4844-8285-0134847047af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a98b1e4431940ab8d1fc95df2858a6c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 16:14:07.681173 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b364c6df-6489-4844-8285-0134847047af admin admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:14:07.681454 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b364c6df-6489-4844-8285-0134847047af admin admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:14:07.681968 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b364c6df-6489-4844-8285-0134847047af admin admin] PATH_INFO: `/v3/roles` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:14:07.683976 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbe5c14b-144b-474d-a6d9-1f2637275b2a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a98b1e4431940ab8d1fc95df2858a6c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81767) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 16:14:07.684599 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe5c14b-144b-474d-a6d9-1f2637275b2a admin admin] REQUEST_METHOD: `GET` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:14:07.684876 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe5c14b-144b-474d-a6d9-1f2637275b2a admin admin] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:14:07.685145 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe5c14b-144b-474d-a6d9-1f2637275b2a admin admin] PATH_INFO: `/v3/roles` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:14:07.685793 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbe5c14b-144b-474d-a6d9-1f2637275b2a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 16:14:07.687471 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbe5c14b-144b-474d-a6d9-1f2637275b2a admin admin] RBAC: Authorization granted {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 16:14:07.688762 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b364c6df-6489-4844-8285-0134847047af admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 16:14:07.691269 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b364c6df-6489-4844-8285-0134847047af admin admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 16:14:07.703655 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 1305/2648] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Aug 22 16:14:07 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 22 16:14:07.710147 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbe5c14b-144b-474d-a6d9-1f2637275b2a admin admin] Authenticating user token {{(pid=81767) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 16:14:07.714749 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 1344/2649] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Aug 22 16:14:07 2022] GET /identity/v3/roles => generated 1420 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 22 16:14:07.720107 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b364c6df-6489-4844-8285-0134847047af admin admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 16:14:07.739680 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbe5c14b-144b-474d-a6d9-1f2637275b2a admin admin] Validating token access rules against request {{(pid=81767) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 16:14:07.745385 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeb1d129-fcfb-414a-9c49-9a8972940a85 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a98b1e4431940ab8d1fc95df2858a6c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81767) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 16:14:07.747750 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb1d129-fcfb-414a-9c49-9a8972940a85 admin admin] REQUEST_METHOD: `GET` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:14:07.748040 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb1d129-fcfb-414a-9c49-9a8972940a85 admin admin] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:14:07.748349 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb1d129-fcfb-414a-9c49-9a8972940a85 admin admin] PATH_INFO: `/v3/roles` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:14:07.749017 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeb1d129-fcfb-414a-9c49-9a8972940a85 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 16:14:07.752664 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeb1d129-fcfb-414a-9c49-9a8972940a85 admin admin] RBAC: Authorization granted {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 16:14:07.762478 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b364c6df-6489-4844-8285-0134847047af admin admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 16:14:07.769934 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 1306/2650] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Aug 22 16:14:07 2022] GET /identity/v3/roles => generated 1420 bytes in 61 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 22 16:14:07.783524 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeb1d129-fcfb-414a-9c49-9a8972940a85 admin admin] Authenticating user token {{(pid=81767) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 16:14:07.784926 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d50cd5a1-3da9-453b-9c48-ba68d60d4aa9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a98b1e4431940ab8d1fc95df2858a6c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 16:14:07.785672 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d50cd5a1-3da9-453b-9c48-ba68d60d4aa9 admin admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:14:07.786000 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d50cd5a1-3da9-453b-9c48-ba68d60d4aa9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:14:07.786374 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d50cd5a1-3da9-453b-9c48-ba68d60d4aa9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:14:07.787201 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d50cd5a1-3da9-453b-9c48-ba68d60d4aa9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 16:14:07.789255 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d50cd5a1-3da9-453b-9c48-ba68d60d4aa9 admin admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 16:14:07.797109 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeb1d129-fcfb-414a-9c49-9a8972940a85 admin admin] Validating token access rules against request {{(pid=81767) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 16:14:07.802022 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 1345/2651] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Aug 22 16:14:07 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 22 16:14:07.802959 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abac450e-7d5c-4ff3-a548-6b8cc871b167 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a98b1e4431940ab8d1fc95df2858a6c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81767) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 16:14:07.803884 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abac450e-7d5c-4ff3-a548-6b8cc871b167 admin admin] REQUEST_METHOD: `PUT` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:14:07.804239 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abac450e-7d5c-4ff3-a548-6b8cc871b167 admin admin] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:14:07.804601 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abac450e-7d5c-4ff3-a548-6b8cc871b167 admin admin] PATH_INFO: `/v3/projects/a719d562b1d848e682148155af24d610/users/78933c65a377452da78e98db95ef57ea/roles/e6fc1ee3fb974125ba9b4fe55b12b45e` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:14:07.806233 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d50cd5a1-3da9-453b-9c48-ba68d60d4aa9 admin admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 16:14:07.817753 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d50cd5a1-3da9-453b-9c48-ba68d60d4aa9 admin admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 16:14:07.823093 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abac450e-7d5c-4ff3-a548-6b8cc871b167 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a719d562b1d848e682148155af24d610, user_id=78933c65a377452da78e98db95ef57ea, role_id=e6fc1ee3fb974125ba9b4fe55b12b45e)` {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 16:14:07.824672 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c870c14-ba3d-44e6-a8c5-e7a9cb3be449 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a98b1e4431940ab8d1fc95df2858a6c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 16:14:07.825667 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c870c14-ba3d-44e6-a8c5-e7a9cb3be449 admin admin] REQUEST_METHOD: `PUT` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:14:07.825667 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c870c14-ba3d-44e6-a8c5-e7a9cb3be449 admin admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:14:07.825860 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c870c14-ba3d-44e6-a8c5-e7a9cb3be449 admin admin] PATH_INFO: `/v3/projects/b0e6712e5be748ac987e09af2f3422ad/users/2779dbc1258842a48eb70daeecdfeb18/roles/e6fc1ee3fb974125ba9b4fe55b12b45e` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:14:07.826630 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abac450e-7d5c-4ff3-a548-6b8cc871b167 admin admin] RBAC: Authorization granted {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 16:14:07.854739 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c870c14-ba3d-44e6-a8c5-e7a9cb3be449 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b0e6712e5be748ac987e09af2f3422ad, user_id=2779dbc1258842a48eb70daeecdfeb18, role_id=e6fc1ee3fb974125ba9b4fe55b12b45e)` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 16:14:07.854739 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c870c14-ba3d-44e6-a8c5-e7a9cb3be449 admin admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 16:14:07.976653 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 1307/2652] 10.222.0.15 () {60 vars in 1483 bytes} [Mon Aug 22 16:14:07 2022] PUT /identity/v3/projects/a719d562b1d848e682148155af24d610/users/78933c65a377452da78e98db95ef57ea/roles/e6fc1ee3fb974125ba9b4fe55b12b45e => generated 0 bytes in 192 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 16:14:08.026004 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 1346/2653] 10.222.0.15 () {60 vars in 1483 bytes} [Mon Aug 22 16:14:07 2022] PUT /identity/v3/projects/b0e6712e5be748ac987e09af2f3422ad/users/2779dbc1258842a48eb70daeecdfeb18/roles/e6fc1ee3fb974125ba9b4fe55b12b45e => generated 0 bytes in 220 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 22 16:14:08.044095 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a03d3f18-5aad-43e0-9097-fa10bf801c4a None None] REQUEST_METHOD: `POST` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:14:08.050451 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a03d3f18-5aad-43e0-9097-fa10bf801c4a None None] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:14:08.050451 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a03d3f18-5aad-43e0-9097-fa10bf801c4a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:14:08.056474 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a27cb350-3c43-4256-84ae-1ffe39077f9b None None] REQUEST_METHOD: `POST` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:14:08.056474 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a27cb350-3c43-4256-84ae-1ffe39077f9b None None] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:14:08.056857 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a27cb350-3c43-4256-84ae-1ffe39077f9b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:14:08.099746 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.auth.core [None req-a03d3f18-5aad-43e0-9097-fa10bf801c4a None None] MFA Rules not processed for user `2779dbc1258842a48eb70daeecdfeb18`. Rule list: `[]` (Enabled: `True`). {{(pid=81767) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 16:14:08.101020 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.fernet_utils [None req-a03d3f18-5aad-43e0-9097-fa10bf801c4a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81767) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 16:14:08.104315 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.auth.core [None req-a27cb350-3c43-4256-84ae-1ffe39077f9b None None] MFA Rules not processed for user `78933c65a377452da78e98db95ef57ea`. Rule list: `[]` (Enabled: `True`). {{(pid=81768) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 16:14:08.107700 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-a27cb350-3c43-4256-84ae-1ffe39077f9b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 16:14:08.196004 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 1347/2654] 10.222.0.15 () {58 vars in 958 bytes} [Mon Aug 22 16:14:08 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 22 16:14:08.200604 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 1308/2655] 10.222.0.15 () {58 vars in 958 bytes} [Mon Aug 22 16:14:08 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 16:14:08.209361 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a27cb350-3c43-4256-84ae-1ffe39077f9b None None] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 16:14:08.211022 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-a27cb350-3c43-4256-84ae-1ffe39077f9b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 16:14:08.213286 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a03d3f18-5aad-43e0-9097-fa10bf801c4a None None] Authenticating user token {{(pid=81767) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 16:14:08.214823 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.fernet_utils [None req-a03d3f18-5aad-43e0-9097-fa10bf801c4a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81767) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 16:14:08.316120 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a03d3f18-5aad-43e0-9097-fa10bf801c4a None None] Validating token access rules against request {{(pid=81767) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 16:14:08.316968 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a27cb350-3c43-4256-84ae-1ffe39077f9b None None] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 16:14:08.320132 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be8d880a-fc1c-4677-8e97-ae3928f2f0d7 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91f565473c824a15930e2b12863c1ed3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4ee0413f9ef4d4fb1f95a660babe090', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81767) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 16:14:08.320750 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be8d880a-fc1c-4677-8e97-ae3928f2f0d7 service glance] REQUEST_METHOD: `GET` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:14:08.321029 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be8d880a-fc1c-4677-8e97-ae3928f2f0d7 service glance] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:14:08.321300 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be8d880a-fc1c-4677-8e97-ae3928f2f0d7 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:14:08.322158 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6623fc6-8fef-44ed-8445-fe2684c1fd2c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79706d480f864998b9f8594be407ceab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4ee0413f9ef4d4fb1f95a660babe090', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 16:14:08.322941 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6623fc6-8fef-44ed-8445-fe2684c1fd2c service cinder] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:14:08.323350 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6623fc6-8fef-44ed-8445-fe2684c1fd2c service cinder] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:14:08.323742 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6623fc6-8fef-44ed-8445-fe2684c1fd2c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:14:08.339488 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be8d880a-fc1c-4677-8e97-ae3928f2f0d7 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 16:14:08.341484 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6623fc6-8fef-44ed-8445-fe2684c1fd2c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 16:14:08.343015 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6623fc6-8fef-44ed-8445-fe2684c1fd2c service cinder] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 16:14:08.344198 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be8d880a-fc1c-4677-8e97-ae3928f2f0d7 service glance] RBAC: Authorization granted {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 16:14:08.354153 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 1348/2656] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Aug 22 16:14:08 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 145 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 16:14:08.360169 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 1309/2657] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Aug 22 16:14:08 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 147 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 22 16:14:09.133076 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03b06ec6-5f4e-4b12-837c-485f22816f41 None None] REQUEST_METHOD: `POST` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:14:09.133368 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03b06ec6-5f4e-4b12-837c-485f22816f41 None None] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:14:09.133648 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03b06ec6-5f4e-4b12-837c-485f22816f41 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:14:09.147878 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 16:14:09.177543 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.auth.core [None req-03b06ec6-5f4e-4b12-837c-485f22816f41 None None] MFA Rules not processed for user `4e4eff0c95194682ade3e24aec451cd7`. Rule list: `[]` (Enabled: `True`). {{(pid=81768) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 16:14:09.178786 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-03b06ec6-5f4e-4b12-837c-485f22816f41 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 16:14:09.271784 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 1349/2658] 10.222.0.15 () {58 vars in 958 bytes} [Mon Aug 22 16:14:09 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 16:14:09.273314 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be8d880a-fc1c-4677-8e97-ae3928f2f0d7 service glance] Authenticating user token {{(pid=81767) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 16:14:09.276263 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.fernet_utils [None req-be8d880a-fc1c-4677-8e97-ae3928f2f0d7 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81767) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 16:14:09.368778 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be8d880a-fc1c-4677-8e97-ae3928f2f0d7 service glance] Validating token access rules against request {{(pid=81767) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 16:14:09.373723 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea2568fc-6aa6-45c7-88ef-140991256520 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a98b1e4431940ab8d1fc95df2858a6c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81767) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 16:14:09.374351 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea2568fc-6aa6-45c7-88ef-140991256520 admin admin] REQUEST_METHOD: `POST` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:14:09.374649 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea2568fc-6aa6-45c7-88ef-140991256520 admin admin] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:14:09.374953 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea2568fc-6aa6-45c7-88ef-140991256520 admin admin] PATH_INFO: `/v3/projects` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:14:09.375632 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea2568fc-6aa6-45c7-88ef-140991256520 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 16:14:09.377601 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea2568fc-6aa6-45c7-88ef-140991256520 admin admin] RBAC: Authorization granted {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 16:14:09.486813 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 1310/2659] 10.222.0.15 () {60 vars in 1153 bytes} [Mon Aug 22 16:14:09 2022] POST /identity/v3/projects => generated 377 bytes in 214 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 22 16:14:09.490990 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03b06ec6-5f4e-4b12-837c-485f22816f41 None None] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 16:14:09.505951 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03b06ec6-5f4e-4b12-837c-485f22816f41 None None] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 16:14:09.543239 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea2568fc-6aa6-45c7-88ef-140991256520 admin admin] Authenticating user token {{(pid=81767) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 16:14:09.545300 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.fernet_utils [None req-ea2568fc-6aa6-45c7-88ef-140991256520 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81767) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 16:14:09.646443 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea2568fc-6aa6-45c7-88ef-140991256520 admin admin] Validating token access rules against request {{(pid=81767) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 16:14:09.650997 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-637ff2a0-53be-4192-82dc-3887f16b133e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a98b1e4431940ab8d1fc95df2858a6c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 16:14:09.651503 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-637ff2a0-53be-4192-82dc-3887f16b133e admin admin] REQUEST_METHOD: `POST` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:14:09.651787 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-637ff2a0-53be-4192-82dc-3887f16b133e admin admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:14:09.652309 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-637ff2a0-53be-4192-82dc-3887f16b133e admin admin] PATH_INFO: `/v3/users` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:14:09.653314 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-637ff2a0-53be-4192-82dc-3887f16b133e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 16:14:09.655956 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-637ff2a0-53be-4192-82dc-3887f16b133e admin admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 16:14:09.656587 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58c86396-9c3f-4cc2-9178-c87204f54f42 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a98b1e4431940ab8d1fc95df2858a6c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81767) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 16:14:09.657549 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c86396-9c3f-4cc2-9178-c87204f54f42 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:14:09.657836 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c86396-9c3f-4cc2-9178-c87204f54f42 admin admin] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:14:09.658182 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c86396-9c3f-4cc2-9178-c87204f54f42 admin admin] PATH_INFO: `/v3/users/12883cc302cc404493734439ba2c3e28` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:14:09.659654 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58c86396-9c3f-4cc2-9178-c87204f54f42 admin admin] RBAC: Authorizing `identity:delete_user(user_id=12883cc302cc404493734439ba2c3e28)` {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 16:14:09.661653 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58c86396-9c3f-4cc2-9178-c87204f54f42 admin admin] RBAC: Authorization granted {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 16:14:09.766605 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 1350/2660] 10.222.0.15 () {60 vars in 1144 bytes} [Mon Aug 22 16:14:09 2022] POST /identity/v3/users => generated 295 bytes in 276 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 22 16:14:09.770688 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-637ff2a0-53be-4192-82dc-3887f16b133e admin admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 16:14:09.787457 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-637ff2a0-53be-4192-82dc-3887f16b133e admin admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 16:14:09.793631 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca6027e7-6912-42f9-bb3e-8d2422fe468d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a98b1e4431940ab8d1fc95df2858a6c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 16:14:09.794576 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca6027e7-6912-42f9-bb3e-8d2422fe468d admin admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:14:09.794862 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca6027e7-6912-42f9-bb3e-8d2422fe468d admin admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:14:09.795138 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca6027e7-6912-42f9-bb3e-8d2422fe468d admin admin] PATH_INFO: `/v3/roles` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:14:09.795834 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca6027e7-6912-42f9-bb3e-8d2422fe468d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 16:14:09.797514 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca6027e7-6912-42f9-bb3e-8d2422fe468d admin admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 16:14:09.809420 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 1351/2661] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Aug 22 16:14:09 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 22 16:14:09.811494 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG dbcounter [-] [81767] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81767) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 16:14:09.814137 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca6027e7-6912-42f9-bb3e-8d2422fe468d admin admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 16:14:09.827431 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca6027e7-6912-42f9-bb3e-8d2422fe468d admin admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 16:14:09.837768 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c5f9b87-0aba-41e4-b6de-7d3da6aaae41 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a98b1e4431940ab8d1fc95df2858a6c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 16:14:09.837768 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c5f9b87-0aba-41e4-b6de-7d3da6aaae41 admin admin] REQUEST_METHOD: `PUT` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:14:09.838190 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c5f9b87-0aba-41e4-b6de-7d3da6aaae41 admin admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:14:09.838495 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c5f9b87-0aba-41e4-b6de-7d3da6aaae41 admin admin] PATH_INFO: `/v3/projects/7845efac021d4df19d745e6cc4d00d67/users/0e9eda76738143a793abe3173c669a31/roles/9d9848cfd22e4b089db443c350e56670` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:14:09.862895 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c5f9b87-0aba-41e4-b6de-7d3da6aaae41 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7845efac021d4df19d745e6cc4d00d67, user_id=0e9eda76738143a793abe3173c669a31, role_id=9d9848cfd22e4b089db443c350e56670)` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 16:14:09.865848 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c5f9b87-0aba-41e4-b6de-7d3da6aaae41 admin admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 16:14:09.920463 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.notifications [None req-58c86396-9c3f-4cc2-9178-c87204f54f42 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '12883cc302cc404493734439ba2c3e28'} {{(pid=81767) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 16:14:09.920463 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.notifications [None req-58c86396-9c3f-4cc2-9178-c87204f54f42 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '12883cc302cc404493734439ba2c3e28'} {{(pid=81767) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 16:14:09.997085 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 1352/2662] 10.222.0.15 () {60 vars in 1483 bytes} [Mon Aug 22 16:14:09 2022] PUT /identity/v3/projects/7845efac021d4df19d745e6cc4d00d67/users/0e9eda76738143a793abe3173c669a31/roles/9d9848cfd22e4b089db443c350e56670 => generated 0 bytes in 183 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 22 16:14:10.001765 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c5f9b87-0aba-41e4-b6de-7d3da6aaae41 admin admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 16:14:10.003476 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-2c5f9b87-0aba-41e4-b6de-7d3da6aaae41 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 16:14:10.017795 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.notifications [None req-58c86396-9c3f-4cc2-9178-c87204f54f42 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '12883cc302cc404493734439ba2c3e28'} {{(pid=81767) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 16:14:10.027868 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.notifications [None req-58c86396-9c3f-4cc2-9178-c87204f54f42 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '12883cc302cc404493734439ba2c3e28'} {{(pid=81767) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 16:14:10.048929 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.notifications [None req-58c86396-9c3f-4cc2-9178-c87204f54f42 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '12883cc302cc404493734439ba2c3e28'} {{(pid=81767) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 16:14:10.056173 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 1311/2663] 10.222.0.15 () {58 vars in 1224 bytes} [Mon Aug 22 16:14:09 2022] DELETE /identity/v3/users/12883cc302cc404493734439ba2c3e28 => generated 0 bytes in 513 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 22 16:14:10.060609 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c6df59c-8bf6-480c-a319-c4d7f5b8e0d5 None None] REQUEST_METHOD: `POST` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:14:10.060892 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c6df59c-8bf6-480c-a319-c4d7f5b8e0d5 None None] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:14:10.061153 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c6df59c-8bf6-480c-a319-c4d7f5b8e0d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:14:10.081491 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.auth.core [None req-4c6df59c-8bf6-480c-a319-c4d7f5b8e0d5 None None] MFA Rules not processed for user `6b2199ae0fb0480aa917ada391f5ec54`. Rule list: `[]` (Enabled: `True`). {{(pid=81767) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 16:14:10.082326 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.fernet_utils [None req-4c6df59c-8bf6-480c-a319-c4d7f5b8e0d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81767) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 16:14:10.089035 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c5f9b87-0aba-41e4-b6de-7d3da6aaae41 admin admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 16:14:10.149929 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 1312/2664] 10.222.0.15 () {58 vars in 958 bytes} [Mon Aug 22 16:14:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 22 16:14:10.150081 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94d8999b-511c-4d5a-b8d0-c34237c5eebc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a98b1e4431940ab8d1fc95df2858a6c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 16:14:10.150508 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94d8999b-511c-4d5a-b8d0-c34237c5eebc admin admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:14:10.150793 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94d8999b-511c-4d5a-b8d0-c34237c5eebc admin admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:14:10.151073 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94d8999b-511c-4d5a-b8d0-c34237c5eebc admin admin] PATH_INFO: `/v3/roles` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:14:10.151773 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94d8999b-511c-4d5a-b8d0-c34237c5eebc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 16:14:10.158880 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94d8999b-511c-4d5a-b8d0-c34237c5eebc admin admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 16:14:10.160231 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c6df59c-8bf6-480c-a319-c4d7f5b8e0d5 None None] Authenticating user token {{(pid=81767) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 16:14:10.161643 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.fernet_utils [None req-4c6df59c-8bf6-480c-a319-c4d7f5b8e0d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81767) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 16:14:10.168856 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 1353/2665] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Aug 22 16:14:10 2022] GET /identity/v3/roles => generated 1420 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 22 16:14:10.170247 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94d8999b-511c-4d5a-b8d0-c34237c5eebc admin admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 16:14:10.185299 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94d8999b-511c-4d5a-b8d0-c34237c5eebc admin admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 16:14:10.191133 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38a360b9-1905-4586-9017-f37f14d4ccdb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a98b1e4431940ab8d1fc95df2858a6c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 16:14:10.191864 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38a360b9-1905-4586-9017-f37f14d4ccdb admin admin] REQUEST_METHOD: `PUT` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:14:10.192197 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38a360b9-1905-4586-9017-f37f14d4ccdb admin admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:14:10.192612 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38a360b9-1905-4586-9017-f37f14d4ccdb admin admin] PATH_INFO: `/v3/projects/7845efac021d4df19d745e6cc4d00d67/users/0e9eda76738143a793abe3173c669a31/roles/9d9848cfd22e4b089db443c350e56670` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:14:10.194731 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38a360b9-1905-4586-9017-f37f14d4ccdb admin admin] RBAC: Authorizing `identity:create_grant(project_id=7845efac021d4df19d745e6cc4d00d67, user_id=0e9eda76738143a793abe3173c669a31, role_id=9d9848cfd22e4b089db443c350e56670)` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 16:14:10.201783 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38a360b9-1905-4586-9017-f37f14d4ccdb admin admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 16:14:10.213610 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 1354/2666] 10.222.0.15 () {60 vars in 1483 bytes} [Mon Aug 22 16:14:10 2022] PUT /identity/v3/projects/7845efac021d4df19d745e6cc4d00d67/users/0e9eda76738143a793abe3173c669a31/roles/9d9848cfd22e4b089db443c350e56670 => generated 0 bytes in 44 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 16:14:10.226301 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c6df59c-8bf6-480c-a319-c4d7f5b8e0d5 None None] Validating token access rules against request {{(pid=81767) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 16:14:10.285848 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a14c7c31-c9a7-45c9-8ecc-cfbc9b3848bf service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '234687adad594f1aa45775a565ffbdc7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4ee0413f9ef4d4fb1f95a660babe090', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81767) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 16:14:10.286435 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a14c7c31-c9a7-45c9-8ecc-cfbc9b3848bf service neutron] REQUEST_METHOD: `GET` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:14:10.286710 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a14c7c31-c9a7-45c9-8ecc-cfbc9b3848bf service neutron] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:14:10.287047 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a14c7c31-c9a7-45c9-8ecc-cfbc9b3848bf service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:14:10.368000 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 16:14:10.378194 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a14c7c31-c9a7-45c9-8ecc-cfbc9b3848bf service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 16:14:10.379890 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a14c7c31-c9a7-45c9-8ecc-cfbc9b3848bf service neutron] RBAC: Authorization granted {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 16:14:10.391718 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 1313/2667] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Aug 22 16:14:10 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 232 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 16:14:10.918598 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38a360b9-1905-4586-9017-f37f14d4ccdb admin admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 16:14:10.920421 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-38a360b9-1905-4586-9017-f37f14d4ccdb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 16:14:10.937664 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG dbcounter [-] [81767] Writing DB stats keystone:SELECT=93,keystone:INSERT=1,keystone:DELETE=6 {{(pid=81767) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 16:14:10.938878 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38a360b9-1905-4586-9017-f37f14d4ccdb admin admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 16:14:10.945355 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8053f924-cce5-432c-8bf0-66aabf4209f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a98b1e4431940ab8d1fc95df2858a6c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 16:14:10.946053 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8053f924-cce5-432c-8bf0-66aabf4209f0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:14:10.948629 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8053f924-cce5-432c-8bf0-66aabf4209f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:14:10.948629 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8053f924-cce5-432c-8bf0-66aabf4209f0 admin admin] PATH_INFO: `/v3/projects/505e7545cef44a3483118ab73b66c5ba` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:14:10.950648 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8053f924-cce5-432c-8bf0-66aabf4209f0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=505e7545cef44a3483118ab73b66c5ba)` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 16:14:10.955100 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8053f924-cce5-432c-8bf0-66aabf4209f0 admin admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 16:14:11.128436 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-8053f924-cce5-432c-8bf0-66aabf4209f0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '505e7545cef44a3483118ab73b66c5ba', 'request_id': 'req-8053f924-cce5-432c-8bf0-66aabf4209f0'} {{(pid=81768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 16:14:11.134759 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-8053f924-cce5-432c-8bf0-66aabf4209f0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '505e7545cef44a3483118ab73b66c5ba', 'request_id': 'req-8053f924-cce5-432c-8bf0-66aabf4209f0'} {{(pid=81768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 16:14:11.224820 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-8053f924-cce5-432c-8bf0-66aabf4209f0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '505e7545cef44a3483118ab73b66c5ba', 'request_id': 'req-8053f924-cce5-432c-8bf0-66aabf4209f0'} {{(pid=81768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 16:14:11.238971 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-8053f924-cce5-432c-8bf0-66aabf4209f0 admin admin] The token cache is being invalidate because project 505e7545cef44a3483118ab73b66c5ba was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81768) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 22 16:14:11.239586 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-8053f924-cce5-432c-8bf0-66aabf4209f0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 16:14:11.240252 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-8053f924-cce5-432c-8bf0-66aabf4209f0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 16:14:11.242065 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 1355/2668] 10.222.0.15 () {58 vars in 1233 bytes} [Mon Aug 22 16:14:10 2022] DELETE /identity/v3/projects/505e7545cef44a3483118ab73b66c5ba => generated 0 bytes in 324 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 16:14:11.247234 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11aa6fe1-4689-4ed3-b2cc-54c30f58bb76 None None] REQUEST_METHOD: `POST` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:14:11.247555 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11aa6fe1-4689-4ed3-b2cc-54c30f58bb76 None None] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:14:11.247828 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11aa6fe1-4689-4ed3-b2cc-54c30f58bb76 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:14:11.273189 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.auth.core [None req-11aa6fe1-4689-4ed3-b2cc-54c30f58bb76 None None] MFA Rules not processed for user `6b2199ae0fb0480aa917ada391f5ec54`. Rule list: `[]` (Enabled: `True`). {{(pid=81767) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 16:14:11.274400 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.fernet_utils [None req-11aa6fe1-4689-4ed3-b2cc-54c30f58bb76 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81767) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 16:14:11.350811 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 1314/2669] 10.222.0.15 () {58 vars in 958 bytes} [Mon Aug 22 16:14:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 16:14:11.358784 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8900988f-8477-4c5f-a538-783d84490416 None None] REQUEST_METHOD: `POST` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:14:11.359492 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8900988f-8477-4c5f-a538-783d84490416 None None] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:14:11.359781 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8900988f-8477-4c5f-a538-783d84490416 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:14:11.380797 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.auth.core [None req-8900988f-8477-4c5f-a538-783d84490416 None None] MFA Rules not processed for user `6b2199ae0fb0480aa917ada391f5ec54`. Rule list: `[]` (Enabled: `True`). {{(pid=81768) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 16:14:11.382484 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-8900988f-8477-4c5f-a538-783d84490416 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 16:14:11.391952 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 1356/2670] 10.222.0.15 () {58 vars in 958 bytes} [Mon Aug 22 16:14:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 16:14:11.396190 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11aa6fe1-4689-4ed3-b2cc-54c30f58bb76 None None] Authenticating user token {{(pid=81767) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 16:14:11.405658 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11aa6fe1-4689-4ed3-b2cc-54c30f58bb76 None None] Validating token access rules against request {{(pid=81767) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 16:14:11.410458 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f575a82-8d6d-4a1c-8b45-c01cea16eec8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a98b1e4431940ab8d1fc95df2858a6c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81767) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 16:14:11.411353 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f575a82-8d6d-4a1c-8b45-c01cea16eec8 admin admin] REQUEST_METHOD: `GET` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:14:11.411812 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f575a82-8d6d-4a1c-8b45-c01cea16eec8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:14:11.412244 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f575a82-8d6d-4a1c-8b45-c01cea16eec8 admin admin] PATH_INFO: `/v3/domains` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:14:11.412924 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f575a82-8d6d-4a1c-8b45-c01cea16eec8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81767) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 16:14:11.413211 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f575a82-8d6d-4a1c-8b45-c01cea16eec8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 16:14:11.414990 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f575a82-8d6d-4a1c-8b45-c01cea16eec8 admin admin] RBAC: Authorization granted {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 16:14:11.428019 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 1315/2671] 10.222.0.15 () {58 vars in 1153 bytes} [Mon Aug 22 16:14:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 16:14:11.432569 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8900988f-8477-4c5f-a538-783d84490416 None None] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 16:14:11.442437 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8900988f-8477-4c5f-a538-783d84490416 None None] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 16:14:11.447093 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf5055bb-113a-4c41-950d-f3b4194f654b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a98b1e4431940ab8d1fc95df2858a6c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 16:14:11.447705 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf5055bb-113a-4c41-950d-f3b4194f654b admin admin] REQUEST_METHOD: `POST` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:14:11.447979 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf5055bb-113a-4c41-950d-f3b4194f654b admin admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:14:11.448244 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf5055bb-113a-4c41-950d-f3b4194f654b admin admin] PATH_INFO: `/v3/projects` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:14:11.448851 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf5055bb-113a-4c41-950d-f3b4194f654b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 16:14:11.450729 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf5055bb-113a-4c41-950d-f3b4194f654b admin admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 16:14:11.590880 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 1357/2672] 10.222.0.15 () {60 vars in 1153 bytes} [Mon Aug 22 16:14:11 2022] POST /identity/v3/projects => generated 377 bytes in 159 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 22 16:14:11.591721 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f575a82-8d6d-4a1c-8b45-c01cea16eec8 admin admin] Authenticating user token {{(pid=81767) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 16:14:11.623708 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f575a82-8d6d-4a1c-8b45-c01cea16eec8 admin admin] Validating token access rules against request {{(pid=81767) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 16:14:11.680341 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:INSERT=4,keystone:SELECT=93,keystone:DELETE=3 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 16:14:11.696629 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-318d991f-ea5c-4891-855f-21e73e8188fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a98b1e4431940ab8d1fc95df2858a6c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81767) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 16:14:11.697456 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-318d991f-ea5c-4891-855f-21e73e8188fd admin admin] REQUEST_METHOD: `POST` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:14:11.698436 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-318d991f-ea5c-4891-855f-21e73e8188fd admin admin] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:14:11.698881 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-318d991f-ea5c-4891-855f-21e73e8188fd admin admin] PATH_INFO: `/v3/users` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:14:11.699542 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-318d991f-ea5c-4891-855f-21e73e8188fd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 16:14:11.701868 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-318d991f-ea5c-4891-855f-21e73e8188fd admin admin] RBAC: Authorization granted {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 16:14:11.799383 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 1316/2673] 10.222.0.15 () {60 vars in 1144 bytes} [Mon Aug 22 16:14:11 2022] POST /identity/v3/users => generated 289 bytes in 208 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 22 16:14:11.803868 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf5055bb-113a-4c41-950d-f3b4194f654b admin admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 16:14:11.821776 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf5055bb-113a-4c41-950d-f3b4194f654b admin admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 16:14:11.841109 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2a8b951-0f89-4b34-aa9c-e87a01f477e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a98b1e4431940ab8d1fc95df2858a6c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 16:14:11.841109 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2a8b951-0f89-4b34-aa9c-e87a01f477e7 admin admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:14:11.841109 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2a8b951-0f89-4b34-aa9c-e87a01f477e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:14:11.841109 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2a8b951-0f89-4b34-aa9c-e87a01f477e7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:14:11.841878 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2a8b951-0f89-4b34-aa9c-e87a01f477e7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 16:14:11.841878 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2a8b951-0f89-4b34-aa9c-e87a01f477e7 admin admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 16:14:11.865202 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 1358/2674] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Aug 22 16:14:11 2022] GET /identity/v3/roles => generated 1420 bytes in 62 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 22 16:14:11.869448 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-318d991f-ea5c-4891-855f-21e73e8188fd admin admin] Authenticating user token {{(pid=81767) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 16:14:11.884338 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-318d991f-ea5c-4891-855f-21e73e8188fd admin admin] Validating token access rules against request {{(pid=81767) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 16:14:11.891538 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-345873aa-64a7-42dc-a771-46c1540d075d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a98b1e4431940ab8d1fc95df2858a6c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81767) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 16:14:11.892638 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-345873aa-64a7-42dc-a771-46c1540d075d admin admin] REQUEST_METHOD: `GET` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:14:11.893243 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-345873aa-64a7-42dc-a771-46c1540d075d admin admin] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:14:11.894200 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-345873aa-64a7-42dc-a771-46c1540d075d admin admin] PATH_INFO: `/v3/roles` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:14:11.895043 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-345873aa-64a7-42dc-a771-46c1540d075d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 16:14:11.897863 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-345873aa-64a7-42dc-a771-46c1540d075d admin admin] RBAC: Authorization granted {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 16:14:11.914625 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 1317/2675] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Aug 22 16:14:11 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 22 16:14:11.918648 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2a8b951-0f89-4b34-aa9c-e87a01f477e7 admin admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 16:14:11.931233 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2a8b951-0f89-4b34-aa9c-e87a01f477e7 admin admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 16:14:11.936510 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b444cbdb-ca1b-4509-96b9-270b083b8ed1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a98b1e4431940ab8d1fc95df2858a6c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 16:14:11.937233 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b444cbdb-ca1b-4509-96b9-270b083b8ed1 admin admin] REQUEST_METHOD: `PUT` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:14:11.937512 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b444cbdb-ca1b-4509-96b9-270b083b8ed1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:14:11.937776 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b444cbdb-ca1b-4509-96b9-270b083b8ed1 admin admin] PATH_INFO: `/v3/projects/fe13b32187b74ac995fe3f664e7acc77/users/3885352af4e049ca99dad86b3dad97e3/roles/e6fc1ee3fb974125ba9b4fe55b12b45e` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:14:11.956148 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b444cbdb-ca1b-4509-96b9-270b083b8ed1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fe13b32187b74ac995fe3f664e7acc77, user_id=3885352af4e049ca99dad86b3dad97e3, role_id=e6fc1ee3fb974125ba9b4fe55b12b45e)` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 16:14:11.959779 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b444cbdb-ca1b-4509-96b9-270b083b8ed1 admin admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 16:14:12.099194 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 1359/2676] 10.222.0.15 () {60 vars in 1483 bytes} [Mon Aug 22 16:14:11 2022] PUT /identity/v3/projects/fe13b32187b74ac995fe3f664e7acc77/users/3885352af4e049ca99dad86b3dad97e3/roles/e6fc1ee3fb974125ba9b4fe55b12b45e => generated 0 bytes in 181 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 16:14:12.112329 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcf15eff-54b4-466e-8d5b-45e44bea8a1b None None] REQUEST_METHOD: `POST` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:14:12.112858 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcf15eff-54b4-466e-8d5b-45e44bea8a1b None None] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:14:12.113373 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcf15eff-54b4-466e-8d5b-45e44bea8a1b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:14:12.169786 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.auth.core [None req-fcf15eff-54b4-466e-8d5b-45e44bea8a1b None None] MFA Rules not processed for user `3885352af4e049ca99dad86b3dad97e3`. Rule list: `[]` (Enabled: `True`). {{(pid=81767) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 16:14:12.172846 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.fernet_utils [None req-fcf15eff-54b4-466e-8d5b-45e44bea8a1b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81767) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 16:14:12.291443 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 1318/2677] 10.222.0.15 () {58 vars in 958 bytes} [Mon Aug 22 16:14:12 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 16:14:12.296424 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b444cbdb-ca1b-4509-96b9-270b083b8ed1 admin admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 16:14:12.412890 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b444cbdb-ca1b-4509-96b9-270b083b8ed1 admin admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 16:14:12.426842 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a953a22c-6599-469f-92fe-cd66ba63acbf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a98b1e4431940ab8d1fc95df2858a6c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 16:14:12.427510 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a953a22c-6599-469f-92fe-cd66ba63acbf admin admin] REQUEST_METHOD: `POST` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:14:12.427794 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a953a22c-6599-469f-92fe-cd66ba63acbf admin admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:14:12.428071 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a953a22c-6599-469f-92fe-cd66ba63acbf admin admin] PATH_INFO: `/v3/projects` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:14:12.434097 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a953a22c-6599-469f-92fe-cd66ba63acbf admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 16:14:12.435826 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a953a22c-6599-469f-92fe-cd66ba63acbf admin admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 16:14:12.539948 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 1360/2678] 10.222.0.15 () {60 vars in 1153 bytes} [Mon Aug 22 16:14:12 2022] POST /identity/v3/projects => generated 375 bytes in 241 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 22 16:14:12.545512 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcf15eff-54b4-466e-8d5b-45e44bea8a1b None None] Authenticating user token {{(pid=81767) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 16:14:12.560157 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcf15eff-54b4-466e-8d5b-45e44bea8a1b None None] Validating token access rules against request {{(pid=81767) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 16:14:12.665676 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84cc06ee-52d7-484e-a4fc-b3cc8928d8d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a98b1e4431940ab8d1fc95df2858a6c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81767) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 16:14:12.666440 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84cc06ee-52d7-484e-a4fc-b3cc8928d8d2 admin admin] REQUEST_METHOD: `POST` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:14:12.667097 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84cc06ee-52d7-484e-a4fc-b3cc8928d8d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:14:12.667652 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84cc06ee-52d7-484e-a4fc-b3cc8928d8d2 admin admin] PATH_INFO: `/v3/users` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:14:12.668723 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84cc06ee-52d7-484e-a4fc-b3cc8928d8d2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 16:14:12.679073 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84cc06ee-52d7-484e-a4fc-b3cc8928d8d2 admin admin] RBAC: Authorization granted {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 16:14:12.769863 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG dbcounter [-] [81767] Writing DB stats keystone:SELECT=95,keystone:DELETE=4,keystone:INSERT=1 {{(pid=81767) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 16:14:12.776264 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 1319/2679] 10.222.0.15 () {60 vars in 1144 bytes} [Mon Aug 22 16:14:12 2022] POST /identity/v3/users => generated 288 bytes in 235 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 22 16:14:12.780220 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a953a22c-6599-469f-92fe-cd66ba63acbf admin admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 16:14:12.815535 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a953a22c-6599-469f-92fe-cd66ba63acbf admin admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 16:14:12.820305 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd8b12c9-1cb3-4e92-8a92-fa375f286aba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a98b1e4431940ab8d1fc95df2858a6c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 16:14:12.821250 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd8b12c9-1cb3-4e92-8a92-fa375f286aba admin admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:14:12.821862 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd8b12c9-1cb3-4e92-8a92-fa375f286aba admin admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:14:12.822342 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd8b12c9-1cb3-4e92-8a92-fa375f286aba admin admin] PATH_INFO: `/v3/roles` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:14:12.823212 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd8b12c9-1cb3-4e92-8a92-fa375f286aba admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 16:14:12.825424 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd8b12c9-1cb3-4e92-8a92-fa375f286aba admin admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 16:14:12.849357 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 1361/2680] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Aug 22 16:14:12 2022] GET /identity/v3/roles => generated 1420 bytes in 70 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 22 16:14:12.853611 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84cc06ee-52d7-484e-a4fc-b3cc8928d8d2 admin admin] Authenticating user token {{(pid=81767) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 16:14:12.867286 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84cc06ee-52d7-484e-a4fc-b3cc8928d8d2 admin admin] Validating token access rules against request {{(pid=81767) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 16:14:12.883210 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bbc92e3-4526-4881-91c1-4be150757665 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a98b1e4431940ab8d1fc95df2858a6c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81767) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 16:14:12.883898 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bbc92e3-4526-4881-91c1-4be150757665 admin admin] REQUEST_METHOD: `GET` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:14:12.884180 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bbc92e3-4526-4881-91c1-4be150757665 admin admin] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:14:12.884457 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bbc92e3-4526-4881-91c1-4be150757665 admin admin] PATH_INFO: `/v3/roles` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:14:12.885249 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bbc92e3-4526-4881-91c1-4be150757665 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 16:14:12.887688 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bbc92e3-4526-4881-91c1-4be150757665 admin admin] RBAC: Authorization granted {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 16:14:12.914706 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 1320/2681] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Aug 22 16:14:12 2022] GET /identity/v3/roles => generated 1420 bytes in 61 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 22 16:14:12.920800 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd8b12c9-1cb3-4e92-8a92-fa375f286aba admin admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 16:14:12.934135 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd8b12c9-1cb3-4e92-8a92-fa375f286aba admin admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 16:14:12.940674 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1525eb3d-a6af-44aa-8e89-b463efc63401 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a98b1e4431940ab8d1fc95df2858a6c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 16:14:12.942027 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1525eb3d-a6af-44aa-8e89-b463efc63401 admin admin] REQUEST_METHOD: `PUT` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:14:12.942450 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1525eb3d-a6af-44aa-8e89-b463efc63401 admin admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:14:12.942858 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1525eb3d-a6af-44aa-8e89-b463efc63401 admin admin] PATH_INFO: `/v3/projects/4ad284e04f904cb7a1fd0181f96ad71c/users/7870c436e21a4434b62bfef1a001dce8/roles/e6fc1ee3fb974125ba9b4fe55b12b45e` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:14:13.013237 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1525eb3d-a6af-44aa-8e89-b463efc63401 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4ad284e04f904cb7a1fd0181f96ad71c, user_id=7870c436e21a4434b62bfef1a001dce8, role_id=e6fc1ee3fb974125ba9b4fe55b12b45e)` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 16:14:13.017067 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1525eb3d-a6af-44aa-8e89-b463efc63401 admin admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 16:14:13.147609 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 1362/2682] 10.222.0.15 () {60 vars in 1483 bytes} [Mon Aug 22 16:14:12 2022] PUT /identity/v3/projects/4ad284e04f904cb7a1fd0181f96ad71c/users/7870c436e21a4434b62bfef1a001dce8/roles/e6fc1ee3fb974125ba9b4fe55b12b45e => generated 0 bytes in 228 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 16:14:13.158273 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-956b2ba7-d6c9-41bd-8acf-4a6a371552a7 None None] REQUEST_METHOD: `POST` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:14:13.158563 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-956b2ba7-d6c9-41bd-8acf-4a6a371552a7 None None] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:14:13.158832 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-956b2ba7-d6c9-41bd-8acf-4a6a371552a7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:14:13.201459 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.auth.core [None req-956b2ba7-d6c9-41bd-8acf-4a6a371552a7 None None] MFA Rules not processed for user `7870c436e21a4434b62bfef1a001dce8`. Rule list: `[]` (Enabled: `True`). {{(pid=81767) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 16:14:13.202000 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.fernet_utils [None req-956b2ba7-d6c9-41bd-8acf-4a6a371552a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81767) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 16:14:13.257473 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1525eb3d-a6af-44aa-8e89-b463efc63401 admin admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 16:14:13.260270 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-1525eb3d-a6af-44aa-8e89-b463efc63401 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 16:14:13.303744 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 1321/2683] 10.222.0.15 () {58 vars in 958 bytes} [Mon Aug 22 16:14:13 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 16:14:13.304801 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-956b2ba7-d6c9-41bd-8acf-4a6a371552a7 None None] Authenticating user token {{(pid=81767) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 16:14:13.362154 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1525eb3d-a6af-44aa-8e89-b463efc63401 admin admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 16:14:13.368150 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b080df1-e11c-45e9-808d-cdb744ca63f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a98b1e4431940ab8d1fc95df2858a6c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 16:14:13.368849 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b080df1-e11c-45e9-808d-cdb744ca63f4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:14:13.369334 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b080df1-e11c-45e9-808d-cdb744ca63f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:14:13.372900 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b080df1-e11c-45e9-808d-cdb744ca63f4 admin admin] PATH_INFO: `/v3/users/78933c65a377452da78e98db95ef57ea` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:14:13.374544 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-956b2ba7-d6c9-41bd-8acf-4a6a371552a7 None None] Validating token access rules against request {{(pid=81767) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 16:14:13.376523 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b080df1-e11c-45e9-808d-cdb744ca63f4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=78933c65a377452da78e98db95ef57ea)` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 16:14:13.379254 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b080df1-e11c-45e9-808d-cdb744ca63f4 admin admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 16:14:13.379447 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7df87a5-85a7-4e79-9162-e2bcc8a58e67 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a98b1e4431940ab8d1fc95df2858a6c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81767) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 16:14:13.380101 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7df87a5-85a7-4e79-9162-e2bcc8a58e67 admin admin] REQUEST_METHOD: `POST` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:14:13.380391 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7df87a5-85a7-4e79-9162-e2bcc8a58e67 admin admin] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:14:13.380665 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7df87a5-85a7-4e79-9162-e2bcc8a58e67 admin admin] PATH_INFO: `/v3/projects` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:14:13.381298 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7df87a5-85a7-4e79-9162-e2bcc8a58e67 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 16:14:13.383197 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7df87a5-85a7-4e79-9162-e2bcc8a58e67 admin admin] RBAC: Authorization granted {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 16:14:13.485030 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 1322/2684] 10.222.0.15 () {60 vars in 1153 bytes} [Mon Aug 22 16:14:13 2022] POST /identity/v3/projects => generated 377 bytes in 181 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 22 16:14:13.492430 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7df87a5-85a7-4e79-9162-e2bcc8a58e67 admin admin] Authenticating user token {{(pid=81767) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 16:14:13.587155 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-2b080df1-e11c-45e9-808d-cdb744ca63f4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '78933c65a377452da78e98db95ef57ea'} {{(pid=81768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 16:14:13.588147 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-2b080df1-e11c-45e9-808d-cdb744ca63f4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '78933c65a377452da78e98db95ef57ea'} {{(pid=81768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 16:14:13.588981 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-2b080df1-e11c-45e9-808d-cdb744ca63f4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '78933c65a377452da78e98db95ef57ea'} {{(pid=81768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 16:14:13.590585 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7df87a5-85a7-4e79-9162-e2bcc8a58e67 admin admin] Validating token access rules against request {{(pid=81767) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 16:14:13.597647 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f90765e-73f8-4de2-8846-8b28a3483f74 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a98b1e4431940ab8d1fc95df2858a6c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81767) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 16:14:13.598282 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f90765e-73f8-4de2-8846-8b28a3483f74 admin admin] REQUEST_METHOD: `POST` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:14:13.598951 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f90765e-73f8-4de2-8846-8b28a3483f74 admin admin] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:14:13.599364 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f90765e-73f8-4de2-8846-8b28a3483f74 admin admin] PATH_INFO: `/v3/users` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:14:13.600065 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f90765e-73f8-4de2-8846-8b28a3483f74 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 16:14:13.600264 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-2b080df1-e11c-45e9-808d-cdb744ca63f4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '78933c65a377452da78e98db95ef57ea'} {{(pid=81768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 16:14:13.602149 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f90765e-73f8-4de2-8846-8b28a3483f74 admin admin] RBAC: Authorization granted {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 16:14:13.630996 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=91,keystone:DELETE=4,keystone:INSERT=5 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 16:14:13.659290 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-2b080df1-e11c-45e9-808d-cdb744ca63f4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '78933c65a377452da78e98db95ef57ea'} {{(pid=81768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 16:14:13.684188 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 1363/2685] 10.222.0.15 () {58 vars in 1224 bytes} [Mon Aug 22 16:14:13 2022] DELETE /identity/v3/users/78933c65a377452da78e98db95ef57ea => generated 0 bytes in 427 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 16:14:13.689481 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74a6769e-c38d-4a1a-aab9-3b59b391a37b None None] REQUEST_METHOD: `POST` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:14:13.690161 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74a6769e-c38d-4a1a-aab9-3b59b391a37b None None] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:14:13.690443 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74a6769e-c38d-4a1a-aab9-3b59b391a37b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:14:13.727958 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 1323/2686] 10.222.0.15 () {60 vars in 1144 bytes} [Mon Aug 22 16:14:13 2022] POST /identity/v3/users => generated 295 bytes in 239 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 22 16:14:13.728983 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f90765e-73f8-4de2-8846-8b28a3483f74 admin admin] Authenticating user token {{(pid=81767) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 16:14:13.734125 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.auth.core [None req-74a6769e-c38d-4a1a-aab9-3b59b391a37b None None] MFA Rules not processed for user `6b2199ae0fb0480aa917ada391f5ec54`. Rule list: `[]` (Enabled: `True`). {{(pid=81768) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 16:14:13.736828 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-74a6769e-c38d-4a1a-aab9-3b59b391a37b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 16:14:13.739269 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.fernet_utils [None req-3f90765e-73f8-4de2-8846-8b28a3483f74 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81767) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 16:14:13.775871 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f90765e-73f8-4de2-8846-8b28a3483f74 admin admin] Validating token access rules against request {{(pid=81767) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 16:14:13.866479 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1078897-36a8-4381-ae29-2f6be192e005 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a98b1e4431940ab8d1fc95df2858a6c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81767) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 16:14:13.871716 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1078897-36a8-4381-ae29-2f6be192e005 admin admin] REQUEST_METHOD: `GET` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:14:13.871716 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1078897-36a8-4381-ae29-2f6be192e005 admin admin] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:14:13.871716 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1078897-36a8-4381-ae29-2f6be192e005 admin admin] PATH_INFO: `/v3/roles` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:14:13.871716 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1078897-36a8-4381-ae29-2f6be192e005 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 16:14:13.871716 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1078897-36a8-4381-ae29-2f6be192e005 admin admin] RBAC: Authorization granted {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 16:14:13.876435 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 1364/2687] 10.222.0.15 () {58 vars in 958 bytes} [Mon Aug 22 16:14:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 16:14:13.882652 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74a6769e-c38d-4a1a-aab9-3b59b391a37b None None] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 16:14:13.884758 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-74a6769e-c38d-4a1a-aab9-3b59b391a37b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 16:14:13.885936 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 1324/2688] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Aug 22 16:14:13 2022] GET /identity/v3/roles => generated 1420 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 22 16:14:13.891688 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1078897-36a8-4381-ae29-2f6be192e005 admin admin] Authenticating user token {{(pid=81767) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 16:14:13.903226 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1078897-36a8-4381-ae29-2f6be192e005 admin admin] Validating token access rules against request {{(pid=81767) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 16:14:13.926384 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-308fd154-cb6a-4d0e-a153-9e2ba465d0e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a98b1e4431940ab8d1fc95df2858a6c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81767) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 16:14:13.929018 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-308fd154-cb6a-4d0e-a153-9e2ba465d0e8 admin admin] REQUEST_METHOD: `PUT` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:14:13.929018 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-308fd154-cb6a-4d0e-a153-9e2ba465d0e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:14:13.929018 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-308fd154-cb6a-4d0e-a153-9e2ba465d0e8 admin admin] PATH_INFO: `/v3/projects/74a18ea26b044f788f061c2a7a2ceebe/users/f7e4945c5bb549ba8b26af82e8d8ce61/roles/9d9848cfd22e4b089db443c350e56670` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:14:13.968433 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-308fd154-cb6a-4d0e-a153-9e2ba465d0e8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=74a18ea26b044f788f061c2a7a2ceebe, user_id=f7e4945c5bb549ba8b26af82e8d8ce61, role_id=9d9848cfd22e4b089db443c350e56670)` {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 16:14:13.975798 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-308fd154-cb6a-4d0e-a153-9e2ba465d0e8 admin admin] RBAC: Authorization granted {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 16:14:13.998236 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74a6769e-c38d-4a1a-aab9-3b59b391a37b None None] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 16:14:14.003955 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-458012e7-00f5-4453-8d6d-232d9f5b2d8d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '234687adad594f1aa45775a565ffbdc7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4ee0413f9ef4d4fb1f95a660babe090', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 16:14:14.005053 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-458012e7-00f5-4453-8d6d-232d9f5b2d8d service neutron] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:14:14.005053 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-458012e7-00f5-4453-8d6d-232d9f5b2d8d service neutron] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:14:14.005461 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-458012e7-00f5-4453-8d6d-232d9f5b2d8d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:14:14.025809 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-458012e7-00f5-4453-8d6d-232d9f5b2d8d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 16:14:14.027262 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-458012e7-00f5-4453-8d6d-232d9f5b2d8d service neutron] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 16:14:14.047255 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 1365/2689] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Aug 22 16:14:13 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 169 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 16:14:14.103228 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 1325/2690] 10.222.0.15 () {60 vars in 1483 bytes} [Mon Aug 22 16:14:13 2022] PUT /identity/v3/projects/74a18ea26b044f788f061c2a7a2ceebe/users/f7e4945c5bb549ba8b26af82e8d8ce61/roles/9d9848cfd22e4b089db443c350e56670 => generated 0 bytes in 212 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 16:14:14.106995 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-458012e7-00f5-4453-8d6d-232d9f5b2d8d service neutron] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 16:14:14.122012 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-458012e7-00f5-4453-8d6d-232d9f5b2d8d service neutron] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 16:14:14.210563 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4a948c8-0c6b-4efa-b222-053e7d1c39cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a98b1e4431940ab8d1fc95df2858a6c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 16:14:14.211171 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4a948c8-0c6b-4efa-b222-053e7d1c39cd admin admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:14:14.211482 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4a948c8-0c6b-4efa-b222-053e7d1c39cd admin admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:14:14.211889 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4a948c8-0c6b-4efa-b222-053e7d1c39cd admin admin] PATH_INFO: `/v3/roles` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:14:14.216166 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4a948c8-0c6b-4efa-b222-053e7d1c39cd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 16:14:14.216166 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4a948c8-0c6b-4efa-b222-053e7d1c39cd admin admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 16:14:14.224898 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 1366/2691] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Aug 22 16:14:14 2022] GET /identity/v3/roles => generated 1420 bytes in 118 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 22 16:14:14.229257 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-308fd154-cb6a-4d0e-a153-9e2ba465d0e8 admin admin] Authenticating user token {{(pid=81767) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 16:14:14.240801 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-308fd154-cb6a-4d0e-a153-9e2ba465d0e8 admin admin] Validating token access rules against request {{(pid=81767) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 16:14:14.245673 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56958896-b45c-4add-95cc-f518d2197146 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a98b1e4431940ab8d1fc95df2858a6c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81767) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 16:14:14.246876 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56958896-b45c-4add-95cc-f518d2197146 admin admin] REQUEST_METHOD: `PUT` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:14:14.246876 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56958896-b45c-4add-95cc-f518d2197146 admin admin] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:14:14.247133 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56958896-b45c-4add-95cc-f518d2197146 admin admin] PATH_INFO: `/v3/projects/74a18ea26b044f788f061c2a7a2ceebe/users/f7e4945c5bb549ba8b26af82e8d8ce61/roles/9d9848cfd22e4b089db443c350e56670` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:14:14.248986 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56958896-b45c-4add-95cc-f518d2197146 admin admin] RBAC: Authorizing `identity:create_grant(project_id=74a18ea26b044f788f061c2a7a2ceebe, user_id=f7e4945c5bb549ba8b26af82e8d8ce61, role_id=9d9848cfd22e4b089db443c350e56670)` {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 16:14:14.252184 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56958896-b45c-4add-95cc-f518d2197146 admin admin] RBAC: Authorization granted {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 16:14:14.268406 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 1326/2692] 10.222.0.15 () {60 vars in 1483 bytes} [Mon Aug 22 16:14:14 2022] PUT /identity/v3/projects/74a18ea26b044f788f061c2a7a2ceebe/users/f7e4945c5bb549ba8b26af82e8d8ce61/roles/9d9848cfd22e4b089db443c350e56670 => generated 0 bytes in 40 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 16:14:14.280357 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f6f76ca-c1bd-4bbc-ae19-59508ce2fbe5 None None] REQUEST_METHOD: `POST` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:14:14.280357 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f6f76ca-c1bd-4bbc-ae19-59508ce2fbe5 None None] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:14:14.280357 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f6f76ca-c1bd-4bbc-ae19-59508ce2fbe5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:14:14.313209 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.auth.core [None req-0f6f76ca-c1bd-4bbc-ae19-59508ce2fbe5 None None] MFA Rules not processed for user `f7e4945c5bb549ba8b26af82e8d8ce61`. Rule list: `[]` (Enabled: `True`). {{(pid=81768) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 16:14:14.314432 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-0f6f76ca-c1bd-4bbc-ae19-59508ce2fbe5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 16:14:14.394765 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG dbcounter [-] [81767] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81767) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 16:14:14.457817 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 1367/2693] 10.222.0.15 () {58 vars in 958 bytes} [Mon Aug 22 16:14:14 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 16:14:14.470136 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56958896-b45c-4add-95cc-f518d2197146 admin admin] Authenticating user token {{(pid=81767) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 16:14:14.472258 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.fernet_utils [None req-56958896-b45c-4add-95cc-f518d2197146 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81767) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 16:14:14.556458 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56958896-b45c-4add-95cc-f518d2197146 admin admin] Validating token access rules against request {{(pid=81767) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 16:14:14.561654 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93235a71-fdaa-4f36-a3c9-07bd5e889362 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79706d480f864998b9f8594be407ceab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4ee0413f9ef4d4fb1f95a660babe090', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81767) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 16:14:14.563046 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93235a71-fdaa-4f36-a3c9-07bd5e889362 service cinder] REQUEST_METHOD: `GET` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:14:14.563679 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93235a71-fdaa-4f36-a3c9-07bd5e889362 service cinder] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:14:14.564260 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93235a71-fdaa-4f36-a3c9-07bd5e889362 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:14:14.566481 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.fernet_utils [None req-93235a71-fdaa-4f36-a3c9-07bd5e889362 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81767) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 16:14:14.647447 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93235a71-fdaa-4f36-a3c9-07bd5e889362 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 16:14:14.649577 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93235a71-fdaa-4f36-a3c9-07bd5e889362 service cinder] RBAC: Authorization granted {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 16:14:14.674729 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 1327/2694] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Aug 22 16:14:14 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 202 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 16:14:14.675611 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f6f76ca-c1bd-4bbc-ae19-59508ce2fbe5 None None] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 16:14:14.679841 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-0f6f76ca-c1bd-4bbc-ae19-59508ce2fbe5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 16:14:14.756561 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 16:14:14.797552 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f6f76ca-c1bd-4bbc-ae19-59508ce2fbe5 None None] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 16:14:14.806936 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4174644d-adfd-4c2f-a7b6-49e1b0b5f897 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a98b1e4431940ab8d1fc95df2858a6c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 16:14:14.808003 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4174644d-adfd-4c2f-a7b6-49e1b0b5f897 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:14:14.808122 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4174644d-adfd-4c2f-a7b6-49e1b0b5f897 admin admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:14:14.808379 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4174644d-adfd-4c2f-a7b6-49e1b0b5f897 admin admin] PATH_INFO: `/v3/projects/a719d562b1d848e682148155af24d610` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:14:14.811130 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4174644d-adfd-4c2f-a7b6-49e1b0b5f897 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a719d562b1d848e682148155af24d610)` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 16:14:14.813808 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4174644d-adfd-4c2f-a7b6-49e1b0b5f897 admin admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 16:14:14.923482 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG dbcounter [-] [81767] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81767) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 16:14:14.950309 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-4174644d-adfd-4c2f-a7b6-49e1b0b5f897 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a719d562b1d848e682148155af24d610', 'request_id': 'req-4174644d-adfd-4c2f-a7b6-49e1b0b5f897'} {{(pid=81768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 16:14:14.956178 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-4174644d-adfd-4c2f-a7b6-49e1b0b5f897 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a719d562b1d848e682148155af24d610', 'request_id': 'req-4174644d-adfd-4c2f-a7b6-49e1b0b5f897'} {{(pid=81768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 16:14:15.036986 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-4174644d-adfd-4c2f-a7b6-49e1b0b5f897 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a719d562b1d848e682148155af24d610', 'request_id': 'req-4174644d-adfd-4c2f-a7b6-49e1b0b5f897'} {{(pid=81768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 16:14:15.059669 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-4174644d-adfd-4c2f-a7b6-49e1b0b5f897 admin admin] The token cache is being invalidate because project a719d562b1d848e682148155af24d610 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81768) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 22 16:14:15.059669 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-4174644d-adfd-4c2f-a7b6-49e1b0b5f897 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 16:14:15.059669 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-4174644d-adfd-4c2f-a7b6-49e1b0b5f897 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 16:14:15.086891 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 1368/2695] 10.222.0.15 () {58 vars in 1233 bytes} [Mon Aug 22 16:14:14 2022] DELETE /identity/v3/projects/a719d562b1d848e682148155af24d610 => generated 0 bytes in 412 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 16:14:15.673455 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG dbcounter [-] [81767] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81767) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 16:14:16.094012 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=100 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 16:14:16.785205 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ffbdcfd-3c32-4d92-b0fe-c56cdf052ecc None None] REQUEST_METHOD: `POST` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:14:16.785690 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ffbdcfd-3c32-4d92-b0fe-c56cdf052ecc None None] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:14:16.786038 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ffbdcfd-3c32-4d92-b0fe-c56cdf052ecc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:14:16.819574 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.auth.core [None req-3ffbdcfd-3c32-4d92-b0fe-c56cdf052ecc None None] MFA Rules not processed for user `0e9eda76738143a793abe3173c669a31`. Rule list: `[]` (Enabled: `True`). {{(pid=81767) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 16:14:16.823696 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.fernet_utils [None req-3ffbdcfd-3c32-4d92-b0fe-c56cdf052ecc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81767) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 16:14:16.923717 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 1328/2696] 10.222.0.15 () {58 vars in 958 bytes} [Mon Aug 22 16:14:16 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 16:14:16.941858 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4174644d-adfd-4c2f-a7b6-49e1b0b5f897 admin admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 16:14:16.943570 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-4174644d-adfd-4c2f-a7b6-49e1b0b5f897 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 16:14:17.055208 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4174644d-adfd-4c2f-a7b6-49e1b0b5f897 admin admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 16:14:17.060666 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d70b9ffa-d4cf-48ae-b987-be17b15f1594 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4ef569e41fb46d0b721bb1c7663616f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4ee0413f9ef4d4fb1f95a660babe090', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 16:14:17.061265 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d70b9ffa-d4cf-48ae-b987-be17b15f1594 service nova] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:14:17.061539 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d70b9ffa-d4cf-48ae-b987-be17b15f1594 service nova] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:14:17.061807 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d70b9ffa-d4cf-48ae-b987-be17b15f1594 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:14:17.064207 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-d70b9ffa-d4cf-48ae-b987-be17b15f1594 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 16:14:17.133905 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d70b9ffa-d4cf-48ae-b987-be17b15f1594 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 16:14:17.135562 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d70b9ffa-d4cf-48ae-b987-be17b15f1594 service nova] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 16:14:17.146700 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 1369/2697] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Aug 22 16:14:16 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 205 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 16:14:17.924828 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ffbdcfd-3c32-4d92-b0fe-c56cdf052ecc None None] Authenticating user token {{(pid=81767) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 16:14:17.926329 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.fernet_utils [None req-3ffbdcfd-3c32-4d92-b0fe-c56cdf052ecc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81767) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 16:14:17.979511 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG dbcounter [-] [81767] Writing DB stats keystone:SELECT=100 {{(pid=81767) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 16:14:17.991445 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ffbdcfd-3c32-4d92-b0fe-c56cdf052ecc None None] Validating token access rules against request {{(pid=81767) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 16:14:17.995812 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07699f7a-c0f1-4a53-936d-d2896ebb27ab service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79706d480f864998b9f8594be407ceab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4ee0413f9ef4d4fb1f95a660babe090', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81767) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 16:14:17.998383 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07699f7a-c0f1-4a53-936d-d2896ebb27ab service cinder] REQUEST_METHOD: `GET` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:14:17.998383 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07699f7a-c0f1-4a53-936d-d2896ebb27ab service cinder] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:14:17.998383 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07699f7a-c0f1-4a53-936d-d2896ebb27ab service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:14:17.999930 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.fernet_utils [None req-07699f7a-c0f1-4a53-936d-d2896ebb27ab service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81767) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 16:14:18.076612 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07699f7a-c0f1-4a53-936d-d2896ebb27ab service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 16:14:18.078786 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07699f7a-c0f1-4a53-936d-d2896ebb27ab service cinder] RBAC: Authorization granted {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 16:14:18.090270 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 1329/2698] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Aug 22 16:14:17 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 166 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 16:14:21.261423 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d70b9ffa-d4cf-48ae-b987-be17b15f1594 service nova] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 16:14:21.262924 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-d70b9ffa-d4cf-48ae-b987-be17b15f1594 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 16:14:21.275225 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d70b9ffa-d4cf-48ae-b987-be17b15f1594 service nova] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 16:14:21.279323 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f6da84c-4cbc-43b3-8b4a-ca49cf6f9b51 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79706d480f864998b9f8594be407ceab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4ee0413f9ef4d4fb1f95a660babe090', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 16:14:21.279959 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f6da84c-4cbc-43b3-8b4a-ca49cf6f9b51 service cinder] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:14:21.280246 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f6da84c-4cbc-43b3-8b4a-ca49cf6f9b51 service cinder] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:14:21.280594 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f6da84c-4cbc-43b3-8b4a-ca49cf6f9b51 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:14:21.282387 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-4f6da84c-4cbc-43b3-8b4a-ca49cf6f9b51 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 16:14:21.314579 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 16:14:21.354799 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f6da84c-4cbc-43b3-8b4a-ca49cf6f9b51 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 16:14:21.356323 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f6da84c-4cbc-43b3-8b4a-ca49cf6f9b51 service cinder] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 16:14:21.370115 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 1370/2699] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Aug 22 16:14:21 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 16:14:28.084671 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG dbcounter [-] [81767] Writing DB stats keystone:SELECT=29 {{(pid=81767) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 16:14:29.358238 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07699f7a-c0f1-4a53-936d-d2896ebb27ab service cinder] Authenticating user token {{(pid=81767) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 16:14:29.360320 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.fernet_utils [None req-07699f7a-c0f1-4a53-936d-d2896ebb27ab service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81767) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 16:14:29.431924 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07699f7a-c0f1-4a53-936d-d2896ebb27ab service cinder] Validating token access rules against request {{(pid=81767) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 16:14:29.436399 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-519d89a4-8082-4c16-ad31-5de1c4495efa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a98b1e4431940ab8d1fc95df2858a6c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81767) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 16:14:29.439468 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-519d89a4-8082-4c16-ad31-5de1c4495efa admin admin] REQUEST_METHOD: `DELETE` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:14:29.439468 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-519d89a4-8082-4c16-ad31-5de1c4495efa admin admin] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:14:29.439468 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-519d89a4-8082-4c16-ad31-5de1c4495efa admin admin] PATH_INFO: `/v3/users/3885352af4e049ca99dad86b3dad97e3` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:14:29.439468 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-519d89a4-8082-4c16-ad31-5de1c4495efa admin admin] RBAC: Authorizing `identity:delete_user(user_id=3885352af4e049ca99dad86b3dad97e3)` {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 16:14:29.440796 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-519d89a4-8082-4c16-ad31-5de1c4495efa admin admin] RBAC: Authorization granted {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 16:14:29.674381 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.notifications [None req-519d89a4-8082-4c16-ad31-5de1c4495efa admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3885352af4e049ca99dad86b3dad97e3'} {{(pid=81767) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 16:14:29.675167 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.notifications [None req-519d89a4-8082-4c16-ad31-5de1c4495efa admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3885352af4e049ca99dad86b3dad97e3'} {{(pid=81767) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 16:14:29.795844 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.notifications [None req-519d89a4-8082-4c16-ad31-5de1c4495efa admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3885352af4e049ca99dad86b3dad97e3'} {{(pid=81767) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 16:14:29.806679 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.notifications [None req-519d89a4-8082-4c16-ad31-5de1c4495efa admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3885352af4e049ca99dad86b3dad97e3'} {{(pid=81767) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 16:14:29.824405 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.notifications [None req-519d89a4-8082-4c16-ad31-5de1c4495efa admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3885352af4e049ca99dad86b3dad97e3'} {{(pid=81767) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 16:14:29.837112 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 1330/2700] 10.222.0.15 () {58 vars in 1224 bytes} [Mon Aug 22 16:14:29 2022] DELETE /identity/v3/users/3885352af4e049ca99dad86b3dad97e3 => generated 0 bytes in 476 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 16:14:29.838871 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f47ea5-87d4-4a44-9deb-10b1cde77a1b None None] REQUEST_METHOD: `POST` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:14:29.839352 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f47ea5-87d4-4a44-9deb-10b1cde77a1b None None] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:14:29.839662 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f47ea5-87d4-4a44-9deb-10b1cde77a1b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:14:29.860206 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.auth.core [None req-00f47ea5-87d4-4a44-9deb-10b1cde77a1b None None] MFA Rules not processed for user `6b2199ae0fb0480aa917ada391f5ec54`. Rule list: `[]` (Enabled: `True`). {{(pid=81768) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 16:14:29.861475 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-00f47ea5-87d4-4a44-9deb-10b1cde77a1b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 16:14:29.932236 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 1371/2701] 10.222.0.15 () {58 vars in 958 bytes} [Mon Aug 22 16:14:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 16:14:29.942477 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-519d89a4-8082-4c16-ad31-5de1c4495efa admin admin] Authenticating user token {{(pid=81767) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 16:14:29.943992 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.fernet_utils [None req-519d89a4-8082-4c16-ad31-5de1c4495efa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81767) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 16:14:30.019520 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-519d89a4-8082-4c16-ad31-5de1c4495efa admin admin] Validating token access rules against request {{(pid=81767) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 16:14:30.023785 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3e38a47-6624-4b22-a4d0-27ff1ddddfee service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '234687adad594f1aa45775a565ffbdc7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4ee0413f9ef4d4fb1f95a660babe090', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81767) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 16:14:30.024485 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3e38a47-6624-4b22-a4d0-27ff1ddddfee service neutron] REQUEST_METHOD: `GET` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:14:30.024836 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3e38a47-6624-4b22-a4d0-27ff1ddddfee service neutron] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:14:30.025098 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3e38a47-6624-4b22-a4d0-27ff1ddddfee service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:14:30.036027 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3e38a47-6624-4b22-a4d0-27ff1ddddfee service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 16:14:30.037516 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3e38a47-6624-4b22-a4d0-27ff1ddddfee service neutron] RBAC: Authorization granted {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 16:14:30.048269 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 1331/2702] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Aug 22 16:14:29 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 16:14:30.489709 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00f47ea5-87d4-4a44-9deb-10b1cde77a1b None None] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 16:14:30.491131 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-00f47ea5-87d4-4a44-9deb-10b1cde77a1b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 16:14:30.506183 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00f47ea5-87d4-4a44-9deb-10b1cde77a1b None None] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 16:14:30.512003 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f027c6d-886f-4c2c-bcfa-360412062997 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a98b1e4431940ab8d1fc95df2858a6c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 16:14:30.512919 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f027c6d-886f-4c2c-bcfa-360412062997 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:14:30.513325 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f027c6d-886f-4c2c-bcfa-360412062997 admin admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:14:30.513605 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f027c6d-886f-4c2c-bcfa-360412062997 admin admin] PATH_INFO: `/v3/projects/fe13b32187b74ac995fe3f664e7acc77` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:14:30.514904 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f027c6d-886f-4c2c-bcfa-360412062997 admin admin] RBAC: Authorizing `identity:delete_project(project_id=fe13b32187b74ac995fe3f664e7acc77)` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 16:14:30.517135 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f027c6d-886f-4c2c-bcfa-360412062997 admin admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 16:14:30.669522 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-3f027c6d-886f-4c2c-bcfa-360412062997 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fe13b32187b74ac995fe3f664e7acc77', 'request_id': 'req-3f027c6d-886f-4c2c-bcfa-360412062997'} {{(pid=81768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 16:14:30.675944 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-3f027c6d-886f-4c2c-bcfa-360412062997 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fe13b32187b74ac995fe3f664e7acc77', 'request_id': 'req-3f027c6d-886f-4c2c-bcfa-360412062997'} {{(pid=81768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 16:14:30.764328 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-3f027c6d-886f-4c2c-bcfa-360412062997 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fe13b32187b74ac995fe3f664e7acc77', 'request_id': 'req-3f027c6d-886f-4c2c-bcfa-360412062997'} {{(pid=81768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 16:14:30.780106 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-3f027c6d-886f-4c2c-bcfa-360412062997 admin admin] The token cache is being invalidate because project fe13b32187b74ac995fe3f664e7acc77 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81768) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 22 16:14:30.780106 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-3f027c6d-886f-4c2c-bcfa-360412062997 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 16:14:30.780106 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-3f027c6d-886f-4c2c-bcfa-360412062997 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 16:14:30.781001 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 1372/2703] 10.222.0.15 () {58 vars in 1233 bytes} [Mon Aug 22 16:14:30 2022] DELETE /identity/v3/projects/fe13b32187b74ac995fe3f664e7acc77 => generated 0 bytes in 292 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 16:14:30.785382 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3e38a47-6624-4b22-a4d0-27ff1ddddfee service neutron] Authenticating user token {{(pid=81767) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 16:14:30.795805 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3e38a47-6624-4b22-a4d0-27ff1ddddfee service neutron] Validating token access rules against request {{(pid=81767) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 16:14:30.831886 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG dbcounter [-] [81767] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81767) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 16:14:30.875940 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f4c6cbe-2a9f-4cbf-a4c4-5a01ed8ecb5a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a98b1e4431940ab8d1fc95df2858a6c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81767) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 16:14:30.876565 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f4c6cbe-2a9f-4cbf-a4c4-5a01ed8ecb5a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:14:30.876838 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f4c6cbe-2a9f-4cbf-a4c4-5a01ed8ecb5a admin admin] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:14:30.877104 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f4c6cbe-2a9f-4cbf-a4c4-5a01ed8ecb5a admin admin] PATH_INFO: `/v3/users/7870c436e21a4434b62bfef1a001dce8` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:14:30.878255 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f4c6cbe-2a9f-4cbf-a4c4-5a01ed8ecb5a admin admin] RBAC: Authorizing `identity:delete_user(user_id=7870c436e21a4434b62bfef1a001dce8)` {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 16:14:30.880188 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f4c6cbe-2a9f-4cbf-a4c4-5a01ed8ecb5a admin admin] RBAC: Authorization granted {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 16:14:31.057469 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.notifications [None req-0f4c6cbe-2a9f-4cbf-a4c4-5a01ed8ecb5a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7870c436e21a4434b62bfef1a001dce8'} {{(pid=81767) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 16:14:31.058268 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.notifications [None req-0f4c6cbe-2a9f-4cbf-a4c4-5a01ed8ecb5a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7870c436e21a4434b62bfef1a001dce8'} {{(pid=81767) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 16:14:31.124876 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.notifications [None req-0f4c6cbe-2a9f-4cbf-a4c4-5a01ed8ecb5a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7870c436e21a4434b62bfef1a001dce8'} {{(pid=81767) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 16:14:31.133605 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.notifications [None req-0f4c6cbe-2a9f-4cbf-a4c4-5a01ed8ecb5a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7870c436e21a4434b62bfef1a001dce8'} {{(pid=81767) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 16:14:31.148527 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.notifications [None req-0f4c6cbe-2a9f-4cbf-a4c4-5a01ed8ecb5a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7870c436e21a4434b62bfef1a001dce8'} {{(pid=81767) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 16:14:31.158474 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 1332/2704] 10.222.0.15 () {58 vars in 1224 bytes} [Mon Aug 22 16:14:30 2022] DELETE /identity/v3/users/7870c436e21a4434b62bfef1a001dce8 => generated 0 bytes in 373 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 16:14:31.632184 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f027c6d-886f-4c2c-bcfa-360412062997 admin admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 16:14:31.632735 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-3f027c6d-886f-4c2c-bcfa-360412062997 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 16:14:31.662791 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f027c6d-886f-4c2c-bcfa-360412062997 admin admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 16:14:31.711051 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 16:14:31.727004 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da080c7a-2345-470a-9c68-fa1f6257b9a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a98b1e4431940ab8d1fc95df2858a6c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 16:14:31.727709 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da080c7a-2345-470a-9c68-fa1f6257b9a4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:14:31.727979 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da080c7a-2345-470a-9c68-fa1f6257b9a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:14:31.728278 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da080c7a-2345-470a-9c68-fa1f6257b9a4 admin admin] PATH_INFO: `/v3/projects/4ad284e04f904cb7a1fd0181f96ad71c` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:14:31.729460 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da080c7a-2345-470a-9c68-fa1f6257b9a4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4ad284e04f904cb7a1fd0181f96ad71c)` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 16:14:31.731522 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da080c7a-2345-470a-9c68-fa1f6257b9a4 admin admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 16:14:31.861754 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-da080c7a-2345-470a-9c68-fa1f6257b9a4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4ad284e04f904cb7a1fd0181f96ad71c', 'request_id': 'req-da080c7a-2345-470a-9c68-fa1f6257b9a4'} {{(pid=81768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 16:14:31.868629 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-da080c7a-2345-470a-9c68-fa1f6257b9a4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4ad284e04f904cb7a1fd0181f96ad71c', 'request_id': 'req-da080c7a-2345-470a-9c68-fa1f6257b9a4'} {{(pid=81768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 16:14:31.996601 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-da080c7a-2345-470a-9c68-fa1f6257b9a4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4ad284e04f904cb7a1fd0181f96ad71c', 'request_id': 'req-da080c7a-2345-470a-9c68-fa1f6257b9a4'} {{(pid=81768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 16:14:32.011549 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-da080c7a-2345-470a-9c68-fa1f6257b9a4 admin admin] The token cache is being invalidate because project 4ad284e04f904cb7a1fd0181f96ad71c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81768) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 22 16:14:32.011549 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-da080c7a-2345-470a-9c68-fa1f6257b9a4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 16:14:32.012676 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-da080c7a-2345-470a-9c68-fa1f6257b9a4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 16:14:32.013826 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 1373/2705] 10.222.0.15 () {58 vars in 1233 bytes} [Mon Aug 22 16:14:31 2022] DELETE /identity/v3/projects/4ad284e04f904cb7a1fd0181f96ad71c => generated 0 bytes in 384 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 16:14:32.019373 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f4c6cbe-2a9f-4cbf-a4c4-5a01ed8ecb5a admin admin] Authenticating user token {{(pid=81767) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 16:14:32.020837 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.fernet_utils [None req-0f4c6cbe-2a9f-4cbf-a4c4-5a01ed8ecb5a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81767) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 16:14:32.034000 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f4c6cbe-2a9f-4cbf-a4c4-5a01ed8ecb5a admin admin] Validating token access rules against request {{(pid=81767) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 16:14:32.106233 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74aaeea6-27e0-4a3d-8ec4-6a2861eda67d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a98b1e4431940ab8d1fc95df2858a6c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81767) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 16:14:32.106983 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74aaeea6-27e0-4a3d-8ec4-6a2861eda67d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:14:32.107493 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74aaeea6-27e0-4a3d-8ec4-6a2861eda67d admin admin] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:14:32.107788 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74aaeea6-27e0-4a3d-8ec4-6a2861eda67d admin admin] PATH_INFO: `/v3/users/f7e4945c5bb549ba8b26af82e8d8ce61` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:14:32.109119 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74aaeea6-27e0-4a3d-8ec4-6a2861eda67d admin admin] RBAC: Authorizing `identity:delete_user(user_id=f7e4945c5bb549ba8b26af82e8d8ce61)` {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 16:14:32.111238 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74aaeea6-27e0-4a3d-8ec4-6a2861eda67d admin admin] RBAC: Authorization granted {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 16:14:32.568819 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG dbcounter [-] [81767] Writing DB stats keystone:SELECT=83,keystone:DELETE=16,keystone:INSERT=1 {{(pid=81767) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 16:14:32.571428 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.notifications [None req-74aaeea6-27e0-4a3d-8ec4-6a2861eda67d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f7e4945c5bb549ba8b26af82e8d8ce61'} {{(pid=81767) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 16:14:32.572086 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.notifications [None req-74aaeea6-27e0-4a3d-8ec4-6a2861eda67d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f7e4945c5bb549ba8b26af82e8d8ce61'} {{(pid=81767) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 16:14:32.647934 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.notifications [None req-74aaeea6-27e0-4a3d-8ec4-6a2861eda67d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f7e4945c5bb549ba8b26af82e8d8ce61'} {{(pid=81767) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 16:14:32.657434 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.notifications [None req-74aaeea6-27e0-4a3d-8ec4-6a2861eda67d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f7e4945c5bb549ba8b26af82e8d8ce61'} {{(pid=81767) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 16:14:32.671225 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.notifications [None req-74aaeea6-27e0-4a3d-8ec4-6a2861eda67d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f7e4945c5bb549ba8b26af82e8d8ce61'} {{(pid=81767) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 16:14:32.677857 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 1333/2706] 10.222.0.15 () {58 vars in 1224 bytes} [Mon Aug 22 16:14:32 2022] DELETE /identity/v3/users/f7e4945c5bb549ba8b26af82e8d8ce61 => generated 0 bytes in 660 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 16:14:33.209000 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da080c7a-2345-470a-9c68-fa1f6257b9a4 admin admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 16:14:33.217462 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-da080c7a-2345-470a-9c68-fa1f6257b9a4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 16:14:33.224402 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da080c7a-2345-470a-9c68-fa1f6257b9a4 admin admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 16:14:33.294491 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa29f451-3b1d-4163-992f-b1252659cd7b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a98b1e4431940ab8d1fc95df2858a6c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 16:14:33.295470 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa29f451-3b1d-4163-992f-b1252659cd7b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:14:33.295856 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa29f451-3b1d-4163-992f-b1252659cd7b admin admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:14:33.296245 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa29f451-3b1d-4163-992f-b1252659cd7b admin admin] PATH_INFO: `/v3/projects/74a18ea26b044f788f061c2a7a2ceebe` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:14:33.297544 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa29f451-3b1d-4163-992f-b1252659cd7b admin admin] RBAC: Authorizing `identity:delete_project(project_id=74a18ea26b044f788f061c2a7a2ceebe)` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 16:14:33.300134 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa29f451-3b1d-4163-992f-b1252659cd7b admin admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 16:14:33.423849 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-aa29f451-3b1d-4163-992f-b1252659cd7b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '74a18ea26b044f788f061c2a7a2ceebe', 'request_id': 'req-aa29f451-3b1d-4163-992f-b1252659cd7b'} {{(pid=81768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 16:14:33.429919 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-aa29f451-3b1d-4163-992f-b1252659cd7b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '74a18ea26b044f788f061c2a7a2ceebe', 'request_id': 'req-aa29f451-3b1d-4163-992f-b1252659cd7b'} {{(pid=81768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 16:14:33.501818 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-aa29f451-3b1d-4163-992f-b1252659cd7b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '74a18ea26b044f788f061c2a7a2ceebe', 'request_id': 'req-aa29f451-3b1d-4163-992f-b1252659cd7b'} {{(pid=81768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 16:14:33.514781 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-aa29f451-3b1d-4163-992f-b1252659cd7b admin admin] The token cache is being invalidate because project 74a18ea26b044f788f061c2a7a2ceebe was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81768) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 22 16:14:33.515080 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-aa29f451-3b1d-4163-992f-b1252659cd7b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 16:14:33.515769 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-aa29f451-3b1d-4163-992f-b1252659cd7b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 16:14:33.517311 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 1374/2707] 10.222.0.15 () {58 vars in 1233 bytes} [Mon Aug 22 16:14:33 2022] DELETE /identity/v3/projects/74a18ea26b044f788f061c2a7a2ceebe => generated 0 bytes in 309 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 16:14:33.522540 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a83695f-95a6-41e8-bdf0-e7b943329dd6 None None] REQUEST_METHOD: `POST` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:14:33.523104 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a83695f-95a6-41e8-bdf0-e7b943329dd6 None None] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:14:33.523104 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a83695f-95a6-41e8-bdf0-e7b943329dd6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:14:33.543435 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.auth.core [None req-0a83695f-95a6-41e8-bdf0-e7b943329dd6 None None] MFA Rules not processed for user `6b2199ae0fb0480aa917ada391f5ec54`. Rule list: `[]` (Enabled: `True`). {{(pid=81767) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 16:14:33.544760 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.fernet_utils [None req-0a83695f-95a6-41e8-bdf0-e7b943329dd6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81767) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 16:14:33.608351 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 1334/2708] 10.222.0.15 () {58 vars in 958 bytes} [Mon Aug 22 16:14:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 16:14:33.617887 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abd040d7-6fbc-4815-a7cf-69db2d895072 None None] REQUEST_METHOD: `POST` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:14:33.618166 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abd040d7-6fbc-4815-a7cf-69db2d895072 None None] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:14:33.618442 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abd040d7-6fbc-4815-a7cf-69db2d895072 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:14:33.627104 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 16:14:33.638984 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.auth.core [None req-abd040d7-6fbc-4815-a7cf-69db2d895072 None None] MFA Rules not processed for user `6b2199ae0fb0480aa917ada391f5ec54`. Rule list: `[]` (Enabled: `True`). {{(pid=81768) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 16:14:33.640342 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-abd040d7-6fbc-4815-a7cf-69db2d895072 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 16:14:33.648559 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 1375/2709] 10.222.0.15 () {58 vars in 958 bytes} [Mon Aug 22 16:14:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 16:14:33.653202 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a83695f-95a6-41e8-bdf0-e7b943329dd6 None None] Authenticating user token {{(pid=81767) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 16:14:33.664249 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a83695f-95a6-41e8-bdf0-e7b943329dd6 None None] Validating token access rules against request {{(pid=81767) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 16:14:33.669776 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-427c3cf1-7438-408d-8c1d-2a5b988f9361 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a98b1e4431940ab8d1fc95df2858a6c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81767) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 16:14:33.674911 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-427c3cf1-7438-408d-8c1d-2a5b988f9361 admin admin] REQUEST_METHOD: `GET` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:14:33.674911 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-427c3cf1-7438-408d-8c1d-2a5b988f9361 admin admin] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:14:33.674911 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-427c3cf1-7438-408d-8c1d-2a5b988f9361 admin admin] PATH_INFO: `/v3/domains` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:14:33.674911 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-427c3cf1-7438-408d-8c1d-2a5b988f9361 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81767) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 16:14:33.674911 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-427c3cf1-7438-408d-8c1d-2a5b988f9361 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 16:14:33.674911 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-427c3cf1-7438-408d-8c1d-2a5b988f9361 admin admin] RBAC: Authorization granted {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 16:14:33.687945 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 1335/2710] 10.222.0.15 () {58 vars in 1153 bytes} [Mon Aug 22 16:14:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 16:14:33.692410 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abd040d7-6fbc-4815-a7cf-69db2d895072 None None] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 16:14:33.709941 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abd040d7-6fbc-4815-a7cf-69db2d895072 None None] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 16:14:33.718125 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e0b78ee-755e-44f2-8fae-58892353fc9a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a98b1e4431940ab8d1fc95df2858a6c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 16:14:33.719075 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e0b78ee-755e-44f2-8fae-58892353fc9a admin admin] REQUEST_METHOD: `POST` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:14:33.719604 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e0b78ee-755e-44f2-8fae-58892353fc9a admin admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:14:33.720031 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e0b78ee-755e-44f2-8fae-58892353fc9a admin admin] PATH_INFO: `/v3/projects` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:14:33.721617 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e0b78ee-755e-44f2-8fae-58892353fc9a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 16:14:33.725951 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e0b78ee-755e-44f2-8fae-58892353fc9a admin admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 16:14:33.838868 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 1376/2711] 10.222.0.15 () {60 vars in 1153 bytes} [Mon Aug 22 16:14:33 2022] POST /identity/v3/projects => generated 371 bytes in 147 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 22 16:14:33.843263 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-427c3cf1-7438-408d-8c1d-2a5b988f9361 admin admin] Authenticating user token {{(pid=81767) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 16:14:33.855062 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-427c3cf1-7438-408d-8c1d-2a5b988f9361 admin admin] Validating token access rules against request {{(pid=81767) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 16:14:33.921978 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-687b8e2d-af9d-46a8-9c9d-820ae436e188 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a98b1e4431940ab8d1fc95df2858a6c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81767) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 16:14:33.922587 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-687b8e2d-af9d-46a8-9c9d-820ae436e188 admin admin] REQUEST_METHOD: `POST` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:14:33.922867 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-687b8e2d-af9d-46a8-9c9d-820ae436e188 admin admin] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:14:33.923139 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-687b8e2d-af9d-46a8-9c9d-820ae436e188 admin admin] PATH_INFO: `/v3/users` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:14:33.923803 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-687b8e2d-af9d-46a8-9c9d-820ae436e188 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 16:14:33.925721 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-687b8e2d-af9d-46a8-9c9d-820ae436e188 admin admin] RBAC: Authorization granted {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 16:14:34.024482 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 1336/2712] 10.222.0.15 () {60 vars in 1144 bytes} [Mon Aug 22 16:14:33 2022] POST /identity/v3/users => generated 286 bytes in 182 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 22 16:14:34.030946 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e0b78ee-755e-44f2-8fae-58892353fc9a admin admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 16:14:34.042033 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e0b78ee-755e-44f2-8fae-58892353fc9a admin admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 16:14:34.046712 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70916c5e-9dcd-4fd9-a359-7e8f275ba599 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a98b1e4431940ab8d1fc95df2858a6c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 16:14:34.047409 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70916c5e-9dcd-4fd9-a359-7e8f275ba599 admin admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:14:34.047692 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70916c5e-9dcd-4fd9-a359-7e8f275ba599 admin admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:14:34.048080 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70916c5e-9dcd-4fd9-a359-7e8f275ba599 admin admin] PATH_INFO: `/v3/roles` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:14:34.048727 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70916c5e-9dcd-4fd9-a359-7e8f275ba599 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 16:14:34.050971 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70916c5e-9dcd-4fd9-a359-7e8f275ba599 admin admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 16:14:34.062031 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 1377/2713] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Aug 22 16:14:34 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 22 16:14:34.077708 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-687b8e2d-af9d-46a8-9c9d-820ae436e188 admin admin] Authenticating user token {{(pid=81767) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 16:14:34.089510 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-687b8e2d-af9d-46a8-9c9d-820ae436e188 admin admin] Validating token access rules against request {{(pid=81767) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 16:14:34.094448 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78965d36-3035-494a-95c0-cc67b00a99d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a98b1e4431940ab8d1fc95df2858a6c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81767) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 16:14:34.095120 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78965d36-3035-494a-95c0-cc67b00a99d2 admin admin] REQUEST_METHOD: `GET` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:14:34.095489 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78965d36-3035-494a-95c0-cc67b00a99d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:14:34.095826 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78965d36-3035-494a-95c0-cc67b00a99d2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:14:34.096545 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78965d36-3035-494a-95c0-cc67b00a99d2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 16:14:34.098222 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78965d36-3035-494a-95c0-cc67b00a99d2 admin admin] RBAC: Authorization granted {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 16:14:34.119312 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 1337/2714] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Aug 22 16:14:34 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 22 16:14:34.123504 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70916c5e-9dcd-4fd9-a359-7e8f275ba599 admin admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 16:14:34.139055 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70916c5e-9dcd-4fd9-a359-7e8f275ba599 admin admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 16:14:34.144359 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a874989-18bc-4234-8954-cbcb50919ccc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a98b1e4431940ab8d1fc95df2858a6c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 16:14:34.145033 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a874989-18bc-4234-8954-cbcb50919ccc admin admin] REQUEST_METHOD: `PUT` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:14:34.145315 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a874989-18bc-4234-8954-cbcb50919ccc admin admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:14:34.145668 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a874989-18bc-4234-8954-cbcb50919ccc admin admin] PATH_INFO: `/v3/projects/92e2834fc9a448a28ee12bfbbfc4fee4/users/521c78587730446297375484f3e9d206/roles/e6fc1ee3fb974125ba9b4fe55b12b45e` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:14:34.166001 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a874989-18bc-4234-8954-cbcb50919ccc admin admin] RBAC: Authorizing `identity:create_grant(project_id=92e2834fc9a448a28ee12bfbbfc4fee4, user_id=521c78587730446297375484f3e9d206, role_id=e6fc1ee3fb974125ba9b4fe55b12b45e)` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 16:14:34.168764 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a874989-18bc-4234-8954-cbcb50919ccc admin admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 16:14:34.279685 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 1378/2715] 10.222.0.15 () {60 vars in 1483 bytes} [Mon Aug 22 16:14:34 2022] PUT /identity/v3/projects/92e2834fc9a448a28ee12bfbbfc4fee4/users/521c78587730446297375484f3e9d206/roles/e6fc1ee3fb974125ba9b4fe55b12b45e => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 16:14:34.285834 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cb19d8b-9b6a-4814-9dbc-ca01f232d330 None None] REQUEST_METHOD: `POST` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:14:34.286118 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cb19d8b-9b6a-4814-9dbc-ca01f232d330 None None] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:14:34.286390 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cb19d8b-9b6a-4814-9dbc-ca01f232d330 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:14:34.308129 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.auth.core [None req-8cb19d8b-9b6a-4814-9dbc-ca01f232d330 None None] MFA Rules not processed for user `6b2199ae0fb0480aa917ada391f5ec54`. Rule list: `[]` (Enabled: `True`). {{(pid=81767) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 16:14:34.309474 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.fernet_utils [None req-8cb19d8b-9b6a-4814-9dbc-ca01f232d330 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81767) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 16:14:34.333332 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG dbcounter [-] [81767] Writing DB stats keystone:DELETE=4,keystone:SELECT=92,keystone:INSERT=4 {{(pid=81767) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 16:14:34.380480 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 1338/2716] 10.222.0.15 () {58 vars in 958 bytes} [Mon Aug 22 16:14:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 22 16:14:34.390110 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a874989-18bc-4234-8954-cbcb50919ccc admin admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 16:14:34.393255 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-1a874989-18bc-4234-8954-cbcb50919ccc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 16:14:34.457659 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a874989-18bc-4234-8954-cbcb50919ccc admin admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 16:14:34.461692 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fef0a78-14e9-4a95-ab18-5c7d8a89749f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '234687adad594f1aa45775a565ffbdc7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4ee0413f9ef4d4fb1f95a660babe090', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 16:14:34.462304 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fef0a78-14e9-4a95-ab18-5c7d8a89749f service neutron] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:14:34.462575 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fef0a78-14e9-4a95-ab18-5c7d8a89749f service neutron] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:14:34.462866 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fef0a78-14e9-4a95-ab18-5c7d8a89749f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:14:34.473306 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fef0a78-14e9-4a95-ab18-5c7d8a89749f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 16:14:34.474746 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fef0a78-14e9-4a95-ab18-5c7d8a89749f service neutron] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 16:14:34.484701 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 1379/2717] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Aug 22 16:14:34 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 16:14:35.694940 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-817d5577-babc-48e8-9ae9-2d57eec87da1 None None] REQUEST_METHOD: `POST` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:14:35.695598 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-817d5577-babc-48e8-9ae9-2d57eec87da1 None None] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:14:35.695598 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-817d5577-babc-48e8-9ae9-2d57eec87da1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:14:35.724525 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.auth.core [None req-817d5577-babc-48e8-9ae9-2d57eec87da1 None None] MFA Rules not processed for user `521c78587730446297375484f3e9d206`. Rule list: `[]` (Enabled: `True`). {{(pid=81767) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 16:14:35.725835 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.fernet_utils [None req-817d5577-babc-48e8-9ae9-2d57eec87da1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81767) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 16:14:35.798051 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 1339/2718] 10.222.0.15 () {58 vars in 958 bytes} [Mon Aug 22 16:14:35 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 16:14:35.809701 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fef0a78-14e9-4a95-ab18-5c7d8a89749f service neutron] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 16:14:35.811525 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-0fef0a78-14e9-4a95-ab18-5c7d8a89749f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 16:14:35.871732 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fef0a78-14e9-4a95-ab18-5c7d8a89749f service neutron] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 16:14:35.875237 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f65090ec-76c0-4df4-ab51-6d7a5b39c430 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79706d480f864998b9f8594be407ceab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4ee0413f9ef4d4fb1f95a660babe090', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 16:14:35.875860 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f65090ec-76c0-4df4-ab51-6d7a5b39c430 service cinder] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:14:35.876132 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f65090ec-76c0-4df4-ab51-6d7a5b39c430 service cinder] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:14:35.876398 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f65090ec-76c0-4df4-ab51-6d7a5b39c430 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:14:35.884684 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f65090ec-76c0-4df4-ab51-6d7a5b39c430 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 16:14:35.888334 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f65090ec-76c0-4df4-ab51-6d7a5b39c430 service cinder] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 16:14:35.894665 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 1380/2719] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Aug 22 16:14:35 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 16:14:37.738020 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96aacd0b-d56b-422b-98d9-e339096e55c2 None None] REQUEST_METHOD: `POST` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:14:37.738591 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96aacd0b-d56b-422b-98d9-e339096e55c2 None None] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:14:37.738591 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96aacd0b-d56b-422b-98d9-e339096e55c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:14:37.758802 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.auth.core [None req-96aacd0b-d56b-422b-98d9-e339096e55c2 None None] MFA Rules not processed for user `521c78587730446297375484f3e9d206`. Rule list: `[]` (Enabled: `True`). {{(pid=81767) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 16:14:37.759967 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.fernet_utils [None req-96aacd0b-d56b-422b-98d9-e339096e55c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81767) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 16:14:37.767204 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 1340/2720] 10.222.0.15 () {58 vars in 958 bytes} [Mon Aug 22 16:14:37 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 16:14:38.485045 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f65090ec-76c0-4df4-ab51-6d7a5b39c430 service cinder] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 16:14:38.487094 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-f65090ec-76c0-4df4-ab51-6d7a5b39c430 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 16:14:38.504397 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f65090ec-76c0-4df4-ab51-6d7a5b39c430 service cinder] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 16:14:38.509075 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66ffd8a1-526e-4f96-be3f-b7631b32eeac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a98b1e4431940ab8d1fc95df2858a6c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 16:14:38.509709 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66ffd8a1-526e-4f96-be3f-b7631b32eeac admin admin] REQUEST_METHOD: `DELETE` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:14:38.509986 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66ffd8a1-526e-4f96-be3f-b7631b32eeac admin admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:14:38.510260 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66ffd8a1-526e-4f96-be3f-b7631b32eeac admin admin] PATH_INFO: `/v3/users/7be15e5dbc3b4419b4ada8e7f075d6a9` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:14:38.511488 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66ffd8a1-526e-4f96-be3f-b7631b32eeac admin admin] RBAC: Authorizing `identity:delete_user(user_id=7be15e5dbc3b4419b4ada8e7f075d6a9)` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 16:14:38.513420 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66ffd8a1-526e-4f96-be3f-b7631b32eeac admin admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 16:14:38.698647 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=92,keystone:INSERT=2,keystone:DELETE=6 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 16:14:38.706022 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-66ffd8a1-526e-4f96-be3f-b7631b32eeac admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7be15e5dbc3b4419b4ada8e7f075d6a9'} {{(pid=81768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 16:14:38.706889 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-66ffd8a1-526e-4f96-be3f-b7631b32eeac admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7be15e5dbc3b4419b4ada8e7f075d6a9'} {{(pid=81768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 16:14:38.707632 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-66ffd8a1-526e-4f96-be3f-b7631b32eeac admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7be15e5dbc3b4419b4ada8e7f075d6a9'} {{(pid=81768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 16:14:38.721566 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-66ffd8a1-526e-4f96-be3f-b7631b32eeac admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7be15e5dbc3b4419b4ada8e7f075d6a9'} {{(pid=81768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 16:14:38.818844 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-66ffd8a1-526e-4f96-be3f-b7631b32eeac admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7be15e5dbc3b4419b4ada8e7f075d6a9'} {{(pid=81768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 16:14:38.837527 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 1381/2721] 10.222.0.15 () {58 vars in 1224 bytes} [Mon Aug 22 16:14:38 2022] DELETE /identity/v3/users/7be15e5dbc3b4419b4ada8e7f075d6a9 => generated 0 bytes in 353 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 16:14:39.114480 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96aacd0b-d56b-422b-98d9-e339096e55c2 None None] Authenticating user token {{(pid=81767) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 16:14:39.131782 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96aacd0b-d56b-422b-98d9-e339096e55c2 None None] Validating token access rules against request {{(pid=81767) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 16:14:39.193623 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b36dcabd-d80c-41ad-b803-b477f3e1b82e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a98b1e4431940ab8d1fc95df2858a6c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81767) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 16:14:39.194278 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b36dcabd-d80c-41ad-b803-b477f3e1b82e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:14:39.194554 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b36dcabd-d80c-41ad-b803-b477f3e1b82e admin admin] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:14:39.194822 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b36dcabd-d80c-41ad-b803-b477f3e1b82e admin admin] PATH_INFO: `/v3/projects/dfe1b430e4694baa861828803211f324` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:14:39.196013 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b36dcabd-d80c-41ad-b803-b477f3e1b82e admin admin] RBAC: Authorizing `identity:delete_project(project_id=dfe1b430e4694baa861828803211f324)` {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 16:14:39.197915 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b36dcabd-d80c-41ad-b803-b477f3e1b82e admin admin] RBAC: Authorization granted {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 16:14:39.315996 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.notifications [None req-b36dcabd-d80c-41ad-b803-b477f3e1b82e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'dfe1b430e4694baa861828803211f324', 'request_id': 'req-b36dcabd-d80c-41ad-b803-b477f3e1b82e'} {{(pid=81767) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 16:14:39.318013 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG dbcounter [-] [81767] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=81767) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 16:14:39.321847 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.notifications [None req-b36dcabd-d80c-41ad-b803-b477f3e1b82e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'dfe1b430e4694baa861828803211f324', 'request_id': 'req-b36dcabd-d80c-41ad-b803-b477f3e1b82e'} {{(pid=81767) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 16:14:39.393274 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.notifications [None req-b36dcabd-d80c-41ad-b803-b477f3e1b82e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'dfe1b430e4694baa861828803211f324', 'request_id': 'req-b36dcabd-d80c-41ad-b803-b477f3e1b82e'} {{(pid=81767) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 16:14:39.410208 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.notifications [None req-b36dcabd-d80c-41ad-b803-b477f3e1b82e admin admin] The token cache is being invalidate because project dfe1b430e4694baa861828803211f324 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81767) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 22 16:14:39.410607 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.notifications [None req-b36dcabd-d80c-41ad-b803-b477f3e1b82e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81767) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 16:14:39.411638 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.notifications [None req-b36dcabd-d80c-41ad-b803-b477f3e1b82e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81767) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 16:14:39.413317 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 1341/2722] 10.222.0.15 () {58 vars in 1233 bytes} [Mon Aug 22 16:14:39 2022] DELETE /identity/v3/projects/dfe1b430e4694baa861828803211f324 => generated 0 bytes in 299 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 16:14:39.418551 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f4d74ce-09c2-43ba-bb84-00c60dfc12a6 None None] REQUEST_METHOD: `POST` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:14:39.418836 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f4d74ce-09c2-43ba-bb84-00c60dfc12a6 None None] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:14:39.423299 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f4d74ce-09c2-43ba-bb84-00c60dfc12a6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:14:39.445921 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.auth.core [None req-1f4d74ce-09c2-43ba-bb84-00c60dfc12a6 None None] MFA Rules not processed for user `6b2199ae0fb0480aa917ada391f5ec54`. Rule list: `[]` (Enabled: `True`). {{(pid=81768) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 16:14:39.447239 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-1f4d74ce-09c2-43ba-bb84-00c60dfc12a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 16:14:39.514125 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 1382/2723] 10.222.0.15 () {58 vars in 958 bytes} [Mon Aug 22 16:14:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 16:14:39.522858 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab221448-ce12-452d-9ecf-2d9cf06df0d1 None None] REQUEST_METHOD: `POST` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:14:39.523164 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab221448-ce12-452d-9ecf-2d9cf06df0d1 None None] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:14:39.523482 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab221448-ce12-452d-9ecf-2d9cf06df0d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:14:39.543503 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.auth.core [None req-ab221448-ce12-452d-9ecf-2d9cf06df0d1 None None] MFA Rules not processed for user `6b2199ae0fb0480aa917ada391f5ec54`. Rule list: `[]` (Enabled: `True`). {{(pid=81767) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 16:14:39.544727 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.fernet_utils [None req-ab221448-ce12-452d-9ecf-2d9cf06df0d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81767) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 16:14:39.553279 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 1342/2724] 10.222.0.15 () {58 vars in 958 bytes} [Mon Aug 22 16:14:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 16:14:39.557554 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f4d74ce-09c2-43ba-bb84-00c60dfc12a6 None None] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 16:14:39.568004 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f4d74ce-09c2-43ba-bb84-00c60dfc12a6 None None] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 16:14:39.572637 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcb46505-ec05-479b-99e8-a9e3b1bf13ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a98b1e4431940ab8d1fc95df2858a6c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 16:14:39.573247 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcb46505-ec05-479b-99e8-a9e3b1bf13ab admin admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:14:39.573522 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcb46505-ec05-479b-99e8-a9e3b1bf13ab admin admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:14:39.573796 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcb46505-ec05-479b-99e8-a9e3b1bf13ab admin admin] PATH_INFO: `/v3/domains` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:14:39.574446 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcb46505-ec05-479b-99e8-a9e3b1bf13ab admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81768) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 16:14:39.574769 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcb46505-ec05-479b-99e8-a9e3b1bf13ab admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 16:14:39.576389 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcb46505-ec05-479b-99e8-a9e3b1bf13ab admin admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 16:14:39.592620 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 1383/2725] 10.222.0.15 () {58 vars in 1153 bytes} [Mon Aug 22 16:14:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 16:14:39.594115 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab221448-ce12-452d-9ecf-2d9cf06df0d1 None None] Authenticating user token {{(pid=81767) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 16:14:39.600949 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10d3194f-a848-4ed5-bda5-e857b343b63d None None] REQUEST_METHOD: `POST` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:14:39.601351 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10d3194f-a848-4ed5-bda5-e857b343b63d None None] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:14:39.601693 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10d3194f-a848-4ed5-bda5-e857b343b63d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:14:39.604388 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab221448-ce12-452d-9ecf-2d9cf06df0d1 None None] Validating token access rules against request {{(pid=81767) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 16:14:39.608308 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application [None req-10d3194f-a848-4ed5-bda5-e857b343b63d None None] Expecting to find domain in project. The server could not comply with the request since it is either malformed or otherwise incorrect. The client is assumed to be in error.: keystone.exception.ValidationError: Expecting to find domain in project. The server could not comply with the request since it is either malformed or otherwise incorrect. The client is assumed to be in error. Aug 22 16:14:39.608308 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 16:14:39.608308 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 16:14:39.608308 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 16:14:39.608308 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 16:14:39.608308 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 16:14:39.608308 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 16:14:39.608308 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 16:14:39.608308 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 16:14:39.608308 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 16:14:39.608308 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 16:14:39.608308 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 16:14:39.608308 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/server/flask/common.py", line 1064, in wrapper Aug 22 16:14:39.609390 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return f(*args, **kwargs) Aug 22 16:14:39.609390 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/auth.py", line 315, in post Aug 22 16:14:39.609390 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application token = authentication.authenticate_for_token(auth_data) Aug 22 16:14:39.609390 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/_shared/authentication.py", line 182, in authenticate_for_token Aug 22 16:14:39.609390 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application auth_info = core.AuthInfo.create(auth=auth) Aug 22 16:14:39.609390 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/auth/core.py", line 126, in create Aug 22 16:14:39.609390 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application auth_info._validate_and_normalize_auth_data(scope_only) Aug 22 16:14:39.609390 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/auth/core.py", line 340, in _validate_and_normalize_auth_data Aug 22 16:14:39.609390 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application self._validate_and_normalize_scope_data() Aug 22 16:14:39.609390 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/auth/core.py", line 290, in _validate_and_normalize_scope_data Aug 22 16:14:39.609390 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application project_ref = self._lookup_project(self.auth['scope']['project']) Aug 22 16:14:39.609390 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/auth/core.py", line 194, in _lookup_project Aug 22 16:14:39.609390 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application raise exception.ValidationError(attribute='domain', Aug 22 16:14:39.609390 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application keystone.exception.ValidationError: Expecting to find domain in project. The server could not comply with the request since it is either malformed or otherwise incorrect. The client is assumed to be in error. Aug 22 16:14:39.609390 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application Aug 22 16:14:39.610505 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-300392b0-058a-438f-9896-74deeb8b8b1d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a98b1e4431940ab8d1fc95df2858a6c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81767) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 16:14:39.610505 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-300392b0-058a-438f-9896-74deeb8b8b1d admin admin] REQUEST_METHOD: `POST` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:14:39.610505 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-300392b0-058a-438f-9896-74deeb8b8b1d admin admin] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:14:39.610505 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-300392b0-058a-438f-9896-74deeb8b8b1d admin admin] PATH_INFO: `/v3/projects` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:14:39.611012 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 1384/2726] 10.222.0.15 () {58 vars in 1009 bytes} [Mon Aug 22 16:14:39 2022] POST /identity/v3/auth/tokens => generated 230 bytes in 11 msecs (HTTP/1.1 400) 5 headers in 186 bytes (2 switches on core 0) Aug 22 16:14:39.611166 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-300392b0-058a-438f-9896-74deeb8b8b1d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 16:14:39.613242 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-300392b0-058a-438f-9896-74deeb8b8b1d admin admin] RBAC: Authorization granted {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 16:14:39.693101 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb9f8a55-4822-466f-8f4c-a2a5ce473b65 None None] REQUEST_METHOD: `POST` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:14:39.693381 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb9f8a55-4822-466f-8f4c-a2a5ce473b65 None None] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:14:39.693652 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb9f8a55-4822-466f-8f4c-a2a5ce473b65 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:14:39.715064 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.auth.core [None req-fb9f8a55-4822-466f-8f4c-a2a5ce473b65 None None] MFA Rules not processed for user `521c78587730446297375484f3e9d206`. Rule list: `[]` (Enabled: `True`). {{(pid=81768) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 16:14:39.716308 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-fb9f8a55-4822-466f-8f4c-a2a5ce473b65 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 16:14:39.731673 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 1343/2727] 10.222.0.15 () {60 vars in 1153 bytes} [Mon Aug 22 16:14:39 2022] POST /identity/v3/projects => generated 373 bytes in 138 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 22 16:14:39.735918 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-300392b0-058a-438f-9896-74deeb8b8b1d admin admin] Authenticating user token {{(pid=81767) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 16:14:39.777848 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 1385/2728] 10.222.0.15 () {58 vars in 958 bytes} [Mon Aug 22 16:14:39 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 22 16:14:39.817633 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-300392b0-058a-438f-9896-74deeb8b8b1d admin admin] Validating token access rules against request {{(pid=81767) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 16:14:39.822329 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47f74fa1-6f6b-46c9-9d70-4331287f662f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a98b1e4431940ab8d1fc95df2858a6c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81767) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 16:14:39.822929 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47f74fa1-6f6b-46c9-9d70-4331287f662f admin admin] REQUEST_METHOD: `POST` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:14:39.823206 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47f74fa1-6f6b-46c9-9d70-4331287f662f admin admin] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:14:39.823515 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47f74fa1-6f6b-46c9-9d70-4331287f662f admin admin] PATH_INFO: `/v3/users` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:14:39.824131 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47f74fa1-6f6b-46c9-9d70-4331287f662f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 16:14:39.826021 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47f74fa1-6f6b-46c9-9d70-4331287f662f admin admin] RBAC: Authorization granted {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 16:14:39.938674 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 1344/2729] 10.222.0.15 () {60 vars in 1144 bytes} [Mon Aug 22 16:14:39 2022] POST /identity/v3/users => generated 287 bytes in 199 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 22 16:14:39.940940 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb9f8a55-4822-466f-8f4c-a2a5ce473b65 None None] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 16:14:39.953887 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb9f8a55-4822-466f-8f4c-a2a5ce473b65 None None] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 16:14:39.961369 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0963f706-6794-43eb-9c0c-074639aeacd8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a98b1e4431940ab8d1fc95df2858a6c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 16:14:39.961369 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0963f706-6794-43eb-9c0c-074639aeacd8 admin admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:14:39.961369 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0963f706-6794-43eb-9c0c-074639aeacd8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:14:39.961686 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0963f706-6794-43eb-9c0c-074639aeacd8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:14:39.962727 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0963f706-6794-43eb-9c0c-074639aeacd8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 16:14:39.969291 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0963f706-6794-43eb-9c0c-074639aeacd8 admin admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 16:14:39.983316 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 1386/2730] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Aug 22 16:14:39 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 22 16:14:39.985828 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47f74fa1-6f6b-46c9-9d70-4331287f662f admin admin] Authenticating user token {{(pid=81767) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 16:14:39.996380 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47f74fa1-6f6b-46c9-9d70-4331287f662f admin admin] Validating token access rules against request {{(pid=81767) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 16:14:40.000998 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e50c2f6-2f9a-4c0d-90ac-e81a9e12cb80 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a98b1e4431940ab8d1fc95df2858a6c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81767) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 16:14:40.001710 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e50c2f6-2f9a-4c0d-90ac-e81a9e12cb80 admin admin] REQUEST_METHOD: `GET` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:14:40.002045 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e50c2f6-2f9a-4c0d-90ac-e81a9e12cb80 admin admin] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:14:40.003063 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e50c2f6-2f9a-4c0d-90ac-e81a9e12cb80 admin admin] PATH_INFO: `/v3/roles` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:14:40.003907 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e50c2f6-2f9a-4c0d-90ac-e81a9e12cb80 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 16:14:40.006322 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e50c2f6-2f9a-4c0d-90ac-e81a9e12cb80 admin admin] RBAC: Authorization granted {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 16:14:40.018434 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 1345/2731] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Aug 22 16:14:39 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 22 16:14:40.022675 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0963f706-6794-43eb-9c0c-074639aeacd8 admin admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 16:14:40.026215 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c649d45f-fd30-4c5e-ad1b-2f01a670c60a None None] REQUEST_METHOD: `POST` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:14:40.026567 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c649d45f-fd30-4c5e-ad1b-2f01a670c60a None None] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:14:40.026933 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c649d45f-fd30-4c5e-ad1b-2f01a670c60a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:14:40.032999 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0963f706-6794-43eb-9c0c-074639aeacd8 admin admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 16:14:40.033390 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application [None req-c649d45f-fd30-4c5e-ad1b-2f01a670c60a None None] Expecting to find domain in project. The server could not comply with the request since it is either malformed or otherwise incorrect. The client is assumed to be in error.: keystone.exception.ValidationError: Expecting to find domain in project. The server could not comply with the request since it is either malformed or otherwise incorrect. The client is assumed to be in error. Aug 22 16:14:40.033390 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 16:14:40.033390 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 16:14:40.033390 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 16:14:40.033390 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 16:14:40.033390 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 16:14:40.033390 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 16:14:40.033390 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 16:14:40.033390 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 16:14:40.033390 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 16:14:40.033390 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 16:14:40.033390 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 16:14:40.033390 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/server/flask/common.py", line 1064, in wrapper Aug 22 16:14:40.034373 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application return f(*args, **kwargs) Aug 22 16:14:40.034373 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/auth.py", line 315, in post Aug 22 16:14:40.034373 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application token = authentication.authenticate_for_token(auth_data) Aug 22 16:14:40.034373 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/_shared/authentication.py", line 182, in authenticate_for_token Aug 22 16:14:40.034373 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application auth_info = core.AuthInfo.create(auth=auth) Aug 22 16:14:40.034373 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/auth/core.py", line 126, in create Aug 22 16:14:40.034373 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application auth_info._validate_and_normalize_auth_data(scope_only) Aug 22 16:14:40.034373 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/auth/core.py", line 340, in _validate_and_normalize_auth_data Aug 22 16:14:40.034373 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application self._validate_and_normalize_scope_data() Aug 22 16:14:40.034373 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/auth/core.py", line 290, in _validate_and_normalize_scope_data Aug 22 16:14:40.034373 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application project_ref = self._lookup_project(self.auth['scope']['project']) Aug 22 16:14:40.034373 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/auth/core.py", line 194, in _lookup_project Aug 22 16:14:40.034373 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application raise exception.ValidationError(attribute='domain', Aug 22 16:14:40.034373 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application keystone.exception.ValidationError: Expecting to find domain in project. The server could not comply with the request since it is either malformed or otherwise incorrect. The client is assumed to be in error. Aug 22 16:14:40.034373 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: ERROR keystone.server.flask.application Aug 22 16:14:40.035470 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 1346/2732] 10.222.0.15 () {58 vars in 1009 bytes} [Mon Aug 22 16:14:40 2022] POST /identity/v3/auth/tokens => generated 230 bytes in 9 msecs (HTTP/1.1 400) 5 headers in 186 bytes (2 switches on core 0) Aug 22 16:14:40.041313 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9747dba8-6354-4286-a4b2-14158dd45ee2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a98b1e4431940ab8d1fc95df2858a6c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 16:14:40.041313 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9747dba8-6354-4286-a4b2-14158dd45ee2 admin admin] REQUEST_METHOD: `PUT` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:14:40.041313 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9747dba8-6354-4286-a4b2-14158dd45ee2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:14:40.041313 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9747dba8-6354-4286-a4b2-14158dd45ee2 admin admin] PATH_INFO: `/v3/projects/21e5a94ced984deea6e3c0a0f08a994b/users/7990d5771e4d4915aa3329d733451d36/roles/e6fc1ee3fb974125ba9b4fe55b12b45e` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:14:40.065779 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9747dba8-6354-4286-a4b2-14158dd45ee2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=21e5a94ced984deea6e3c0a0f08a994b, user_id=7990d5771e4d4915aa3329d733451d36, role_id=e6fc1ee3fb974125ba9b4fe55b12b45e)` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 16:14:40.068644 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9747dba8-6354-4286-a4b2-14158dd45ee2 admin admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 16:14:40.182147 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 1387/2733] 10.222.0.15 () {60 vars in 1483 bytes} [Mon Aug 22 16:14:40 2022] PUT /identity/v3/projects/21e5a94ced984deea6e3c0a0f08a994b/users/7990d5771e4d4915aa3329d733451d36/roles/e6fc1ee3fb974125ba9b4fe55b12b45e => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 22 16:14:40.188239 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bbc27aa-1756-42ee-be1c-8a6de6c52948 None None] REQUEST_METHOD: `POST` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:14:40.188816 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bbc27aa-1756-42ee-be1c-8a6de6c52948 None None] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:14:40.188816 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bbc27aa-1756-42ee-be1c-8a6de6c52948 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:14:40.208695 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.auth.core [None req-2bbc27aa-1756-42ee-be1c-8a6de6c52948 None None] MFA Rules not processed for user `6b2199ae0fb0480aa917ada391f5ec54`. Rule list: `[]` (Enabled: `True`). {{(pid=81767) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 16:14:40.209893 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.fernet_utils [None req-2bbc27aa-1756-42ee-be1c-8a6de6c52948 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81767) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 16:14:40.276301 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 1347/2734] 10.222.0.15 () {58 vars in 958 bytes} [Mon Aug 22 16:14:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 22 16:14:40.286727 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9747dba8-6354-4286-a4b2-14158dd45ee2 admin admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 16:14:40.288490 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-9747dba8-6354-4286-a4b2-14158dd45ee2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 16:14:40.308112 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=94,keystone:DELETE=4,keystone:INSERT=2 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 16:14:40.349819 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9747dba8-6354-4286-a4b2-14158dd45ee2 admin admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 16:14:40.353499 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d00225b-149c-4b6b-b101-2b1f043c79b1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '234687adad594f1aa45775a565ffbdc7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4ee0413f9ef4d4fb1f95a660babe090', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 16:14:40.354113 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d00225b-149c-4b6b-b101-2b1f043c79b1 service neutron] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:14:40.354389 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d00225b-149c-4b6b-b101-2b1f043c79b1 service neutron] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:14:40.354683 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d00225b-149c-4b6b-b101-2b1f043c79b1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:14:40.365898 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d00225b-149c-4b6b-b101-2b1f043c79b1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 16:14:40.367403 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d00225b-149c-4b6b-b101-2b1f043c79b1 service neutron] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 16:14:40.377981 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 1388/2735] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Aug 22 16:14:40 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 16:14:45.903427 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-860ef55a-0171-459a-9104-e9274319c2c7 None None] REQUEST_METHOD: `POST` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:14:45.903427 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-860ef55a-0171-459a-9104-e9274319c2c7 None None] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:14:45.904073 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-860ef55a-0171-459a-9104-e9274319c2c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:14:45.938057 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.auth.core [None req-860ef55a-0171-459a-9104-e9274319c2c7 None None] MFA Rules not processed for user `7990d5771e4d4915aa3329d733451d36`. Rule list: `[]` (Enabled: `True`). {{(pid=81767) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 16:14:45.939381 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.fernet_utils [None req-860ef55a-0171-459a-9104-e9274319c2c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81767) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 16:14:45.965155 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG dbcounter [-] [81767] Writing DB stats keystone:SELECT=93,keystone:DELETE=2,keystone:INSERT=5 {{(pid=81767) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 16:14:46.029000 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 1348/2736] 10.222.0.15 () {58 vars in 958 bytes} [Mon Aug 22 16:14:45 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 16:14:46.034063 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d00225b-149c-4b6b-b101-2b1f043c79b1 service neutron] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 16:14:46.048959 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d00225b-149c-4b6b-b101-2b1f043c79b1 service neutron] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 16:14:46.057221 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-190fe17d-46f9-4c32-9608-c48087ee98f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a98b1e4431940ab8d1fc95df2858a6c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 16:14:46.057852 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-190fe17d-46f9-4c32-9608-c48087ee98f3 admin admin] REQUEST_METHOD: `POST` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:14:46.058134 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-190fe17d-46f9-4c32-9608-c48087ee98f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:14:46.058407 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-190fe17d-46f9-4c32-9608-c48087ee98f3 admin admin] PATH_INFO: `/v3/projects` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:14:46.059052 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-190fe17d-46f9-4c32-9608-c48087ee98f3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 16:14:46.061069 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-190fe17d-46f9-4c32-9608-c48087ee98f3 admin admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 16:14:46.159319 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 1389/2737] 10.222.0.15 () {60 vars in 1153 bytes} [Mon Aug 22 16:14:46 2022] POST /identity/v3/projects => generated 369 bytes in 125 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 22 16:14:46.162212 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-860ef55a-0171-459a-9104-e9274319c2c7 None None] Authenticating user token {{(pid=81767) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 16:14:46.175702 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-860ef55a-0171-459a-9104-e9274319c2c7 None None] Validating token access rules against request {{(pid=81767) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 16:14:46.271972 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed613d7d-29dd-47ff-bcc4-9bd5b351ac49 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a98b1e4431940ab8d1fc95df2858a6c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81767) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 16:14:46.272584 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed613d7d-29dd-47ff-bcc4-9bd5b351ac49 admin admin] REQUEST_METHOD: `POST` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:14:46.272864 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed613d7d-29dd-47ff-bcc4-9bd5b351ac49 admin admin] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:14:46.273134 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed613d7d-29dd-47ff-bcc4-9bd5b351ac49 admin admin] PATH_INFO: `/v3/users` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:14:46.273767 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed613d7d-29dd-47ff-bcc4-9bd5b351ac49 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 16:14:46.275730 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed613d7d-29dd-47ff-bcc4-9bd5b351ac49 admin admin] RBAC: Authorization granted {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 16:14:46.376759 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 1349/2738] 10.222.0.15 () {60 vars in 1144 bytes} [Mon Aug 22 16:14:46 2022] POST /identity/v3/users => generated 291 bytes in 215 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 22 16:14:46.381274 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-190fe17d-46f9-4c32-9608-c48087ee98f3 admin admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 16:14:46.392283 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-190fe17d-46f9-4c32-9608-c48087ee98f3 admin admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 16:14:46.403771 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcde9ee9-9fe1-4ca2-99ef-dd2c46891150 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a98b1e4431940ab8d1fc95df2858a6c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 16:14:46.404417 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcde9ee9-9fe1-4ca2-99ef-dd2c46891150 admin admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:14:46.404688 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcde9ee9-9fe1-4ca2-99ef-dd2c46891150 admin admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:14:46.404965 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcde9ee9-9fe1-4ca2-99ef-dd2c46891150 admin admin] PATH_INFO: `/v3/roles` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:14:46.405633 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcde9ee9-9fe1-4ca2-99ef-dd2c46891150 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 16:14:46.407266 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcde9ee9-9fe1-4ca2-99ef-dd2c46891150 admin admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 16:14:46.426253 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 1390/2739] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Aug 22 16:14:46 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 22 16:14:46.429373 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed613d7d-29dd-47ff-bcc4-9bd5b351ac49 admin admin] Authenticating user token {{(pid=81767) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 16:14:46.440941 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed613d7d-29dd-47ff-bcc4-9bd5b351ac49 admin admin] Validating token access rules against request {{(pid=81767) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 16:14:46.446785 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1dc4681-89dc-4fd7-a025-b83c57e8ab74 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a98b1e4431940ab8d1fc95df2858a6c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81767) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 16:14:46.447563 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1dc4681-89dc-4fd7-a025-b83c57e8ab74 admin admin] REQUEST_METHOD: `PUT` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:14:46.447851 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1dc4681-89dc-4fd7-a025-b83c57e8ab74 admin admin] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:14:46.448127 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1dc4681-89dc-4fd7-a025-b83c57e8ab74 admin admin] PATH_INFO: `/v3/projects/3f19a65d43ad4b1aac97ce02095dbe21/users/543789fa6fe44ebf92b4768de626cc38/roles/9d9848cfd22e4b089db443c350e56670` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:14:46.467124 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1dc4681-89dc-4fd7-a025-b83c57e8ab74 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3f19a65d43ad4b1aac97ce02095dbe21, user_id=543789fa6fe44ebf92b4768de626cc38, role_id=9d9848cfd22e4b089db443c350e56670)` {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 16:14:46.469952 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1dc4681-89dc-4fd7-a025-b83c57e8ab74 admin admin] RBAC: Authorization granted {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 16:14:46.539043 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 1350/2740] 10.222.0.15 () {60 vars in 1483 bytes} [Mon Aug 22 16:14:46 2022] PUT /identity/v3/projects/3f19a65d43ad4b1aac97ce02095dbe21/users/543789fa6fe44ebf92b4768de626cc38/roles/9d9848cfd22e4b089db443c350e56670 => generated 0 bytes in 110 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 16:14:46.542994 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcde9ee9-9fe1-4ca2-99ef-dd2c46891150 admin admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 16:14:46.553182 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcde9ee9-9fe1-4ca2-99ef-dd2c46891150 admin admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 16:14:46.611043 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-083d1100-7491-4754-a689-8c82f0008571 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a98b1e4431940ab8d1fc95df2858a6c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 16:14:46.612220 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-083d1100-7491-4754-a689-8c82f0008571 admin admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:14:46.612501 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-083d1100-7491-4754-a689-8c82f0008571 admin admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:14:46.612795 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-083d1100-7491-4754-a689-8c82f0008571 admin admin] PATH_INFO: `/v3/roles` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:14:46.613440 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-083d1100-7491-4754-a689-8c82f0008571 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 16:14:46.615078 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-083d1100-7491-4754-a689-8c82f0008571 admin admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 16:14:46.625065 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 1391/2741] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Aug 22 16:14:46 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 22 16:14:46.630537 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1dc4681-89dc-4fd7-a025-b83c57e8ab74 admin admin] Authenticating user token {{(pid=81767) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 16:14:46.641005 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1dc4681-89dc-4fd7-a025-b83c57e8ab74 admin admin] Validating token access rules against request {{(pid=81767) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 16:14:46.646630 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-738a7162-ac24-4a25-af57-0aad07e8d216 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a98b1e4431940ab8d1fc95df2858a6c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81767) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 16:14:46.647192 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-738a7162-ac24-4a25-af57-0aad07e8d216 admin admin] REQUEST_METHOD: `PUT` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:14:46.647492 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-738a7162-ac24-4a25-af57-0aad07e8d216 admin admin] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:14:46.647753 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-738a7162-ac24-4a25-af57-0aad07e8d216 admin admin] PATH_INFO: `/v3/projects/3f19a65d43ad4b1aac97ce02095dbe21/users/543789fa6fe44ebf92b4768de626cc38/roles/9d9848cfd22e4b089db443c350e56670` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:14:46.650645 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-738a7162-ac24-4a25-af57-0aad07e8d216 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3f19a65d43ad4b1aac97ce02095dbe21, user_id=543789fa6fe44ebf92b4768de626cc38, role_id=9d9848cfd22e4b089db443c350e56670)` {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 16:14:46.653890 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-738a7162-ac24-4a25-af57-0aad07e8d216 admin admin] RBAC: Authorization granted {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 16:14:46.666908 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 1351/2742] 10.222.0.15 () {60 vars in 1483 bytes} [Mon Aug 22 16:14:46 2022] PUT /identity/v3/projects/3f19a65d43ad4b1aac97ce02095dbe21/users/543789fa6fe44ebf92b4768de626cc38/roles/9d9848cfd22e4b089db443c350e56670 => generated 0 bytes in 37 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 22 16:14:49.935330 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be766af7-2b9f-456c-abd1-9bc37b66a1cd None None] REQUEST_METHOD: `POST` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:14:49.935906 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be766af7-2b9f-456c-abd1-9bc37b66a1cd None None] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:14:49.935906 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be766af7-2b9f-456c-abd1-9bc37b66a1cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:14:49.959650 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.auth.core [None req-be766af7-2b9f-456c-abd1-9bc37b66a1cd None None] MFA Rules not processed for user `058e76354dce412cb78b2875c88f2791`. Rule list: `[]` (Enabled: `True`). {{(pid=81768) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 16:14:49.961319 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-be766af7-2b9f-456c-abd1-9bc37b66a1cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 16:14:50.046459 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 1392/2743] 10.222.0.15 () {58 vars in 1009 bytes} [Mon Aug 22 16:14:49 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 16:14:50.216403 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-738a7162-ac24-4a25-af57-0aad07e8d216 admin admin] Authenticating user token {{(pid=81767) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 16:14:50.218677 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.fernet_utils [None req-738a7162-ac24-4a25-af57-0aad07e8d216 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81767) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 16:14:50.300901 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-738a7162-ac24-4a25-af57-0aad07e8d216 admin admin] Validating token access rules against request {{(pid=81767) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 16:14:50.306303 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-868d0dd4-78fa-49be-8d0f-819d88ca5b50 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a09d128dee9438283633e51f4834643', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4ee0413f9ef4d4fb1f95a660babe090', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81767) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 16:14:50.306912 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-868d0dd4-78fa-49be-8d0f-819d88ca5b50 service swift] REQUEST_METHOD: `GET` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:14:50.307188 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-868d0dd4-78fa-49be-8d0f-819d88ca5b50 service swift] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:14:50.307484 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-868d0dd4-78fa-49be-8d0f-819d88ca5b50 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:14:50.326978 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-868d0dd4-78fa-49be-8d0f-819d88ca5b50 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 16:14:50.328688 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-868d0dd4-78fa-49be-8d0f-819d88ca5b50 service swift] RBAC: Authorization granted {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 16:14:50.345136 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 1352/2744] 10.222.0.15 () {60 vars in 1384 bytes} [Mon Aug 22 16:14:50 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 129 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 22 16:14:53.185956 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b2fa373-b0f7-48aa-bd85-1ca155b8f3cc None None] REQUEST_METHOD: `POST` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:14:53.186558 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b2fa373-b0f7-48aa-bd85-1ca155b8f3cc None None] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:14:53.186681 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b2fa373-b0f7-48aa-bd85-1ca155b8f3cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:14:53.236410 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.auth.core [None req-4b2fa373-b0f7-48aa-bd85-1ca155b8f3cc None None] MFA Rules not processed for user `543789fa6fe44ebf92b4768de626cc38`. Rule list: `[]` (Enabled: `True`). {{(pid=81768) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 16:14:53.237465 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-4b2fa373-b0f7-48aa-bd85-1ca155b8f3cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 16:14:53.265986 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 16:14:53.376446 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 1393/2745] 10.222.0.15 () {58 vars in 958 bytes} [Mon Aug 22 16:14:53 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 16:14:53.386454 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-868d0dd4-78fa-49be-8d0f-819d88ca5b50 service swift] Authenticating user token {{(pid=81767) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 16:14:53.388009 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.fernet_utils [None req-868d0dd4-78fa-49be-8d0f-819d88ca5b50 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81767) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 16:14:53.395443 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG dbcounter [-] [81767] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81767) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 16:14:53.484055 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-868d0dd4-78fa-49be-8d0f-819d88ca5b50 service swift] Validating token access rules against request {{(pid=81767) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 16:14:53.490107 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-115ca820-49f7-461c-b811-37ee6fd3d651 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4ef569e41fb46d0b721bb1c7663616f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4ee0413f9ef4d4fb1f95a660babe090', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81767) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 16:14:53.490950 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-115ca820-49f7-461c-b811-37ee6fd3d651 service nova] REQUEST_METHOD: `GET` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:14:53.491350 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-115ca820-49f7-461c-b811-37ee6fd3d651 service nova] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:14:53.491816 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-115ca820-49f7-461c-b811-37ee6fd3d651 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:14:53.598345 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-115ca820-49f7-461c-b811-37ee6fd3d651 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 16:14:53.600040 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-115ca820-49f7-461c-b811-37ee6fd3d651 service nova] RBAC: Authorization granted {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 16:14:53.610318 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 1353/2746] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Aug 22 16:14:53 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 224 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 16:14:57.323133 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b52666e-0d53-419c-be1a-f2c464c33343 None None] REQUEST_METHOD: `POST` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:14:57.324209 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b52666e-0d53-419c-be1a-f2c464c33343 None None] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:14:57.324637 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b52666e-0d53-419c-be1a-f2c464c33343 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:14:57.350099 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.auth.core [None req-8b52666e-0d53-419c-be1a-f2c464c33343 None None] MFA Rules not processed for user `7990d5771e4d4915aa3329d733451d36`. Rule list: `[]` (Enabled: `True`). {{(pid=81768) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 16:14:57.352039 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-8b52666e-0d53-419c-be1a-f2c464c33343 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 16:14:57.361970 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 1394/2747] 10.222.0.15 () {58 vars in 958 bytes} [Mon Aug 22 16:14:57 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 16:15:03.639562 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG dbcounter [-] [81767] Writing DB stats keystone:SELECT=55 {{(pid=81767) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 16:15:07.344149 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=25 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 16:15:11.771226 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b42d5c11-d2de-4265-bdd3-688dd5cea3b1 None None] REQUEST_METHOD: `POST` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:15:11.772065 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b42d5c11-d2de-4265-bdd3-688dd5cea3b1 None None] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:15:11.772065 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b42d5c11-d2de-4265-bdd3-688dd5cea3b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:15:11.793910 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.auth.core [None req-b42d5c11-d2de-4265-bdd3-688dd5cea3b1 None None] MFA Rules not processed for user `4e4eff0c95194682ade3e24aec451cd7`. Rule list: `[]` (Enabled: `True`). {{(pid=81767) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 16:15:11.795309 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.fernet_utils [None req-b42d5c11-d2de-4265-bdd3-688dd5cea3b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81767) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 16:15:11.860168 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 1354/2748] 10.222.0.15 () {58 vars in 958 bytes} [Mon Aug 22 16:15:11 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 16:15:12.228293 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-294869e4-cadc-4123-aec1-8246e52634e5 None None] REQUEST_METHOD: `POST` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:15:12.228950 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-294869e4-cadc-4123-aec1-8246e52634e5 None None] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:15:12.228950 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-294869e4-cadc-4123-aec1-8246e52634e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:15:12.236372 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application [None req-294869e4-cadc-4123-aec1-8246e52634e5 None None] Expecting to find domain in project. The server could not comply with the request since it is either malformed or otherwise incorrect. The client is assumed to be in error.: keystone.exception.ValidationError: Expecting to find domain in project. The server could not comply with the request since it is either malformed or otherwise incorrect. The client is assumed to be in error. Aug 22 16:15:12.236372 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 16:15:12.236372 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 16:15:12.236372 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 16:15:12.236372 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 16:15:12.236372 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 16:15:12.236372 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 16:15:12.236372 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 16:15:12.236372 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 16:15:12.236372 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 16:15:12.236372 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 16:15:12.236372 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 16:15:12.236372 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/server/flask/common.py", line 1064, in wrapper Aug 22 16:15:12.237360 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return f(*args, **kwargs) Aug 22 16:15:12.237360 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/auth.py", line 315, in post Aug 22 16:15:12.237360 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application token = authentication.authenticate_for_token(auth_data) Aug 22 16:15:12.237360 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/_shared/authentication.py", line 182, in authenticate_for_token Aug 22 16:15:12.237360 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application auth_info = core.AuthInfo.create(auth=auth) Aug 22 16:15:12.237360 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/auth/core.py", line 126, in create Aug 22 16:15:12.237360 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application auth_info._validate_and_normalize_auth_data(scope_only) Aug 22 16:15:12.237360 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/auth/core.py", line 340, in _validate_and_normalize_auth_data Aug 22 16:15:12.237360 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application self._validate_and_normalize_scope_data() Aug 22 16:15:12.237360 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/auth/core.py", line 290, in _validate_and_normalize_scope_data Aug 22 16:15:12.237360 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application project_ref = self._lookup_project(self.auth['scope']['project']) Aug 22 16:15:12.237360 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/auth/core.py", line 194, in _lookup_project Aug 22 16:15:12.237360 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application raise exception.ValidationError(attribute='domain', Aug 22 16:15:12.237360 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application keystone.exception.ValidationError: Expecting to find domain in project. The server could not comply with the request since it is either malformed or otherwise incorrect. The client is assumed to be in error. Aug 22 16:15:12.237360 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application Aug 22 16:15:12.241811 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 1395/2749] 10.222.0.15 () {58 vars in 1009 bytes} [Mon Aug 22 16:15:12 2022] POST /identity/v3/auth/tokens => generated 230 bytes in 14 msecs (HTTP/1.1 400) 5 headers in 186 bytes (1 switches on core 0) Aug 22 16:15:21.852486 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG dbcounter [-] [81767] Writing DB stats keystone:SELECT=25 {{(pid=81767) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 16:15:22.144356 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b42d5c11-d2de-4265-bdd3-688dd5cea3b1 None None] Authenticating user token {{(pid=81767) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 16:15:22.147468 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.fernet_utils [None req-b42d5c11-d2de-4265-bdd3-688dd5cea3b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81767) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 16:15:22.237727 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b42d5c11-d2de-4265-bdd3-688dd5cea3b1 None None] Validating token access rules against request {{(pid=81767) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 16:15:22.242684 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8cd29ba-c338-444f-bc7b-b33fc1c785a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a98b1e4431940ab8d1fc95df2858a6c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81767) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 16:15:22.243267 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8cd29ba-c338-444f-bc7b-b33fc1c785a2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:15:22.243573 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8cd29ba-c338-444f-bc7b-b33fc1c785a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:15:22.243845 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8cd29ba-c338-444f-bc7b-b33fc1c785a2 admin admin] PATH_INFO: `/v3/users/4e4eff0c95194682ade3e24aec451cd7` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:15:22.245012 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8cd29ba-c338-444f-bc7b-b33fc1c785a2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4e4eff0c95194682ade3e24aec451cd7)` {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 16:15:22.246926 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8cd29ba-c338-444f-bc7b-b33fc1c785a2 admin admin] RBAC: Authorization granted {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 16:15:22.385034 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22d40db3-dbc9-44a8-aad9-e073e012ae6f None None] REQUEST_METHOD: `POST` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:15:22.385778 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22d40db3-dbc9-44a8-aad9-e073e012ae6f None None] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:15:22.385778 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22d40db3-dbc9-44a8-aad9-e073e012ae6f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:15:22.405871 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.auth.core [None req-22d40db3-dbc9-44a8-aad9-e073e012ae6f None None] MFA Rules not processed for user `058e76354dce412cb78b2875c88f2791`. Rule list: `[]` (Enabled: `True`). {{(pid=81768) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 16:15:22.408823 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-22d40db3-dbc9-44a8-aad9-e073e012ae6f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 16:15:22.415024 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 1396/2750] 10.222.0.15 () {58 vars in 967 bytes} [Mon Aug 22 16:15:22 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 22 16:15:22.428978 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22d40db3-dbc9-44a8-aad9-e073e012ae6f None None] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 16:15:22.442030 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22d40db3-dbc9-44a8-aad9-e073e012ae6f None None] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 16:15:22.446462 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.notifications [None req-b8cd29ba-c338-444f-bc7b-b33fc1c785a2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4e4eff0c95194682ade3e24aec451cd7'} {{(pid=81767) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 16:15:22.447119 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.notifications [None req-b8cd29ba-c338-444f-bc7b-b33fc1c785a2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4e4eff0c95194682ade3e24aec451cd7'} {{(pid=81767) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 16:15:22.448052 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-859e1e41-b597-4687-9b42-d06af7582fe7 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a09d128dee9438283633e51f4834643', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4ee0413f9ef4d4fb1f95a660babe090', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 16:15:22.448669 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-859e1e41-b597-4687-9b42-d06af7582fe7 service swift] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:15:22.448960 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-859e1e41-b597-4687-9b42-d06af7582fe7 service swift] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:15:22.449360 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-859e1e41-b597-4687-9b42-d06af7582fe7 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:15:22.451376 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-859e1e41-b597-4687-9b42-d06af7582fe7 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 16:15:22.463807 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-859e1e41-b597-4687-9b42-d06af7582fe7 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 16:15:22.465652 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-859e1e41-b597-4687-9b42-d06af7582fe7 service swift] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 16:15:22.474986 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 1397/2751] 10.222.0.15 () {60 vars in 1384 bytes} [Mon Aug 22 16:15:22 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Aug 22 16:15:22.517219 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.notifications [None req-b8cd29ba-c338-444f-bc7b-b33fc1c785a2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4e4eff0c95194682ade3e24aec451cd7'} {{(pid=81767) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 16:15:22.523093 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.notifications [None req-b8cd29ba-c338-444f-bc7b-b33fc1c785a2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4e4eff0c95194682ade3e24aec451cd7'} {{(pid=81767) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 16:15:22.537508 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.notifications [None req-b8cd29ba-c338-444f-bc7b-b33fc1c785a2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4e4eff0c95194682ade3e24aec451cd7'} {{(pid=81767) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 16:15:22.543656 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 1355/2752] 10.222.0.15 () {58 vars in 1224 bytes} [Mon Aug 22 16:15:22 2022] DELETE /identity/v3/users/4e4eff0c95194682ade3e24aec451cd7 => generated 0 bytes in 400 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 16:15:22.850795 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-859e1e41-b597-4687-9b42-d06af7582fe7 service swift] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 16:15:22.852367 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-859e1e41-b597-4687-9b42-d06af7582fe7 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 16:15:22.874103 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-859e1e41-b597-4687-9b42-d06af7582fe7 service swift] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 16:15:22.934653 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98755568-d1e9-4083-ae40-078f2472221d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a98b1e4431940ab8d1fc95df2858a6c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 16:15:22.935315 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98755568-d1e9-4083-ae40-078f2472221d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:15:22.935587 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98755568-d1e9-4083-ae40-078f2472221d admin admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:15:22.935860 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98755568-d1e9-4083-ae40-078f2472221d admin admin] PATH_INFO: `/v3/projects/02aae2708d7b41e7952cfba658be47db` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:15:22.936984 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98755568-d1e9-4083-ae40-078f2472221d admin admin] RBAC: Authorizing `identity:delete_project(project_id=02aae2708d7b41e7952cfba658be47db)` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 16:15:22.939665 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98755568-d1e9-4083-ae40-078f2472221d admin admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 16:15:23.040682 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-98755568-d1e9-4083-ae40-078f2472221d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '02aae2708d7b41e7952cfba658be47db', 'request_id': 'req-98755568-d1e9-4083-ae40-078f2472221d'} {{(pid=81768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 16:15:23.046636 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-98755568-d1e9-4083-ae40-078f2472221d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '02aae2708d7b41e7952cfba658be47db', 'request_id': 'req-98755568-d1e9-4083-ae40-078f2472221d'} {{(pid=81768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 16:15:23.118262 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-98755568-d1e9-4083-ae40-078f2472221d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '02aae2708d7b41e7952cfba658be47db', 'request_id': 'req-98755568-d1e9-4083-ae40-078f2472221d'} {{(pid=81768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 16:15:23.131365 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-98755568-d1e9-4083-ae40-078f2472221d admin admin] The token cache is being invalidate because project 02aae2708d7b41e7952cfba658be47db was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81768) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 22 16:15:23.131665 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-98755568-d1e9-4083-ae40-078f2472221d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 16:15:23.132318 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-98755568-d1e9-4083-ae40-078f2472221d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 16:15:23.133843 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 1398/2753] 10.222.0.15 () {58 vars in 1233 bytes} [Mon Aug 22 16:15:22 2022] DELETE /identity/v3/projects/02aae2708d7b41e7952cfba658be47db => generated 0 bytes in 284 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 16:15:23.138093 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8cd29ba-c338-444f-bc7b-b33fc1c785a2 admin admin] Authenticating user token {{(pid=81767) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 16:15:23.154834 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8cd29ba-c338-444f-bc7b-b33fc1c785a2 admin admin] Validating token access rules against request {{(pid=81767) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 16:15:23.237005 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-536b1829-0bb6-43c2-8c3b-88d0377b422f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a98b1e4431940ab8d1fc95df2858a6c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81767) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 16:15:23.237683 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-536b1829-0bb6-43c2-8c3b-88d0377b422f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:15:23.238861 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-536b1829-0bb6-43c2-8c3b-88d0377b422f admin admin] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:15:23.238861 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-536b1829-0bb6-43c2-8c3b-88d0377b422f admin admin] PATH_INFO: `/v3/users/0e9eda76738143a793abe3173c669a31` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:15:23.239557 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-536b1829-0bb6-43c2-8c3b-88d0377b422f admin admin] RBAC: Authorizing `identity:delete_user(user_id=0e9eda76738143a793abe3173c669a31)` {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 16:15:23.241559 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-536b1829-0bb6-43c2-8c3b-88d0377b422f admin admin] RBAC: Authorization granted {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 16:15:23.265165 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG dbcounter [-] [81767] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=81767) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 16:15:23.482979 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.notifications [None req-536b1829-0bb6-43c2-8c3b-88d0377b422f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0e9eda76738143a793abe3173c669a31'} {{(pid=81767) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 16:15:23.483705 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.notifications [None req-536b1829-0bb6-43c2-8c3b-88d0377b422f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0e9eda76738143a793abe3173c669a31'} {{(pid=81767) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 16:15:23.581047 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.notifications [None req-536b1829-0bb6-43c2-8c3b-88d0377b422f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0e9eda76738143a793abe3173c669a31'} {{(pid=81767) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 16:15:23.588710 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.notifications [None req-536b1829-0bb6-43c2-8c3b-88d0377b422f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0e9eda76738143a793abe3173c669a31'} {{(pid=81767) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 16:15:23.601937 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.notifications [None req-536b1829-0bb6-43c2-8c3b-88d0377b422f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0e9eda76738143a793abe3173c669a31'} {{(pid=81767) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 16:15:23.608435 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 1356/2754] 10.222.0.15 () {58 vars in 1224 bytes} [Mon Aug 22 16:15:23 2022] DELETE /identity/v3/users/0e9eda76738143a793abe3173c669a31 => generated 0 bytes in 471 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 22 16:15:23.856725 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98755568-d1e9-4083-ae40-078f2472221d admin admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 16:15:23.858363 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-98755568-d1e9-4083-ae40-078f2472221d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 16:15:23.873182 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98755568-d1e9-4083-ae40-078f2472221d admin admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 16:15:23.942359 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78a52752-af5f-47ea-a5d5-990f6bb7a2af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a98b1e4431940ab8d1fc95df2858a6c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 16:15:23.943031 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78a52752-af5f-47ea-a5d5-990f6bb7a2af admin admin] REQUEST_METHOD: `DELETE` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:15:23.943328 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78a52752-af5f-47ea-a5d5-990f6bb7a2af admin admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:15:23.943610 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78a52752-af5f-47ea-a5d5-990f6bb7a2af admin admin] PATH_INFO: `/v3/projects/7845efac021d4df19d745e6cc4d00d67` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:15:23.944818 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78a52752-af5f-47ea-a5d5-990f6bb7a2af admin admin] RBAC: Authorizing `identity:delete_project(project_id=7845efac021d4df19d745e6cc4d00d67)` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 16:15:23.946731 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78a52752-af5f-47ea-a5d5-990f6bb7a2af admin admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 16:15:23.964495 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 16:15:24.092795 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-78a52752-af5f-47ea-a5d5-990f6bb7a2af admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7845efac021d4df19d745e6cc4d00d67', 'request_id': 'req-78a52752-af5f-47ea-a5d5-990f6bb7a2af'} {{(pid=81768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 16:15:24.098897 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-78a52752-af5f-47ea-a5d5-990f6bb7a2af admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7845efac021d4df19d745e6cc4d00d67', 'request_id': 'req-78a52752-af5f-47ea-a5d5-990f6bb7a2af'} {{(pid=81768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 16:15:24.189211 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-78a52752-af5f-47ea-a5d5-990f6bb7a2af admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7845efac021d4df19d745e6cc4d00d67', 'request_id': 'req-78a52752-af5f-47ea-a5d5-990f6bb7a2af'} {{(pid=81768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 16:15:24.206692 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-78a52752-af5f-47ea-a5d5-990f6bb7a2af admin admin] The token cache is being invalidate because project 7845efac021d4df19d745e6cc4d00d67 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81768) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 22 16:15:24.206986 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-78a52752-af5f-47ea-a5d5-990f6bb7a2af admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 16:15:24.207684 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-78a52752-af5f-47ea-a5d5-990f6bb7a2af admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 16:15:24.209183 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 1399/2755] 10.222.0.15 () {58 vars in 1233 bytes} [Mon Aug 22 16:15:23 2022] DELETE /identity/v3/projects/7845efac021d4df19d745e6cc4d00d67 => generated 0 bytes in 353 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 16:15:24.522389 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-536b1829-0bb6-43c2-8c3b-88d0377b422f admin admin] Authenticating user token {{(pid=81767) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 16:15:24.523845 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.fernet_utils [None req-536b1829-0bb6-43c2-8c3b-88d0377b422f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81767) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 16:15:24.538284 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-536b1829-0bb6-43c2-8c3b-88d0377b422f admin admin] Validating token access rules against request {{(pid=81767) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 16:15:24.602027 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82723dd8-2dfd-4452-b9cb-04a050bb2834 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a98b1e4431940ab8d1fc95df2858a6c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81767) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 16:15:24.602680 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82723dd8-2dfd-4452-b9cb-04a050bb2834 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:15:24.602961 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82723dd8-2dfd-4452-b9cb-04a050bb2834 admin admin] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:15:24.603237 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82723dd8-2dfd-4452-b9cb-04a050bb2834 admin admin] PATH_INFO: `/v3/users/5e670b956f0c4e4ea8a5ff139742d956` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:15:24.604437 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82723dd8-2dfd-4452-b9cb-04a050bb2834 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5e670b956f0c4e4ea8a5ff139742d956)` {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 16:15:24.606729 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82723dd8-2dfd-4452-b9cb-04a050bb2834 admin admin] RBAC: Authorization granted {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 16:15:24.812372 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.notifications [None req-82723dd8-2dfd-4452-b9cb-04a050bb2834 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5e670b956f0c4e4ea8a5ff139742d956'} {{(pid=81767) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 16:15:24.813070 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.notifications [None req-82723dd8-2dfd-4452-b9cb-04a050bb2834 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5e670b956f0c4e4ea8a5ff139742d956'} {{(pid=81767) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 16:15:24.915028 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.notifications [None req-82723dd8-2dfd-4452-b9cb-04a050bb2834 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5e670b956f0c4e4ea8a5ff139742d956'} {{(pid=81767) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 16:15:24.923262 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.notifications [None req-82723dd8-2dfd-4452-b9cb-04a050bb2834 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5e670b956f0c4e4ea8a5ff139742d956'} {{(pid=81767) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 16:15:24.938243 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.notifications [None req-82723dd8-2dfd-4452-b9cb-04a050bb2834 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5e670b956f0c4e4ea8a5ff139742d956'} {{(pid=81767) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 16:15:24.944345 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 1357/2756] 10.222.0.15 () {58 vars in 1224 bytes} [Mon Aug 22 16:15:24 2022] DELETE /identity/v3/users/5e670b956f0c4e4ea8a5ff139742d956 => generated 0 bytes in 423 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 16:15:24.949117 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80d1e7fe-e154-4ebd-a642-990d0ede3357 None None] REQUEST_METHOD: `POST` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:15:24.949591 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80d1e7fe-e154-4ebd-a642-990d0ede3357 None None] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:15:24.949700 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80d1e7fe-e154-4ebd-a642-990d0ede3357 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:15:24.970429 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.auth.core [None req-80d1e7fe-e154-4ebd-a642-990d0ede3357 None None] MFA Rules not processed for user `6b2199ae0fb0480aa917ada391f5ec54`. Rule list: `[]` (Enabled: `True`). {{(pid=81768) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 16:15:24.971750 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-80d1e7fe-e154-4ebd-a642-990d0ede3357 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 16:15:25.047778 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 1400/2757] 10.222.0.15 () {58 vars in 958 bytes} [Mon Aug 22 16:15:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 16:15:25.057903 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82723dd8-2dfd-4452-b9cb-04a050bb2834 admin admin] Authenticating user token {{(pid=81767) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 16:15:25.059415 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.fernet_utils [None req-82723dd8-2dfd-4452-b9cb-04a050bb2834 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81767) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 16:15:25.081219 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG dbcounter [-] [81767] Writing DB stats keystone:DELETE=17,keystone:SELECT=81,keystone:INSERT=2 {{(pid=81767) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 16:15:25.120862 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82723dd8-2dfd-4452-b9cb-04a050bb2834 admin admin] Validating token access rules against request {{(pid=81767) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 16:15:25.124879 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8cd3f97-a485-4eb2-ad4d-0295f4add27c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '234687adad594f1aa45775a565ffbdc7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4ee0413f9ef4d4fb1f95a660babe090', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81767) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 16:15:25.125707 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8cd3f97-a485-4eb2-ad4d-0295f4add27c service neutron] REQUEST_METHOD: `GET` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:15:25.126385 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8cd3f97-a485-4eb2-ad4d-0295f4add27c service neutron] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:15:25.126952 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8cd3f97-a485-4eb2-ad4d-0295f4add27c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:15:25.138517 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8cd3f97-a485-4eb2-ad4d-0295f4add27c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 16:15:25.140205 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8cd3f97-a485-4eb2-ad4d-0295f4add27c service neutron] RBAC: Authorization granted {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 16:15:25.151815 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 1358/2758] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Aug 22 16:15:25 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 16:15:25.823584 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80d1e7fe-e154-4ebd-a642-990d0ede3357 None None] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 16:15:25.825866 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-80d1e7fe-e154-4ebd-a642-990d0ede3357 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 16:15:25.842028 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80d1e7fe-e154-4ebd-a642-990d0ede3357 None None] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 16:15:25.853659 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c5259cf-0229-4451-996d-75eb4f576d83 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a98b1e4431940ab8d1fc95df2858a6c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 16:15:25.854277 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c5259cf-0229-4451-996d-75eb4f576d83 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:15:25.854562 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c5259cf-0229-4451-996d-75eb4f576d83 admin admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:15:25.854838 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c5259cf-0229-4451-996d-75eb4f576d83 admin admin] PATH_INFO: `/v3/projects/50af9c822c4343409d1063058379b125` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:15:25.856119 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c5259cf-0229-4451-996d-75eb4f576d83 admin admin] RBAC: Authorizing `identity:delete_project(project_id=50af9c822c4343409d1063058379b125)` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 16:15:25.858924 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c5259cf-0229-4451-996d-75eb4f576d83 admin admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 16:15:25.865250 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8cd3f97-a485-4eb2-ad4d-0295f4add27c service neutron] Authenticating user token {{(pid=81767) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 16:15:25.867883 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.fernet_utils [None req-d8cd3f97-a485-4eb2-ad4d-0295f4add27c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81767) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 16:15:25.884429 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8cd3f97-a485-4eb2-ad4d-0295f4add27c service neutron] Validating token access rules against request {{(pid=81767) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 16:15:25.889476 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8826d92-f470-4549-8138-06538afa1ce6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a98b1e4431940ab8d1fc95df2858a6c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81767) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 16:15:25.890222 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8826d92-f470-4549-8138-06538afa1ce6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:15:25.890492 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8826d92-f470-4549-8138-06538afa1ce6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:15:25.891087 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8826d92-f470-4549-8138-06538afa1ce6 admin admin] PATH_INFO: `/v3/users/521c78587730446297375484f3e9d206` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:15:25.892635 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8826d92-f470-4549-8138-06538afa1ce6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=521c78587730446297375484f3e9d206)` {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 16:15:25.894577 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8826d92-f470-4549-8138-06538afa1ce6 admin admin] RBAC: Authorization granted {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 16:15:25.982101 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-3c5259cf-0229-4451-996d-75eb4f576d83 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '50af9c822c4343409d1063058379b125', 'request_id': 'req-3c5259cf-0229-4451-996d-75eb4f576d83'} {{(pid=81768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 16:15:25.988390 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-3c5259cf-0229-4451-996d-75eb4f576d83 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '50af9c822c4343409d1063058379b125', 'request_id': 'req-3c5259cf-0229-4451-996d-75eb4f576d83'} {{(pid=81768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 16:15:26.083744 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-3c5259cf-0229-4451-996d-75eb4f576d83 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '50af9c822c4343409d1063058379b125', 'request_id': 'req-3c5259cf-0229-4451-996d-75eb4f576d83'} {{(pid=81768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 16:15:26.097345 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-3c5259cf-0229-4451-996d-75eb4f576d83 admin admin] The token cache is being invalidate because project 50af9c822c4343409d1063058379b125 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81768) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 22 16:15:26.097677 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-3c5259cf-0229-4451-996d-75eb4f576d83 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 16:15:26.098526 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-3c5259cf-0229-4451-996d-75eb4f576d83 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 16:15:26.102600 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 1401/2759] 10.222.0.15 () {58 vars in 1233 bytes} [Mon Aug 22 16:15:25 2022] DELETE /identity/v3/projects/50af9c822c4343409d1063058379b125 => generated 0 bytes in 280 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 16:15:26.107765 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2e9138c-2606-4d73-bdb8-d82b27ad6803 None None] REQUEST_METHOD: `POST` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:15:26.108083 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2e9138c-2606-4d73-bdb8-d82b27ad6803 None None] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:15:26.108360 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2e9138c-2606-4d73-bdb8-d82b27ad6803 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:15:26.116688 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.notifications [None req-e8826d92-f470-4549-8138-06538afa1ce6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '521c78587730446297375484f3e9d206'} {{(pid=81767) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 16:15:26.117421 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.notifications [None req-e8826d92-f470-4549-8138-06538afa1ce6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '521c78587730446297375484f3e9d206'} {{(pid=81767) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 16:15:26.129648 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.auth.core [None req-a2e9138c-2606-4d73-bdb8-d82b27ad6803 None None] MFA Rules not processed for user `6b2199ae0fb0480aa917ada391f5ec54`. Rule list: `[]` (Enabled: `True`). {{(pid=81768) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 16:15:26.130857 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-a2e9138c-2606-4d73-bdb8-d82b27ad6803 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 16:15:26.150502 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 16:15:26.195265 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 1402/2760] 10.222.0.15 () {58 vars in 958 bytes} [Mon Aug 22 16:15:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 16:15:26.203107 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d299fd2f-5e12-49b7-bede-6040d175f997 None None] REQUEST_METHOD: `POST` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:15:26.204061 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d299fd2f-5e12-49b7-bede-6040d175f997 None None] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:15:26.204396 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d299fd2f-5e12-49b7-bede-6040d175f997 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:15:26.209915 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.notifications [None req-e8826d92-f470-4549-8138-06538afa1ce6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '521c78587730446297375484f3e9d206'} {{(pid=81767) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 16:15:26.219758 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.notifications [None req-e8826d92-f470-4549-8138-06538afa1ce6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '521c78587730446297375484f3e9d206'} {{(pid=81767) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 16:15:26.226337 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.auth.core [None req-d299fd2f-5e12-49b7-bede-6040d175f997 None None] MFA Rules not processed for user `6b2199ae0fb0480aa917ada391f5ec54`. Rule list: `[]` (Enabled: `True`). {{(pid=81768) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 16:15:26.227579 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-d299fd2f-5e12-49b7-bede-6040d175f997 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 16:15:26.236401 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 1403/2761] 10.222.0.15 () {58 vars in 958 bytes} [Mon Aug 22 16:15:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 16:15:26.237633 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.notifications [None req-e8826d92-f470-4549-8138-06538afa1ce6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '521c78587730446297375484f3e9d206'} {{(pid=81767) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 16:15:26.240517 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d299fd2f-5e12-49b7-bede-6040d175f997 None None] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 16:15:26.245473 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 1359/2762] 10.222.0.15 () {58 vars in 1224 bytes} [Mon Aug 22 16:15:25 2022] DELETE /identity/v3/users/521c78587730446297375484f3e9d206 => generated 0 bytes in 381 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 22 16:15:26.249830 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d299fd2f-5e12-49b7-bede-6040d175f997 None None] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 16:15:26.311825 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0524d95a-752d-4fa9-b1be-2d4490c8745a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a98b1e4431940ab8d1fc95df2858a6c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 16:15:26.312664 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0524d95a-752d-4fa9-b1be-2d4490c8745a admin admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:15:26.314683 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0524d95a-752d-4fa9-b1be-2d4490c8745a admin admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:15:26.314683 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0524d95a-752d-4fa9-b1be-2d4490c8745a admin admin] PATH_INFO: `/v3/domains` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:15:26.315043 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0524d95a-752d-4fa9-b1be-2d4490c8745a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81768) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 16:15:26.315470 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0524d95a-752d-4fa9-b1be-2d4490c8745a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 16:15:26.317564 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0524d95a-752d-4fa9-b1be-2d4490c8745a admin admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 16:15:26.338492 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 1404/2763] 10.222.0.15 () {58 vars in 1153 bytes} [Mon Aug 22 16:15:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 95 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 16:15:26.338743 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8826d92-f470-4549-8138-06538afa1ce6 admin admin] Authenticating user token {{(pid=81767) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 16:15:26.353546 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8826d92-f470-4549-8138-06538afa1ce6 admin admin] Validating token access rules against request {{(pid=81767) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 16:15:26.367654 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0d271c9-b745-41cd-999c-308f36471607 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a98b1e4431940ab8d1fc95df2858a6c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81767) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 16:15:26.368620 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0d271c9-b745-41cd-999c-308f36471607 admin admin] REQUEST_METHOD: `POST` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:15:26.369071 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0d271c9-b745-41cd-999c-308f36471607 admin admin] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:15:26.369521 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0d271c9-b745-41cd-999c-308f36471607 admin admin] PATH_INFO: `/v3/projects` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:15:26.370702 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0d271c9-b745-41cd-999c-308f36471607 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 16:15:26.372835 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0d271c9-b745-41cd-999c-308f36471607 admin admin] RBAC: Authorization granted {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 16:15:26.479622 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 1360/2764] 10.222.0.15 () {60 vars in 1153 bytes} [Mon Aug 22 16:15:26 2022] POST /identity/v3/projects => generated 383 bytes in 144 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 22 16:15:26.484054 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0524d95a-752d-4fa9-b1be-2d4490c8745a admin admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 16:15:26.494275 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0524d95a-752d-4fa9-b1be-2d4490c8745a admin admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 16:15:26.550748 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b4cb060-e279-4b7a-a63d-fe7879302022 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a98b1e4431940ab8d1fc95df2858a6c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 16:15:26.551384 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b4cb060-e279-4b7a-a63d-fe7879302022 admin admin] REQUEST_METHOD: `POST` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:15:26.551666 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b4cb060-e279-4b7a-a63d-fe7879302022 admin admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:15:26.551941 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b4cb060-e279-4b7a-a63d-fe7879302022 admin admin] PATH_INFO: `/v3/users` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:15:26.552565 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b4cb060-e279-4b7a-a63d-fe7879302022 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 16:15:26.554675 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b4cb060-e279-4b7a-a63d-fe7879302022 admin admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 16:15:26.573897 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0d271c9-b745-41cd-999c-308f36471607 admin admin] Authenticating user token {{(pid=81767) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 16:15:26.575673 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.fernet_utils [None req-c0d271c9-b745-41cd-999c-308f36471607 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81767) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 16:15:26.589760 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0d271c9-b745-41cd-999c-308f36471607 admin admin] Validating token access rules against request {{(pid=81767) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 16:15:26.595409 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4191d395-537a-4d2b-abeb-3f737046cfc8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a98b1e4431940ab8d1fc95df2858a6c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81767) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 16:15:26.596046 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4191d395-537a-4d2b-abeb-3f737046cfc8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:15:26.596325 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4191d395-537a-4d2b-abeb-3f737046cfc8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:15:26.596596 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4191d395-537a-4d2b-abeb-3f737046cfc8 admin admin] PATH_INFO: `/v3/projects/92e2834fc9a448a28ee12bfbbfc4fee4` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:15:26.597734 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4191d395-537a-4d2b-abeb-3f737046cfc8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=92e2834fc9a448a28ee12bfbbfc4fee4)` {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 16:15:26.599976 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4191d395-537a-4d2b-abeb-3f737046cfc8 admin admin] RBAC: Authorization granted {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 16:15:26.644962 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 1405/2765] 10.222.0.15 () {60 vars in 1144 bytes} [Mon Aug 22 16:15:26 2022] POST /identity/v3/users => generated 292 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 22 16:15:26.648936 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b4cb060-e279-4b7a-a63d-fe7879302022 admin admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 16:15:26.659329 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b4cb060-e279-4b7a-a63d-fe7879302022 admin admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 16:15:26.664003 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c66fe9fa-8790-4a4c-ae14-2ae1609c8b4d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a98b1e4431940ab8d1fc95df2858a6c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 16:15:26.664610 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c66fe9fa-8790-4a4c-ae14-2ae1609c8b4d admin admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:15:26.664901 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c66fe9fa-8790-4a4c-ae14-2ae1609c8b4d admin admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:15:26.665159 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c66fe9fa-8790-4a4c-ae14-2ae1609c8b4d admin admin] PATH_INFO: `/v3/roles` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:15:26.665800 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c66fe9fa-8790-4a4c-ae14-2ae1609c8b4d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 16:15:26.667417 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c66fe9fa-8790-4a4c-ae14-2ae1609c8b4d admin admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 16:15:26.678307 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 1406/2766] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Aug 22 16:15:26 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 22 16:15:26.682345 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c66fe9fa-8790-4a4c-ae14-2ae1609c8b4d admin admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 16:15:26.692458 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c66fe9fa-8790-4a4c-ae14-2ae1609c8b4d admin admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 16:15:26.697076 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcb648b7-b79c-4646-9108-abcf3202ea9a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a98b1e4431940ab8d1fc95df2858a6c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 16:15:26.697625 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcb648b7-b79c-4646-9108-abcf3202ea9a admin admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:15:26.697904 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcb648b7-b79c-4646-9108-abcf3202ea9a admin admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:15:26.698175 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcb648b7-b79c-4646-9108-abcf3202ea9a admin admin] PATH_INFO: `/v3/roles` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:15:26.698790 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcb648b7-b79c-4646-9108-abcf3202ea9a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 16:15:26.700357 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcb648b7-b79c-4646-9108-abcf3202ea9a admin admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 16:15:26.712003 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 1407/2767] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Aug 22 16:15:26 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 22 16:15:26.716246 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcb648b7-b79c-4646-9108-abcf3202ea9a admin admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 16:15:26.723318 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.notifications [None req-4191d395-537a-4d2b-abeb-3f737046cfc8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '92e2834fc9a448a28ee12bfbbfc4fee4', 'request_id': 'req-4191d395-537a-4d2b-abeb-3f737046cfc8'} {{(pid=81767) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 16:15:26.726293 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcb648b7-b79c-4646-9108-abcf3202ea9a admin admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 16:15:26.731299 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.notifications [None req-4191d395-537a-4d2b-abeb-3f737046cfc8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '92e2834fc9a448a28ee12bfbbfc4fee4', 'request_id': 'req-4191d395-537a-4d2b-abeb-3f737046cfc8'} {{(pid=81767) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 16:15:26.782949 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b79123a-fba1-41c1-809d-f5906869a430 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a98b1e4431940ab8d1fc95df2858a6c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 16:15:26.783620 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b79123a-fba1-41c1-809d-f5906869a430 admin admin] REQUEST_METHOD: `PUT` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:15:26.783903 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b79123a-fba1-41c1-809d-f5906869a430 admin admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:15:26.784179 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b79123a-fba1-41c1-809d-f5906869a430 admin admin] PATH_INFO: `/v3/projects/dd773e1588cb44459ec79f593c35a860/users/6413010933f6416a8d2e3ee82c1946e2/roles/e6fc1ee3fb974125ba9b4fe55b12b45e` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:15:26.797984 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b79123a-fba1-41c1-809d-f5906869a430 admin admin] RBAC: Authorizing `identity:create_grant(project_id=dd773e1588cb44459ec79f593c35a860, user_id=6413010933f6416a8d2e3ee82c1946e2, role_id=e6fc1ee3fb974125ba9b4fe55b12b45e)` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 16:15:26.800849 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b79123a-fba1-41c1-809d-f5906869a430 admin admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 16:15:26.837724 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.notifications [None req-4191d395-537a-4d2b-abeb-3f737046cfc8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '92e2834fc9a448a28ee12bfbbfc4fee4', 'request_id': 'req-4191d395-537a-4d2b-abeb-3f737046cfc8'} {{(pid=81767) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 16:15:26.839098 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG dbcounter [-] [81767] Writing DB stats keystone:SELECT=80,keystone:DELETE=17,keystone:INSERT=3 {{(pid=81767) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 16:15:26.850344 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.notifications [None req-4191d395-537a-4d2b-abeb-3f737046cfc8 admin admin] The token cache is being invalidate because project 92e2834fc9a448a28ee12bfbbfc4fee4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81767) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 22 16:15:26.850630 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.notifications [None req-4191d395-537a-4d2b-abeb-3f737046cfc8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81767) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 16:15:26.851295 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.notifications [None req-4191d395-537a-4d2b-abeb-3f737046cfc8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81767) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 16:15:26.852778 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 1361/2768] 10.222.0.15 () {58 vars in 1233 bytes} [Mon Aug 22 16:15:26 2022] DELETE /identity/v3/projects/92e2834fc9a448a28ee12bfbbfc4fee4 => generated 0 bytes in 279 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 22 16:15:26.857638 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46010cc4-e08e-4129-a628-02490864b8c5 None None] REQUEST_METHOD: `POST` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:15:26.857864 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46010cc4-e08e-4129-a628-02490864b8c5 None None] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:15:26.858133 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46010cc4-e08e-4129-a628-02490864b8c5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:15:26.877246 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.auth.core [None req-46010cc4-e08e-4129-a628-02490864b8c5 None None] MFA Rules not processed for user `6b2199ae0fb0480aa917ada391f5ec54`. Rule list: `[]` (Enabled: `True`). {{(pid=81767) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 16:15:26.880883 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.fernet_utils [None req-46010cc4-e08e-4129-a628-02490864b8c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81767) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 16:15:26.886258 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 1362/2769] 10.222.0.15 () {58 vars in 958 bytes} [Mon Aug 22 16:15:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 22 16:15:26.894959 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98feb123-7c8b-4741-8997-88c83a43a7f0 None None] REQUEST_METHOD: `POST` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:15:26.895246 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98feb123-7c8b-4741-8997-88c83a43a7f0 None None] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:15:26.895567 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98feb123-7c8b-4741-8997-88c83a43a7f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:15:26.915184 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.auth.core [None req-98feb123-7c8b-4741-8997-88c83a43a7f0 None None] MFA Rules not processed for user `6b2199ae0fb0480aa917ada391f5ec54`. Rule list: `[]` (Enabled: `True`). {{(pid=81767) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 16:15:26.916594 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.fernet_utils [None req-98feb123-7c8b-4741-8997-88c83a43a7f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81767) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 16:15:26.931894 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 1408/2770] 10.222.0.15 () {60 vars in 1483 bytes} [Mon Aug 22 16:15:26 2022] PUT /identity/v3/projects/dd773e1588cb44459ec79f593c35a860/users/6413010933f6416a8d2e3ee82c1946e2/roles/e6fc1ee3fb974125ba9b4fe55b12b45e => generated 0 bytes in 208 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 22 16:15:26.935683 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04381cba-2b46-4eec-b0f7-0dbd9367aecf None None] REQUEST_METHOD: `POST` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:15:26.935985 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04381cba-2b46-4eec-b0f7-0dbd9367aecf None None] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:15:26.936241 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04381cba-2b46-4eec-b0f7-0dbd9367aecf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:15:26.956111 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.auth.core [None req-04381cba-2b46-4eec-b0f7-0dbd9367aecf None None] MFA Rules not processed for user `6b2199ae0fb0480aa917ada391f5ec54`. Rule list: `[]` (Enabled: `True`). {{(pid=81768) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 16:15:26.957294 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-04381cba-2b46-4eec-b0f7-0dbd9367aecf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 16:15:26.977512 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 1363/2771] 10.222.0.15 () {58 vars in 958 bytes} [Mon Aug 22 16:15:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 16:15:26.984208 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98feb123-7c8b-4741-8997-88c83a43a7f0 None None] Authenticating user token {{(pid=81767) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 16:15:26.990845 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98feb123-7c8b-4741-8997-88c83a43a7f0 None None] Validating token access rules against request {{(pid=81767) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 16:15:26.996621 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcf4f349-db57-452f-8063-a145260466ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a98b1e4431940ab8d1fc95df2858a6c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81767) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 16:15:26.997638 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcf4f349-db57-452f-8063-a145260466ff admin admin] REQUEST_METHOD: `GET` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:15:26.997911 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcf4f349-db57-452f-8063-a145260466ff admin admin] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:15:26.998182 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcf4f349-db57-452f-8063-a145260466ff admin admin] PATH_INFO: `/v3/domains` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:15:26.998831 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcf4f349-db57-452f-8063-a145260466ff admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81767) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 16:15:26.999128 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcf4f349-db57-452f-8063-a145260466ff admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 16:15:27.001279 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcf4f349-db57-452f-8063-a145260466ff admin admin] RBAC: Authorization granted {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 16:15:27.013553 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 1364/2772] 10.222.0.15 () {58 vars in 1153 bytes} [Mon Aug 22 16:15:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 22 16:15:27.019158 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 1409/2773] 10.222.0.15 () {58 vars in 958 bytes} [Mon Aug 22 16:15:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 22 16:15:27.028775 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcf4f349-db57-452f-8063-a145260466ff admin admin] Authenticating user token {{(pid=81767) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 16:15:27.035652 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04381cba-2b46-4eec-b0f7-0dbd9367aecf None None] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 16:15:27.037150 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-04381cba-2b46-4eec-b0f7-0dbd9367aecf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 16:15:27.039845 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcf4f349-db57-452f-8063-a145260466ff admin admin] Validating token access rules against request {{(pid=81767) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 16:15:27.051944 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1343745c-e6a2-4809-9f73-e8a37486951c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a98b1e4431940ab8d1fc95df2858a6c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81767) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 16:15:27.053548 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1343745c-e6a2-4809-9f73-e8a37486951c admin admin] REQUEST_METHOD: `POST` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:15:27.053836 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1343745c-e6a2-4809-9f73-e8a37486951c admin admin] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:15:27.054303 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1343745c-e6a2-4809-9f73-e8a37486951c admin admin] PATH_INFO: `/v3/projects` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:15:27.054945 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1343745c-e6a2-4809-9f73-e8a37486951c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 16:15:27.059248 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1343745c-e6a2-4809-9f73-e8a37486951c admin admin] RBAC: Authorization granted {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 16:15:27.097938 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04381cba-2b46-4eec-b0f7-0dbd9367aecf None None] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 16:15:27.101719 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47e08533-fc6e-425c-8c9e-2874cd3abfa1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '234687adad594f1aa45775a565ffbdc7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4ee0413f9ef4d4fb1f95a660babe090', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 16:15:27.102318 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e08533-fc6e-425c-8c9e-2874cd3abfa1 service neutron] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:15:27.102597 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e08533-fc6e-425c-8c9e-2874cd3abfa1 service neutron] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:15:27.102873 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e08533-fc6e-425c-8c9e-2874cd3abfa1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:15:27.112588 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47e08533-fc6e-425c-8c9e-2874cd3abfa1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 16:15:27.113997 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47e08533-fc6e-425c-8c9e-2874cd3abfa1 service neutron] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 16:15:27.123385 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 1410/2774] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Aug 22 16:15:27 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 16:15:27.168209 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 1365/2775] 10.222.0.15 () {60 vars in 1153 bytes} [Mon Aug 22 16:15:27 2022] POST /identity/v3/projects => generated 369 bytes in 140 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 22 16:15:27.173120 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47e08533-fc6e-425c-8c9e-2874cd3abfa1 service neutron] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 16:15:27.183611 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47e08533-fc6e-425c-8c9e-2874cd3abfa1 service neutron] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 16:15:27.190317 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 16:15:27.242924 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d6bb3dd-8c43-4d7f-ae35-4af3de6ea5c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a98b1e4431940ab8d1fc95df2858a6c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 16:15:27.243556 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d6bb3dd-8c43-4d7f-ae35-4af3de6ea5c1 admin admin] REQUEST_METHOD: `POST` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:15:27.243845 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d6bb3dd-8c43-4d7f-ae35-4af3de6ea5c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:15:27.244109 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d6bb3dd-8c43-4d7f-ae35-4af3de6ea5c1 admin admin] PATH_INFO: `/v3/users` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:15:27.244718 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d6bb3dd-8c43-4d7f-ae35-4af3de6ea5c1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 16:15:27.246632 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d6bb3dd-8c43-4d7f-ae35-4af3de6ea5c1 admin admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 16:15:27.331555 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 1411/2776] 10.222.0.15 () {60 vars in 1144 bytes} [Mon Aug 22 16:15:27 2022] POST /identity/v3/users => generated 285 bytes in 158 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 22 16:15:27.340526 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1343745c-e6a2-4809-9f73-e8a37486951c admin admin] Authenticating user token {{(pid=81767) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 16:15:27.354394 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1343745c-e6a2-4809-9f73-e8a37486951c admin admin] Validating token access rules against request {{(pid=81767) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 16:15:27.359420 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13c52c25-df13-47b2-a8e5-c7407d03bfb3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a98b1e4431940ab8d1fc95df2858a6c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81767) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 16:15:27.360700 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13c52c25-df13-47b2-a8e5-c7407d03bfb3 admin admin] REQUEST_METHOD: `GET` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:15:27.360980 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13c52c25-df13-47b2-a8e5-c7407d03bfb3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:15:27.361257 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13c52c25-df13-47b2-a8e5-c7407d03bfb3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:15:27.361999 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13c52c25-df13-47b2-a8e5-c7407d03bfb3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 16:15:27.363667 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13c52c25-df13-47b2-a8e5-c7407d03bfb3 admin admin] RBAC: Authorization granted {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 16:15:27.374235 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 1366/2777] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Aug 22 16:15:27 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 22 16:15:27.378854 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d6bb3dd-8c43-4d7f-ae35-4af3de6ea5c1 admin admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 16:15:27.388965 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d6bb3dd-8c43-4d7f-ae35-4af3de6ea5c1 admin admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 16:15:27.393720 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deea1b11-2c16-4db3-957b-0faeadc40bc4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a98b1e4431940ab8d1fc95df2858a6c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 16:15:27.394311 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deea1b11-2c16-4db3-957b-0faeadc40bc4 admin admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:15:27.394667 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deea1b11-2c16-4db3-957b-0faeadc40bc4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:15:27.394847 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deea1b11-2c16-4db3-957b-0faeadc40bc4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:15:27.395542 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deea1b11-2c16-4db3-957b-0faeadc40bc4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 16:15:27.397122 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deea1b11-2c16-4db3-957b-0faeadc40bc4 admin admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 16:15:27.410519 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 1412/2778] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Aug 22 16:15:27 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 22 16:15:27.411933 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13c52c25-df13-47b2-a8e5-c7407d03bfb3 admin admin] Authenticating user token {{(pid=81767) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 16:15:27.422767 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13c52c25-df13-47b2-a8e5-c7407d03bfb3 admin admin] Validating token access rules against request {{(pid=81767) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 16:15:27.427691 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bffff852-42a5-4a4e-9d12-7dc9002edd38 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a98b1e4431940ab8d1fc95df2858a6c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81767) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 16:15:27.428375 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bffff852-42a5-4a4e-9d12-7dc9002edd38 admin admin] REQUEST_METHOD: `PUT` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:15:27.428720 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bffff852-42a5-4a4e-9d12-7dc9002edd38 admin admin] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:15:27.429134 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bffff852-42a5-4a4e-9d12-7dc9002edd38 admin admin] PATH_INFO: `/v3/projects/28bf590bea514dbcaedcd6c65a23bcf2/users/290b9ef0f970472291c232682e4690f7/roles/e6fc1ee3fb974125ba9b4fe55b12b45e` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:15:27.446480 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bffff852-42a5-4a4e-9d12-7dc9002edd38 admin admin] RBAC: Authorizing `identity:create_grant(project_id=28bf590bea514dbcaedcd6c65a23bcf2, user_id=290b9ef0f970472291c232682e4690f7, role_id=e6fc1ee3fb974125ba9b4fe55b12b45e)` {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 16:15:27.449271 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bffff852-42a5-4a4e-9d12-7dc9002edd38 admin admin] RBAC: Authorization granted {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 16:15:27.509880 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 16:15:27.540825 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 1367/2779] 10.222.0.15 () {60 vars in 1483 bytes} [Mon Aug 22 16:15:27 2022] PUT /identity/v3/projects/28bf590bea514dbcaedcd6c65a23bcf2/users/290b9ef0f970472291c232682e4690f7/roles/e6fc1ee3fb974125ba9b4fe55b12b45e => generated 0 bytes in 129 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 16:15:27.552644 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c953666c-1a77-40fb-9eca-4372929d3d19 None None] REQUEST_METHOD: `POST` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:15:27.552916 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c953666c-1a77-40fb-9eca-4372929d3d19 None None] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:15:27.553185 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c953666c-1a77-40fb-9eca-4372929d3d19 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:15:27.581789 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.auth.core [None req-c953666c-1a77-40fb-9eca-4372929d3d19 None None] MFA Rules not processed for user `290b9ef0f970472291c232682e4690f7`. Rule list: `[]` (Enabled: `True`). {{(pid=81768) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 16:15:27.583004 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-c953666c-1a77-40fb-9eca-4372929d3d19 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 16:15:27.656938 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 1413/2780] 10.222.0.15 () {58 vars in 958 bytes} [Mon Aug 22 16:15:27 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 22 16:15:27.669986 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bffff852-42a5-4a4e-9d12-7dc9002edd38 admin admin] Authenticating user token {{(pid=81767) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 16:15:27.671926 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.fernet_utils [None req-bffff852-42a5-4a4e-9d12-7dc9002edd38 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81767) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 16:15:27.736837 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bffff852-42a5-4a4e-9d12-7dc9002edd38 admin admin] Validating token access rules against request {{(pid=81767) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 16:15:27.741257 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab36f005-435c-4b6f-bfef-cb8ad0f843e7 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91f565473c824a15930e2b12863c1ed3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4ee0413f9ef4d4fb1f95a660babe090', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81767) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 16:15:27.741871 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab36f005-435c-4b6f-bfef-cb8ad0f843e7 service glance] REQUEST_METHOD: `GET` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:15:27.742150 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab36f005-435c-4b6f-bfef-cb8ad0f843e7 service glance] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:15:27.742419 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab36f005-435c-4b6f-bfef-cb8ad0f843e7 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:15:27.751424 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab36f005-435c-4b6f-bfef-cb8ad0f843e7 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 16:15:27.752855 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab36f005-435c-4b6f-bfef-cb8ad0f843e7 service glance] RBAC: Authorization granted {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 16:15:27.763063 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 1368/2781] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Aug 22 16:15:27 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 22 16:15:29.797156 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c953666c-1a77-40fb-9eca-4372929d3d19 None None] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 16:15:29.798691 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-c953666c-1a77-40fb-9eca-4372929d3d19 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 16:15:29.884934 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c953666c-1a77-40fb-9eca-4372929d3d19 None None] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 16:15:29.889941 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6d49547-c099-418f-9429-6c8e7fc7b68e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a98b1e4431940ab8d1fc95df2858a6c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 16:15:29.890876 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6d49547-c099-418f-9429-6c8e7fc7b68e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:15:29.891686 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6d49547-c099-418f-9429-6c8e7fc7b68e admin admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:15:29.891973 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6d49547-c099-418f-9429-6c8e7fc7b68e admin admin] PATH_INFO: `/v3/users/2779dbc1258842a48eb70daeecdfeb18` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:15:29.893120 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6d49547-c099-418f-9429-6c8e7fc7b68e admin admin] RBAC: Authorizing `identity:delete_user(user_id=2779dbc1258842a48eb70daeecdfeb18)` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 16:15:29.895195 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6d49547-c099-418f-9429-6c8e7fc7b68e admin admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 16:15:30.112539 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-a6d49547-c099-418f-9429-6c8e7fc7b68e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2779dbc1258842a48eb70daeecdfeb18'} {{(pid=81768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 16:15:30.113289 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-a6d49547-c099-418f-9429-6c8e7fc7b68e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2779dbc1258842a48eb70daeecdfeb18'} {{(pid=81768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 16:15:30.113945 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-a6d49547-c099-418f-9429-6c8e7fc7b68e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2779dbc1258842a48eb70daeecdfeb18'} {{(pid=81768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 16:15:30.126989 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-a6d49547-c099-418f-9429-6c8e7fc7b68e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2779dbc1258842a48eb70daeecdfeb18'} {{(pid=81768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 16:15:30.130680 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=90,keystone:INSERT=3,keystone:DELETE=7 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 16:15:30.207208 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-a6d49547-c099-418f-9429-6c8e7fc7b68e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2779dbc1258842a48eb70daeecdfeb18'} {{(pid=81768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 16:15:30.243300 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 1414/2782] 10.222.0.15 () {58 vars in 1224 bytes} [Mon Aug 22 16:15:29 2022] DELETE /identity/v3/users/2779dbc1258842a48eb70daeecdfeb18 => generated 0 bytes in 447 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 16:15:30.247539 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8efb5d19-4779-43e3-8ed4-543f64727a49 None None] REQUEST_METHOD: `POST` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:15:30.248087 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8efb5d19-4779-43e3-8ed4-543f64727a49 None None] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:15:30.248087 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8efb5d19-4779-43e3-8ed4-543f64727a49 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:15:30.278520 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.auth.core [None req-8efb5d19-4779-43e3-8ed4-543f64727a49 None None] MFA Rules not processed for user `6b2199ae0fb0480aa917ada391f5ec54`. Rule list: `[]` (Enabled: `True`). {{(pid=81767) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 16:15:30.288567 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.fernet_utils [None req-8efb5d19-4779-43e3-8ed4-543f64727a49 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81767) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 16:15:30.299643 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG dbcounter [-] [81767] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81767) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 16:15:30.381088 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 1369/2783] 10.222.0.15 () {58 vars in 958 bytes} [Mon Aug 22 16:15:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 22 16:15:30.392500 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6d49547-c099-418f-9429-6c8e7fc7b68e admin admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 16:15:30.397004 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-a6d49547-c099-418f-9429-6c8e7fc7b68e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 16:15:30.508359 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6d49547-c099-418f-9429-6c8e7fc7b68e admin admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 16:15:30.517514 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-662285e2-8086-40fe-8fbe-0550230803bd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '234687adad594f1aa45775a565ffbdc7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4ee0413f9ef4d4fb1f95a660babe090', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 16:15:30.518164 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-662285e2-8086-40fe-8fbe-0550230803bd service neutron] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:15:30.518439 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-662285e2-8086-40fe-8fbe-0550230803bd service neutron] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:15:30.521950 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-662285e2-8086-40fe-8fbe-0550230803bd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:15:30.537964 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-662285e2-8086-40fe-8fbe-0550230803bd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 16:15:30.539549 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-662285e2-8086-40fe-8fbe-0550230803bd service neutron] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 16:15:30.567748 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 1415/2784] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Aug 22 16:15:30 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 176 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 16:15:31.103229 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8efb5d19-4779-43e3-8ed4-543f64727a49 None None] Authenticating user token {{(pid=81767) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 16:15:31.105588 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.fernet_utils [None req-8efb5d19-4779-43e3-8ed4-543f64727a49 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81767) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 16:15:31.131332 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8efb5d19-4779-43e3-8ed4-543f64727a49 None None] Validating token access rules against request {{(pid=81767) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 16:15:31.139943 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-398211a3-5251-459c-a390-36c2ff403ba6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a98b1e4431940ab8d1fc95df2858a6c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81767) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 16:15:31.140701 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-398211a3-5251-459c-a390-36c2ff403ba6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:15:31.140991 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-398211a3-5251-459c-a390-36c2ff403ba6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:15:31.141847 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-398211a3-5251-459c-a390-36c2ff403ba6 admin admin] PATH_INFO: `/v3/projects/b0e6712e5be748ac987e09af2f3422ad` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:15:31.143076 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-398211a3-5251-459c-a390-36c2ff403ba6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b0e6712e5be748ac987e09af2f3422ad)` {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 16:15:31.145501 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-398211a3-5251-459c-a390-36c2ff403ba6 admin admin] RBAC: Authorization granted {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 16:15:31.349217 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.notifications [None req-398211a3-5251-459c-a390-36c2ff403ba6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b0e6712e5be748ac987e09af2f3422ad', 'request_id': 'req-398211a3-5251-459c-a390-36c2ff403ba6'} {{(pid=81767) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 16:15:31.361557 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.notifications [None req-398211a3-5251-459c-a390-36c2ff403ba6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b0e6712e5be748ac987e09af2f3422ad', 'request_id': 'req-398211a3-5251-459c-a390-36c2ff403ba6'} {{(pid=81767) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 16:15:31.442419 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.notifications [None req-398211a3-5251-459c-a390-36c2ff403ba6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b0e6712e5be748ac987e09af2f3422ad', 'request_id': 'req-398211a3-5251-459c-a390-36c2ff403ba6'} {{(pid=81767) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 16:15:31.456470 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.notifications [None req-398211a3-5251-459c-a390-36c2ff403ba6 admin admin] The token cache is being invalidate because project b0e6712e5be748ac987e09af2f3422ad was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81767) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 22 16:15:31.457030 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.notifications [None req-398211a3-5251-459c-a390-36c2ff403ba6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81767) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 16:15:31.457710 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.notifications [None req-398211a3-5251-459c-a390-36c2ff403ba6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81767) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 16:15:31.460228 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 1370/2785] 10.222.0.15 () {58 vars in 1233 bytes} [Mon Aug 22 16:15:31 2022] DELETE /identity/v3/projects/b0e6712e5be748ac987e09af2f3422ad => generated 0 bytes in 358 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 16:15:31.465115 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e94981-c5e6-468e-a791-9a029cdedbdc None None] REQUEST_METHOD: `POST` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:15:31.465631 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e94981-c5e6-468e-a791-9a029cdedbdc None None] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:15:31.465871 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e94981-c5e6-468e-a791-9a029cdedbdc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:15:31.489302 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.auth.core [None req-e0e94981-c5e6-468e-a791-9a029cdedbdc None None] MFA Rules not processed for user `6b2199ae0fb0480aa917ada391f5ec54`. Rule list: `[]` (Enabled: `True`). {{(pid=81768) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 16:15:31.491221 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-e0e94981-c5e6-468e-a791-9a029cdedbdc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 16:15:31.577843 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 1416/2786] 10.222.0.15 () {58 vars in 958 bytes} [Mon Aug 22 16:15:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 16:15:31.586441 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa117dab-86dc-4aa1-a9f2-aa2ad02beeda None None] REQUEST_METHOD: `POST` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:15:31.587166 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa117dab-86dc-4aa1-a9f2-aa2ad02beeda None None] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:15:31.587697 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa117dab-86dc-4aa1-a9f2-aa2ad02beeda None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:15:31.613283 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.auth.core [None req-aa117dab-86dc-4aa1-a9f2-aa2ad02beeda None None] MFA Rules not processed for user `6b2199ae0fb0480aa917ada391f5ec54`. Rule list: `[]` (Enabled: `True`). {{(pid=81767) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 16:15:31.614936 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.fernet_utils [None req-aa117dab-86dc-4aa1-a9f2-aa2ad02beeda None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81767) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 16:15:31.623674 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 1371/2787] 10.222.0.15 () {58 vars in 958 bytes} [Mon Aug 22 16:15:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 16:15:31.628444 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0e94981-c5e6-468e-a791-9a029cdedbdc None None] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 16:15:31.639925 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0e94981-c5e6-468e-a791-9a029cdedbdc None None] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 16:15:31.645744 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3126973a-0340-40c9-afa7-a9219fabb105 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a98b1e4431940ab8d1fc95df2858a6c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 16:15:31.646564 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3126973a-0340-40c9-afa7-a9219fabb105 admin admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:15:31.646692 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3126973a-0340-40c9-afa7-a9219fabb105 admin admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:15:31.646898 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3126973a-0340-40c9-afa7-a9219fabb105 admin admin] PATH_INFO: `/v3/domains` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:15:31.647628 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3126973a-0340-40c9-afa7-a9219fabb105 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81768) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 16:15:31.647999 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3126973a-0340-40c9-afa7-a9219fabb105 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 16:15:31.649543 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3126973a-0340-40c9-afa7-a9219fabb105 admin admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 16:15:31.668742 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 1417/2788] 10.222.0.15 () {58 vars in 1153 bytes} [Mon Aug 22 16:15:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 22 16:15:31.671089 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa117dab-86dc-4aa1-a9f2-aa2ad02beeda None None] Authenticating user token {{(pid=81767) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 16:15:31.684780 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa117dab-86dc-4aa1-a9f2-aa2ad02beeda None None] Validating token access rules against request {{(pid=81767) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 16:15:31.690549 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5858e6e-9603-4eb9-809b-2c3ce5c7280e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a98b1e4431940ab8d1fc95df2858a6c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81767) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 16:15:31.691177 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5858e6e-9603-4eb9-809b-2c3ce5c7280e admin admin] REQUEST_METHOD: `POST` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:15:31.691491 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5858e6e-9603-4eb9-809b-2c3ce5c7280e admin admin] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:15:31.691765 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5858e6e-9603-4eb9-809b-2c3ce5c7280e admin admin] PATH_INFO: `/v3/projects` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:15:31.692618 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5858e6e-9603-4eb9-809b-2c3ce5c7280e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 16:15:31.695004 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5858e6e-9603-4eb9-809b-2c3ce5c7280e admin admin] RBAC: Authorization granted {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 16:15:31.814601 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 1372/2789] 10.222.0.15 () {60 vars in 1153 bytes} [Mon Aug 22 16:15:31 2022] POST /identity/v3/projects => generated 373 bytes in 144 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 22 16:15:31.819529 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3126973a-0340-40c9-afa7-a9219fabb105 admin admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 16:15:31.831910 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3126973a-0340-40c9-afa7-a9219fabb105 admin admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 16:15:31.916668 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e8fb262-7755-43d1-859e-3b1c6a513a2c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a98b1e4431940ab8d1fc95df2858a6c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 16:15:31.917291 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e8fb262-7755-43d1-859e-3b1c6a513a2c admin admin] REQUEST_METHOD: `POST` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:15:31.927665 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e8fb262-7755-43d1-859e-3b1c6a513a2c admin admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:15:31.927993 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e8fb262-7755-43d1-859e-3b1c6a513a2c admin admin] PATH_INFO: `/v3/users` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:15:31.928690 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e8fb262-7755-43d1-859e-3b1c6a513a2c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 16:15:31.930640 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e8fb262-7755-43d1-859e-3b1c6a513a2c admin admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 16:15:31.968237 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:INSERT=3,keystone:DELETE=3,keystone:SELECT=94 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 16:15:32.071855 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 1418/2790] 10.222.0.15 () {60 vars in 1144 bytes} [Mon Aug 22 16:15:31 2022] POST /identity/v3/users => generated 287 bytes in 244 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 22 16:15:32.072137 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5858e6e-9603-4eb9-809b-2c3ce5c7280e admin admin] Authenticating user token {{(pid=81767) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 16:15:32.079124 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5858e6e-9603-4eb9-809b-2c3ce5c7280e admin admin] Validating token access rules against request {{(pid=81767) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 16:15:32.096731 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a767feae-5af1-4aa2-b876-27d563600698 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a98b1e4431940ab8d1fc95df2858a6c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81767) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 16:15:32.097429 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a767feae-5af1-4aa2-b876-27d563600698 admin admin] REQUEST_METHOD: `GET` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:15:32.097705 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a767feae-5af1-4aa2-b876-27d563600698 admin admin] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:15:32.097978 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a767feae-5af1-4aa2-b876-27d563600698 admin admin] PATH_INFO: `/v3/roles` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:15:32.098680 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a767feae-5af1-4aa2-b876-27d563600698 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 16:15:32.100444 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a767feae-5af1-4aa2-b876-27d563600698 admin admin] RBAC: Authorization granted {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 16:15:32.121769 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 1373/2791] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Aug 22 16:15:32 2022] GET /identity/v3/roles => generated 1420 bytes in 58 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 22 16:15:32.134890 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e8fb262-7755-43d1-859e-3b1c6a513a2c admin admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 16:15:32.140026 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e8fb262-7755-43d1-859e-3b1c6a513a2c admin admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 16:15:32.153488 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fbbbf6b-dcab-4232-bc45-2c34d26fba41 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a98b1e4431940ab8d1fc95df2858a6c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 16:15:32.154508 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fbbbf6b-dcab-4232-bc45-2c34d26fba41 admin admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:15:32.154964 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fbbbf6b-dcab-4232-bc45-2c34d26fba41 admin admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:15:32.155264 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fbbbf6b-dcab-4232-bc45-2c34d26fba41 admin admin] PATH_INFO: `/v3/roles` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:15:32.158426 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fbbbf6b-dcab-4232-bc45-2c34d26fba41 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 16:15:32.161108 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fbbbf6b-dcab-4232-bc45-2c34d26fba41 admin admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 16:15:32.183253 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 1419/2792] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Aug 22 16:15:32 2022] GET /identity/v3/roles => generated 1420 bytes in 59 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 22 16:15:32.189215 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a767feae-5af1-4aa2-b876-27d563600698 admin admin] Authenticating user token {{(pid=81767) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 16:15:32.209663 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a767feae-5af1-4aa2-b876-27d563600698 admin admin] Validating token access rules against request {{(pid=81767) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 16:15:32.218405 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf7050e6-c8ac-49de-b40a-3ca3d354d797 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a98b1e4431940ab8d1fc95df2858a6c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81767) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 16:15:32.221099 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf7050e6-c8ac-49de-b40a-3ca3d354d797 admin admin] REQUEST_METHOD: `PUT` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:15:32.221387 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf7050e6-c8ac-49de-b40a-3ca3d354d797 admin admin] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:15:32.222490 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf7050e6-c8ac-49de-b40a-3ca3d354d797 admin admin] PATH_INFO: `/v3/projects/8edb7b480ffc4f2aac495bbf7873cd7a/users/1b1be8566e844190bc4052eef9693f67/roles/e6fc1ee3fb974125ba9b4fe55b12b45e` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:15:32.265237 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf7050e6-c8ac-49de-b40a-3ca3d354d797 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8edb7b480ffc4f2aac495bbf7873cd7a, user_id=1b1be8566e844190bc4052eef9693f67, role_id=e6fc1ee3fb974125ba9b4fe55b12b45e)` {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 16:15:32.270718 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf7050e6-c8ac-49de-b40a-3ca3d354d797 admin admin] RBAC: Authorization granted {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 16:15:32.375908 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54dff837-a132-4e23-835d-ee370a527e19 None None] REQUEST_METHOD: `POST` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:15:32.376535 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54dff837-a132-4e23-835d-ee370a527e19 None None] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:15:32.377000 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54dff837-a132-4e23-835d-ee370a527e19 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:15:32.384560 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application [None req-54dff837-a132-4e23-835d-ee370a527e19 None None] Expecting to find domain in project. The server could not comply with the request since it is either malformed or otherwise incorrect. The client is assumed to be in error.: keystone.exception.ValidationError: Expecting to find domain in project. The server could not comply with the request since it is either malformed or otherwise incorrect. The client is assumed to be in error. Aug 22 16:15:32.384560 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 16:15:32.384560 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 16:15:32.384560 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 16:15:32.384560 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 16:15:32.384560 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 16:15:32.384560 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 16:15:32.384560 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 16:15:32.384560 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 16:15:32.384560 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 16:15:32.384560 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 16:15:32.384560 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 16:15:32.384560 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/server/flask/common.py", line 1064, in wrapper Aug 22 16:15:32.388679 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return f(*args, **kwargs) Aug 22 16:15:32.388679 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/auth.py", line 315, in post Aug 22 16:15:32.388679 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application token = authentication.authenticate_for_token(auth_data) Aug 22 16:15:32.388679 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/_shared/authentication.py", line 182, in authenticate_for_token Aug 22 16:15:32.388679 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application auth_info = core.AuthInfo.create(auth=auth) Aug 22 16:15:32.388679 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/auth/core.py", line 126, in create Aug 22 16:15:32.388679 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application auth_info._validate_and_normalize_auth_data(scope_only) Aug 22 16:15:32.388679 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/auth/core.py", line 340, in _validate_and_normalize_auth_data Aug 22 16:15:32.388679 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application self._validate_and_normalize_scope_data() Aug 22 16:15:32.388679 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/auth/core.py", line 290, in _validate_and_normalize_scope_data Aug 22 16:15:32.388679 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application project_ref = self._lookup_project(self.auth['scope']['project']) Aug 22 16:15:32.388679 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/auth/core.py", line 194, in _lookup_project Aug 22 16:15:32.388679 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application raise exception.ValidationError(attribute='domain', Aug 22 16:15:32.388679 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application keystone.exception.ValidationError: Expecting to find domain in project. The server could not comply with the request since it is either malformed or otherwise incorrect. The client is assumed to be in error. Aug 22 16:15:32.388679 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application Aug 22 16:15:32.389818 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 1420/2793] 10.222.0.15 () {58 vars in 1009 bytes} [Mon Aug 22 16:15:32 2022] POST /identity/v3/auth/tokens => generated 230 bytes in 11 msecs (HTTP/1.1 400) 5 headers in 186 bytes (2 switches on core 0) Aug 22 16:15:32.420568 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 1374/2794] 10.222.0.15 () {60 vars in 1483 bytes} [Mon Aug 22 16:15:32 2022] PUT /identity/v3/projects/8edb7b480ffc4f2aac495bbf7873cd7a/users/1b1be8566e844190bc4052eef9693f67/roles/e6fc1ee3fb974125ba9b4fe55b12b45e => generated 0 bytes in 233 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 16:15:32.432779 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22b028d1-e0a7-47c9-8549-04a5a95e9afa None None] REQUEST_METHOD: `POST` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:15:32.433056 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22b028d1-e0a7-47c9-8549-04a5a95e9afa None None] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:15:32.433324 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22b028d1-e0a7-47c9-8549-04a5a95e9afa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:15:32.465274 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.auth.core [None req-22b028d1-e0a7-47c9-8549-04a5a95e9afa None None] MFA Rules not processed for user `1b1be8566e844190bc4052eef9693f67`. Rule list: `[]` (Enabled: `True`). {{(pid=81768) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 16:15:32.466514 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-22b028d1-e0a7-47c9-8549-04a5a95e9afa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 16:15:32.555421 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 1421/2795] 10.222.0.15 () {58 vars in 958 bytes} [Mon Aug 22 16:15:32 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 16:15:32.566831 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf7050e6-c8ac-49de-b40a-3ca3d354d797 admin admin] Authenticating user token {{(pid=81767) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 16:15:32.568852 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.fernet_utils [None req-bf7050e6-c8ac-49de-b40a-3ca3d354d797 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81767) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 16:15:32.658296 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG dbcounter [-] [81767] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81767) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 16:15:32.684304 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf7050e6-c8ac-49de-b40a-3ca3d354d797 admin admin] Validating token access rules against request {{(pid=81767) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 16:15:32.697744 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b3a8457-c354-489e-bfdb-3dfb137fb8b1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79706d480f864998b9f8594be407ceab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4ee0413f9ef4d4fb1f95a660babe090', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81767) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 16:15:32.698391 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b3a8457-c354-489e-bfdb-3dfb137fb8b1 service cinder] REQUEST_METHOD: `GET` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:15:32.698684 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b3a8457-c354-489e-bfdb-3dfb137fb8b1 service cinder] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:15:32.700023 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b3a8457-c354-489e-bfdb-3dfb137fb8b1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:15:32.724665 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b3a8457-c354-489e-bfdb-3dfb137fb8b1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 16:15:32.726155 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b3a8457-c354-489e-bfdb-3dfb137fb8b1 service cinder] RBAC: Authorization granted {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 16:15:32.739837 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 1375/2796] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Aug 22 16:15:32 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 173 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 16:15:32.782709 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22b028d1-e0a7-47c9-8549-04a5a95e9afa None None] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 16:15:32.870534 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22b028d1-e0a7-47c9-8549-04a5a95e9afa None None] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 16:15:32.875531 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17686f95-a1ce-45cd-87b9-0426bd820884 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a98b1e4431940ab8d1fc95df2858a6c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 16:15:32.876374 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17686f95-a1ce-45cd-87b9-0426bd820884 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:15:32.876705 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17686f95-a1ce-45cd-87b9-0426bd820884 admin admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:15:32.876984 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17686f95-a1ce-45cd-87b9-0426bd820884 admin admin] PATH_INFO: `/v3/users/1b1be8566e844190bc4052eef9693f67` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:15:32.878331 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17686f95-a1ce-45cd-87b9-0426bd820884 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1b1be8566e844190bc4052eef9693f67)` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 16:15:32.880706 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17686f95-a1ce-45cd-87b9-0426bd820884 admin admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 16:15:33.122834 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-17686f95-a1ce-45cd-87b9-0426bd820884 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1b1be8566e844190bc4052eef9693f67'} {{(pid=81768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 16:15:33.132077 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-17686f95-a1ce-45cd-87b9-0426bd820884 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1b1be8566e844190bc4052eef9693f67'} {{(pid=81768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 16:15:33.132077 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-17686f95-a1ce-45cd-87b9-0426bd820884 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1b1be8566e844190bc4052eef9693f67'} {{(pid=81768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 16:15:33.146003 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-17686f95-a1ce-45cd-87b9-0426bd820884 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1b1be8566e844190bc4052eef9693f67'} {{(pid=81768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 16:15:33.244051 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-17686f95-a1ce-45cd-87b9-0426bd820884 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1b1be8566e844190bc4052eef9693f67'} {{(pid=81768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 16:15:33.256170 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:INSERT=2,keystone:SELECT=89,keystone:DELETE=9 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 16:15:33.285150 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38a95909-a795-45c1-91b0-615487121db3 None None] REQUEST_METHOD: `POST` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:15:33.285150 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38a95909-a795-45c1-91b0-615487121db3 None None] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:15:33.285150 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38a95909-a795-45c1-91b0-615487121db3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:15:33.286574 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 1422/2797] 10.222.0.15 () {58 vars in 1224 bytes} [Mon Aug 22 16:15:32 2022] DELETE /identity/v3/users/1b1be8566e844190bc4052eef9693f67 => generated 0 bytes in 496 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 16:15:33.307892 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.auth.core [None req-38a95909-a795-45c1-91b0-615487121db3 None None] MFA Rules not processed for user `6b2199ae0fb0480aa917ada391f5ec54`. Rule list: `[]` (Enabled: `True`). {{(pid=81767) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 16:15:33.309585 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.fernet_utils [None req-38a95909-a795-45c1-91b0-615487121db3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81767) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 16:15:33.407980 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 1376/2798] 10.222.0.15 () {58 vars in 958 bytes} [Mon Aug 22 16:15:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 16:15:33.436220 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17686f95-a1ce-45cd-87b9-0426bd820884 admin admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 16:15:33.438200 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-17686f95-a1ce-45cd-87b9-0426bd820884 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 16:15:33.540313 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17686f95-a1ce-45cd-87b9-0426bd820884 admin admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 16:15:33.546390 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c621d15-bff7-42ae-aa33-476e5cfdc76f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '234687adad594f1aa45775a565ffbdc7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4ee0413f9ef4d4fb1f95a660babe090', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 16:15:33.546390 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c621d15-bff7-42ae-aa33-476e5cfdc76f service neutron] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:15:33.546711 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c621d15-bff7-42ae-aa33-476e5cfdc76f service neutron] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:15:33.547029 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c621d15-bff7-42ae-aa33-476e5cfdc76f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:15:33.560324 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c621d15-bff7-42ae-aa33-476e5cfdc76f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 16:15:33.560324 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c621d15-bff7-42ae-aa33-476e5cfdc76f service neutron] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 16:15:33.581247 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 1423/2799] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Aug 22 16:15:33 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 147 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 16:15:33.913468 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc1f5fbe-7fad-4ce7-9b67-d5608751efa9 None None] REQUEST_METHOD: `POST` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:15:33.913931 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc1f5fbe-7fad-4ce7-9b67-d5608751efa9 None None] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:15:33.914017 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc1f5fbe-7fad-4ce7-9b67-d5608751efa9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:15:33.970762 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.auth.core [None req-cc1f5fbe-7fad-4ce7-9b67-d5608751efa9 None None] MFA Rules not processed for user `6413010933f6416a8d2e3ee82c1946e2`. Rule list: `[]` (Enabled: `True`). {{(pid=81767) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 16:15:33.973248 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.fernet_utils [None req-cc1f5fbe-7fad-4ce7-9b67-d5608751efa9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81767) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 16:15:34.086357 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 1377/2800] 10.222.0.15 () {58 vars in 958 bytes} [Mon Aug 22 16:15:33 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 22 16:15:34.091173 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c621d15-bff7-42ae-aa33-476e5cfdc76f service neutron] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 16:15:34.092725 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-4c621d15-bff7-42ae-aa33-476e5cfdc76f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 16:15:34.111024 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c621d15-bff7-42ae-aa33-476e5cfdc76f service neutron] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 16:15:34.116571 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-170f977a-9bb9-4a68-b3aa-c73eeef3f027 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a98b1e4431940ab8d1fc95df2858a6c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 16:15:34.117623 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-170f977a-9bb9-4a68-b3aa-c73eeef3f027 admin admin] REQUEST_METHOD: `POST` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:15:34.118150 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-170f977a-9bb9-4a68-b3aa-c73eeef3f027 admin admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:15:34.118594 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-170f977a-9bb9-4a68-b3aa-c73eeef3f027 admin admin] PATH_INFO: `/v3/projects` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:15:34.119462 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-170f977a-9bb9-4a68-b3aa-c73eeef3f027 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 16:15:34.122444 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-170f977a-9bb9-4a68-b3aa-c73eeef3f027 admin admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 16:15:34.198857 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc1f5fbe-7fad-4ce7-9b67-d5608751efa9 None None] Authenticating user token {{(pid=81767) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 16:15:34.200262 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.fernet_utils [None req-cc1f5fbe-7fad-4ce7-9b67-d5608751efa9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81767) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 16:15:34.242833 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc1f5fbe-7fad-4ce7-9b67-d5608751efa9 None None] Validating token access rules against request {{(pid=81767) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 16:15:34.243029 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 1424/2801] 10.222.0.15 () {60 vars in 1153 bytes} [Mon Aug 22 16:15:34 2022] POST /identity/v3/projects => generated 385 bytes in 152 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 22 16:15:34.243491 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-170f977a-9bb9-4a68-b3aa-c73eeef3f027 admin admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 16:15:34.339981 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2681cad7-ff80-47f8-96ea-5d12cf8c7f7f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a98b1e4431940ab8d1fc95df2858a6c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81767) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 16:15:34.601610 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2681cad7-ff80-47f8-96ea-5d12cf8c7f7f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:15:34.601610 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2681cad7-ff80-47f8-96ea-5d12cf8c7f7f admin admin] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:15:34.601610 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2681cad7-ff80-47f8-96ea-5d12cf8c7f7f admin admin] PATH_INFO: `/v3/projects/8edb7b480ffc4f2aac495bbf7873cd7a` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:15:34.612928 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-170f977a-9bb9-4a68-b3aa-c73eeef3f027 admin admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 16:15:34.613379 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2681cad7-ff80-47f8-96ea-5d12cf8c7f7f admin admin] RBAC: Authorizing `identity:delete_project(project_id=8edb7b480ffc4f2aac495bbf7873cd7a)` {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 16:15:34.613379 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2681cad7-ff80-47f8-96ea-5d12cf8c7f7f admin admin] RBAC: Authorization granted {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 16:15:34.613570 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-569bcea4-5d78-4672-a430-d9e087fac23e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a98b1e4431940ab8d1fc95df2858a6c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 16:15:34.613570 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-569bcea4-5d78-4672-a430-d9e087fac23e admin admin] REQUEST_METHOD: `POST` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:15:34.613570 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-569bcea4-5d78-4672-a430-d9e087fac23e admin admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:15:34.613570 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-569bcea4-5d78-4672-a430-d9e087fac23e admin admin] PATH_INFO: `/v3/users` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:15:34.613936 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-569bcea4-5d78-4672-a430-d9e087fac23e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 16:15:34.613936 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-569bcea4-5d78-4672-a430-d9e087fac23e admin admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 16:15:34.614115 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG dbcounter [-] [81767] Writing DB stats keystone:SELECT=100 {{(pid=81767) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 16:15:34.651397 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 1425/2802] 10.222.0.15 () {60 vars in 1144 bytes} [Mon Aug 22 16:15:34 2022] POST /identity/v3/users => generated 299 bytes in 407 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 22 16:15:34.654242 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-569bcea4-5d78-4672-a430-d9e087fac23e admin admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 16:15:34.699228 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-569bcea4-5d78-4672-a430-d9e087fac23e admin admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 16:15:34.699498 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.notifications [None req-2681cad7-ff80-47f8-96ea-5d12cf8c7f7f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8edb7b480ffc4f2aac495bbf7873cd7a', 'request_id': 'req-2681cad7-ff80-47f8-96ea-5d12cf8c7f7f'} {{(pid=81767) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 16:15:34.699498 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.notifications [None req-2681cad7-ff80-47f8-96ea-5d12cf8c7f7f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8edb7b480ffc4f2aac495bbf7873cd7a', 'request_id': 'req-2681cad7-ff80-47f8-96ea-5d12cf8c7f7f'} {{(pid=81767) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 16:15:34.772056 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.notifications [None req-2681cad7-ff80-47f8-96ea-5d12cf8c7f7f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8edb7b480ffc4f2aac495bbf7873cd7a', 'request_id': 'req-2681cad7-ff80-47f8-96ea-5d12cf8c7f7f'} {{(pid=81767) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 16:15:34.786383 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff8c46b0-5831-47fa-9020-144e2c0049b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a98b1e4431940ab8d1fc95df2858a6c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 16:15:34.786383 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff8c46b0-5831-47fa-9020-144e2c0049b6 admin admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:15:34.786383 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff8c46b0-5831-47fa-9020-144e2c0049b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:15:34.786383 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff8c46b0-5831-47fa-9020-144e2c0049b6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:15:34.786937 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.notifications [None req-2681cad7-ff80-47f8-96ea-5d12cf8c7f7f admin admin] The token cache is being invalidate because project 8edb7b480ffc4f2aac495bbf7873cd7a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81767) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 22 16:15:34.787083 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff8c46b0-5831-47fa-9020-144e2c0049b6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 16:15:34.787419 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.notifications [None req-2681cad7-ff80-47f8-96ea-5d12cf8c7f7f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81767) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 16:15:34.788271 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.notifications [None req-2681cad7-ff80-47f8-96ea-5d12cf8c7f7f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81767) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 16:15:34.790759 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff8c46b0-5831-47fa-9020-144e2c0049b6 admin admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 16:15:34.792523 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=95,keystone:DELETE=1,keystone:INSERT=4 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 16:15:34.800771 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 1378/2803] 10.222.0.15 () {58 vars in 1233 bytes} [Mon Aug 22 16:15:34 2022] DELETE /identity/v3/projects/8edb7b480ffc4f2aac495bbf7873cd7a => generated 0 bytes in 602 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 16:15:34.803154 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc36ecf2-6e2a-4e24-a410-a8be89ef135a None None] REQUEST_METHOD: `POST` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:15:34.803535 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc36ecf2-6e2a-4e24-a410-a8be89ef135a None None] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:15:34.804242 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc36ecf2-6e2a-4e24-a410-a8be89ef135a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:15:34.809430 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 1426/2804] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Aug 22 16:15:34 2022] GET /identity/v3/roles => generated 1420 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 22 16:15:34.811318 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff8c46b0-5831-47fa-9020-144e2c0049b6 admin admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 16:15:34.815554 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-ff8c46b0-5831-47fa-9020-144e2c0049b6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 16:15:34.844880 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.auth.core [None req-cc36ecf2-6e2a-4e24-a410-a8be89ef135a None None] MFA Rules not processed for user `6b2199ae0fb0480aa917ada391f5ec54`. Rule list: `[]` (Enabled: `True`). {{(pid=81767) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 16:15:34.845737 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff8c46b0-5831-47fa-9020-144e2c0049b6 admin admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 16:15:34.848428 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.fernet_utils [None req-cc36ecf2-6e2a-4e24-a410-a8be89ef135a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81767) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 16:15:34.855210 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ad40128-1a0b-488c-972c-3af49247c98d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a98b1e4431940ab8d1fc95df2858a6c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 16:15:34.860904 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ad40128-1a0b-488c-972c-3af49247c98d admin admin] REQUEST_METHOD: `PUT` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:15:34.860904 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ad40128-1a0b-488c-972c-3af49247c98d admin admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:15:34.860904 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ad40128-1a0b-488c-972c-3af49247c98d admin admin] PATH_INFO: `/v3/projects/f8def62fc2424a13a013f65f44bd4f5f/users/a73acaa341eb4f538e81c4aa6b3034cf/roles/9d9848cfd22e4b089db443c350e56670` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:15:34.879150 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 1379/2805] 10.222.0.15 () {58 vars in 958 bytes} [Mon Aug 22 16:15:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 71 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 16:15:34.886038 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ad40128-1a0b-488c-972c-3af49247c98d admin admin] RBAC: Authorizing `identity:create_grant(project_id=f8def62fc2424a13a013f65f44bd4f5f, user_id=a73acaa341eb4f538e81c4aa6b3034cf, role_id=9d9848cfd22e4b089db443c350e56670)` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 16:15:34.889220 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ad40128-1a0b-488c-972c-3af49247c98d admin admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 16:15:34.898893 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-120f7b5b-e3a4-4bba-befc-05aaccd32e0a None None] REQUEST_METHOD: `POST` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:15:34.899443 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-120f7b5b-e3a4-4bba-befc-05aaccd32e0a None None] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:15:34.901150 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-120f7b5b-e3a4-4bba-befc-05aaccd32e0a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:15:34.957156 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.auth.core [None req-120f7b5b-e3a4-4bba-befc-05aaccd32e0a None None] MFA Rules not processed for user `6b2199ae0fb0480aa917ada391f5ec54`. Rule list: `[]` (Enabled: `True`). {{(pid=81767) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 16:15:34.958523 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.fernet_utils [None req-120f7b5b-e3a4-4bba-befc-05aaccd32e0a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81767) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 16:15:34.984418 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 1380/2806] 10.222.0.15 () {58 vars in 958 bytes} [Mon Aug 22 16:15:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 16:15:34.985391 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-120f7b5b-e3a4-4bba-befc-05aaccd32e0a None None] Authenticating user token {{(pid=81767) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 16:15:34.999128 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-120f7b5b-e3a4-4bba-befc-05aaccd32e0a None None] Validating token access rules against request {{(pid=81767) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 16:15:35.004640 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27c12184-a501-45ff-b1d9-6081aa7a286a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a98b1e4431940ab8d1fc95df2858a6c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81767) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 16:15:35.005259 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c12184-a501-45ff-b1d9-6081aa7a286a admin admin] REQUEST_METHOD: `GET` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:15:35.005604 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c12184-a501-45ff-b1d9-6081aa7a286a admin admin] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:15:35.005786 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c12184-a501-45ff-b1d9-6081aa7a286a admin admin] PATH_INFO: `/v3/domains` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:15:35.006434 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27c12184-a501-45ff-b1d9-6081aa7a286a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81767) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 16:15:35.006736 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27c12184-a501-45ff-b1d9-6081aa7a286a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 16:15:35.008384 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27c12184-a501-45ff-b1d9-6081aa7a286a admin admin] RBAC: Authorization granted {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 16:15:35.013578 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 1427/2807] 10.222.0.15 () {60 vars in 1483 bytes} [Mon Aug 22 16:15:34 2022] PUT /identity/v3/projects/f8def62fc2424a13a013f65f44bd4f5f/users/a73acaa341eb4f538e81c4aa6b3034cf/roles/9d9848cfd22e4b089db443c350e56670 => generated 0 bytes in 201 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 22 16:15:35.021706 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ad40128-1a0b-488c-972c-3af49247c98d admin admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 16:15:35.027335 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 1381/2808] 10.222.0.15 () {58 vars in 1153 bytes} [Mon Aug 22 16:15:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 16:15:35.028696 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27c12184-a501-45ff-b1d9-6081aa7a286a admin admin] Authenticating user token {{(pid=81767) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 16:15:35.040173 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27c12184-a501-45ff-b1d9-6081aa7a286a admin admin] Validating token access rules against request {{(pid=81767) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 16:15:35.124366 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ad40128-1a0b-488c-972c-3af49247c98d admin admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 16:15:35.131623 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a993a43d-1714-4f14-8db9-8fa009e5e6f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a98b1e4431940ab8d1fc95df2858a6c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 16:15:35.132595 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a993a43d-1714-4f14-8db9-8fa009e5e6f0 admin admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:15:35.133235 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a993a43d-1714-4f14-8db9-8fa009e5e6f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:15:35.133728 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a993a43d-1714-4f14-8db9-8fa009e5e6f0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:15:35.134613 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a993a43d-1714-4f14-8db9-8fa009e5e6f0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 16:15:35.136411 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a993a43d-1714-4f14-8db9-8fa009e5e6f0 admin admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 16:15:35.137253 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbf74ac7-c651-4781-b1c3-6451da1e434f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a98b1e4431940ab8d1fc95df2858a6c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81767) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 16:15:35.138083 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf74ac7-c651-4781-b1c3-6451da1e434f admin admin] REQUEST_METHOD: `POST` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:15:35.138362 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf74ac7-c651-4781-b1c3-6451da1e434f admin admin] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:15:35.138630 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf74ac7-c651-4781-b1c3-6451da1e434f admin admin] PATH_INFO: `/v3/projects` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:15:35.139676 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbf74ac7-c651-4781-b1c3-6451da1e434f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 16:15:35.142278 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbf74ac7-c651-4781-b1c3-6451da1e434f admin admin] RBAC: Authorization granted {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 16:15:35.151917 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 1428/2809] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Aug 22 16:15:35 2022] GET /identity/v3/roles => generated 1420 bytes in 131 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 22 16:15:35.157994 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a993a43d-1714-4f14-8db9-8fa009e5e6f0 admin admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 16:15:35.176791 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a993a43d-1714-4f14-8db9-8fa009e5e6f0 admin admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 16:15:35.184788 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d62a247-ae6e-4ea1-b7aa-fba2f04b8672 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a98b1e4431940ab8d1fc95df2858a6c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 16:15:35.185457 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d62a247-ae6e-4ea1-b7aa-fba2f04b8672 admin admin] REQUEST_METHOD: `PUT` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:15:35.191465 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d62a247-ae6e-4ea1-b7aa-fba2f04b8672 admin admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:15:35.191465 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d62a247-ae6e-4ea1-b7aa-fba2f04b8672 admin admin] PATH_INFO: `/v3/projects/f8def62fc2424a13a013f65f44bd4f5f/users/a73acaa341eb4f538e81c4aa6b3034cf/roles/9d9848cfd22e4b089db443c350e56670` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:15:35.193139 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d62a247-ae6e-4ea1-b7aa-fba2f04b8672 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f8def62fc2424a13a013f65f44bd4f5f, user_id=a73acaa341eb4f538e81c4aa6b3034cf, role_id=9d9848cfd22e4b089db443c350e56670)` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 16:15:35.196713 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d62a247-ae6e-4ea1-b7aa-fba2f04b8672 admin admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 16:15:35.209783 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 1429/2810] 10.222.0.15 () {60 vars in 1483 bytes} [Mon Aug 22 16:15:35 2022] PUT /identity/v3/projects/f8def62fc2424a13a013f65f44bd4f5f/users/a73acaa341eb4f538e81c4aa6b3034cf/roles/9d9848cfd22e4b089db443c350e56670 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 16:15:35.250052 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 1382/2811] 10.222.0.15 () {60 vars in 1153 bytes} [Mon Aug 22 16:15:35 2022] POST /identity/v3/projects => generated 373 bytes in 222 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 22 16:15:35.254080 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d62a247-ae6e-4ea1-b7aa-fba2f04b8672 admin admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 16:15:35.364674 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d62a247-ae6e-4ea1-b7aa-fba2f04b8672 admin admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 16:15:35.473011 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cdec82e-606b-45e7-aa70-798da588a83d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a98b1e4431940ab8d1fc95df2858a6c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 16:15:35.474150 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cdec82e-606b-45e7-aa70-798da588a83d admin admin] REQUEST_METHOD: `POST` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:15:35.475375 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cdec82e-606b-45e7-aa70-798da588a83d admin admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:15:35.475667 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cdec82e-606b-45e7-aa70-798da588a83d admin admin] PATH_INFO: `/v3/users` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:15:35.476513 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cdec82e-606b-45e7-aa70-798da588a83d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 16:15:35.479131 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cdec82e-606b-45e7-aa70-798da588a83d admin admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 16:15:35.494722 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 16:15:35.642426 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 1430/2812] 10.222.0.15 () {60 vars in 1144 bytes} [Mon Aug 22 16:15:35 2022] POST /identity/v3/users => generated 287 bytes in 378 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 22 16:15:35.643074 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbf74ac7-c651-4781-b1c3-6451da1e434f admin admin] Authenticating user token {{(pid=81767) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 16:15:35.658921 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbf74ac7-c651-4781-b1c3-6451da1e434f admin admin] Validating token access rules against request {{(pid=81767) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 16:15:35.670547 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eafb9bd-07e6-481b-aac1-e8c09dc45869 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a98b1e4431940ab8d1fc95df2858a6c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81767) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 16:15:35.671205 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eafb9bd-07e6-481b-aac1-e8c09dc45869 admin admin] REQUEST_METHOD: `GET` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:15:35.671521 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eafb9bd-07e6-481b-aac1-e8c09dc45869 admin admin] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:15:35.671790 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eafb9bd-07e6-481b-aac1-e8c09dc45869 admin admin] PATH_INFO: `/v3/roles` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:15:35.673009 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eafb9bd-07e6-481b-aac1-e8c09dc45869 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 16:15:35.674764 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eafb9bd-07e6-481b-aac1-e8c09dc45869 admin admin] RBAC: Authorization granted {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 16:15:35.694518 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 1383/2813] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Aug 22 16:15:35 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 22 16:15:35.698331 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cdec82e-606b-45e7-aa70-798da588a83d admin admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 16:15:35.710995 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cdec82e-606b-45e7-aa70-798da588a83d admin admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 16:15:35.718038 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f73b7a1-2de9-4234-a5bd-cd84e0dc5eb9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a98b1e4431940ab8d1fc95df2858a6c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 16:15:35.718792 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f73b7a1-2de9-4234-a5bd-cd84e0dc5eb9 admin admin] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:15:35.719230 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f73b7a1-2de9-4234-a5bd-cd84e0dc5eb9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:15:35.719557 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f73b7a1-2de9-4234-a5bd-cd84e0dc5eb9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:15:35.720215 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f73b7a1-2de9-4234-a5bd-cd84e0dc5eb9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 16:15:35.721888 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f73b7a1-2de9-4234-a5bd-cd84e0dc5eb9 admin admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 16:15:35.733398 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 1431/2814] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Aug 22 16:15:35 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 22 16:15:35.737628 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eafb9bd-07e6-481b-aac1-e8c09dc45869 admin admin] Authenticating user token {{(pid=81767) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 16:15:35.749379 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eafb9bd-07e6-481b-aac1-e8c09dc45869 admin admin] Validating token access rules against request {{(pid=81767) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 16:15:35.760408 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f18c86ff-958e-4840-932b-f857321bb24b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a98b1e4431940ab8d1fc95df2858a6c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81767) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 16:15:35.761089 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f18c86ff-958e-4840-932b-f857321bb24b admin admin] REQUEST_METHOD: `PUT` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:15:35.761366 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f18c86ff-958e-4840-932b-f857321bb24b admin admin] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:15:35.762349 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f18c86ff-958e-4840-932b-f857321bb24b admin admin] PATH_INFO: `/v3/projects/58437ee91d73418d9d717997c88ac2e1/users/8630362971bf46efae56179cb5c607d4/roles/e6fc1ee3fb974125ba9b4fe55b12b45e` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:15:35.796904 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f18c86ff-958e-4840-932b-f857321bb24b admin admin] RBAC: Authorizing `identity:create_grant(project_id=58437ee91d73418d9d717997c88ac2e1, user_id=8630362971bf46efae56179cb5c607d4, role_id=e6fc1ee3fb974125ba9b4fe55b12b45e)` {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 16:15:35.801303 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f18c86ff-958e-4840-932b-f857321bb24b admin admin] RBAC: Authorization granted {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 16:15:35.985682 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 1384/2815] 10.222.0.15 () {60 vars in 1483 bytes} [Mon Aug 22 16:15:35 2022] PUT /identity/v3/projects/58437ee91d73418d9d717997c88ac2e1/users/8630362971bf46efae56179cb5c607d4/roles/e6fc1ee3fb974125ba9b4fe55b12b45e => generated 0 bytes in 246 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 16:15:36.005188 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efdc223a-f340-4ddc-9288-09487f4b3f89 None None] REQUEST_METHOD: `POST` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:15:36.005629 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efdc223a-f340-4ddc-9288-09487f4b3f89 None None] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:15:36.005963 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efdc223a-f340-4ddc-9288-09487f4b3f89 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:15:36.047456 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.auth.core [None req-efdc223a-f340-4ddc-9288-09487f4b3f89 None None] MFA Rules not processed for user `8630362971bf46efae56179cb5c607d4`. Rule list: `[]` (Enabled: `True`). {{(pid=81768) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 16:15:36.049091 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-efdc223a-f340-4ddc-9288-09487f4b3f89 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 16:15:36.171655 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 1432/2816] 10.222.0.15 () {58 vars in 958 bytes} [Mon Aug 22 16:15:35 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 16:15:36.185847 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f18c86ff-958e-4840-932b-f857321bb24b admin admin] Authenticating user token {{(pid=81767) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 16:15:36.188696 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.fernet_utils [None req-f18c86ff-958e-4840-932b-f857321bb24b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81767) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 16:15:36.227101 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG dbcounter [-] [81767] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81767) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 16:15:36.265061 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f18c86ff-958e-4840-932b-f857321bb24b admin admin] Validating token access rules against request {{(pid=81767) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 16:15:36.279764 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c97dca6d-47e4-4b79-a500-1d215f0d90a3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '79706d480f864998b9f8594be407ceab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4ee0413f9ef4d4fb1f95a660babe090', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81767) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 16:15:36.281785 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c97dca6d-47e4-4b79-a500-1d215f0d90a3 service cinder] REQUEST_METHOD: `GET` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:15:36.281785 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c97dca6d-47e4-4b79-a500-1d215f0d90a3 service cinder] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:15:36.281785 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c97dca6d-47e4-4b79-a500-1d215f0d90a3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:15:36.297897 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c97dca6d-47e4-4b79-a500-1d215f0d90a3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 16:15:36.299799 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c97dca6d-47e4-4b79-a500-1d215f0d90a3 service cinder] RBAC: Authorization granted {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 16:15:36.326176 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 1385/2817] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Aug 22 16:15:36 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 141 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 16:15:43.941528 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcd463f7-aeea-41df-8dd2-f7dc271a5660 None None] REQUEST_METHOD: `POST` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:15:43.942464 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcd463f7-aeea-41df-8dd2-f7dc271a5660 None None] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:15:43.942934 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcd463f7-aeea-41df-8dd2-f7dc271a5660 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:15:43.976177 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.auth.core [None req-dcd463f7-aeea-41df-8dd2-f7dc271a5660 None None] MFA Rules not processed for user `a73acaa341eb4f538e81c4aa6b3034cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81768) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 16:15:43.977415 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-dcd463f7-aeea-41df-8dd2-f7dc271a5660 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 16:15:44.048797 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 1433/2818] 10.222.0.15 () {58 vars in 958 bytes} [Mon Aug 22 16:15:43 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 16:15:44.063210 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c97dca6d-47e4-4b79-a500-1d215f0d90a3 service cinder] Authenticating user token {{(pid=81767) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 16:15:44.065913 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.fernet_utils [None req-c97dca6d-47e4-4b79-a500-1d215f0d90a3 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81767) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 16:15:44.244256 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c97dca6d-47e4-4b79-a500-1d215f0d90a3 service cinder] Validating token access rules against request {{(pid=81767) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 16:15:44.244256 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-753d9601-f621-4be4-8727-f2f0bee76080 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91f565473c824a15930e2b12863c1ed3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4ee0413f9ef4d4fb1f95a660babe090', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81767) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 16:15:44.244256 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-753d9601-f621-4be4-8727-f2f0bee76080 service glance] REQUEST_METHOD: `GET` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:15:44.244256 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-753d9601-f621-4be4-8727-f2f0bee76080 service glance] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:15:44.244977 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-753d9601-f621-4be4-8727-f2f0bee76080 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:15:44.244977 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.fernet_utils [None req-753d9601-f621-4be4-8727-f2f0bee76080 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81767) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 16:15:44.323525 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-753d9601-f621-4be4-8727-f2f0bee76080 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 16:15:44.331029 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-753d9601-f621-4be4-8727-f2f0bee76080 service glance] RBAC: Authorization granted {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 16:15:44.364189 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 1386/2819] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Aug 22 16:15:44 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 300 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 16:15:44.658689 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cc389c3-0f82-4788-93a0-c2135fc1d316 None None] REQUEST_METHOD: `POST` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:15:44.658968 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cc389c3-0f82-4788-93a0-c2135fc1d316 None None] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:15:44.659203 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cc389c3-0f82-4788-93a0-c2135fc1d316 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:15:44.687586 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.auth.core [None req-2cc389c3-0f82-4788-93a0-c2135fc1d316 None None] MFA Rules not processed for user `058e76354dce412cb78b2875c88f2791`. Rule list: `[]` (Enabled: `True`). {{(pid=81768) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 16:15:44.688814 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-2cc389c3-0f82-4788-93a0-c2135fc1d316 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 16:15:44.778662 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 1434/2820] 10.222.0.15 () {58 vars in 1009 bytes} [Mon Aug 22 16:15:44 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 16:15:44.797537 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-753d9601-f621-4be4-8727-f2f0bee76080 service glance] Authenticating user token {{(pid=81767) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 16:15:44.799203 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.fernet_utils [None req-753d9601-f621-4be4-8727-f2f0bee76080 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81767) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 16:15:44.893635 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-753d9601-f621-4be4-8727-f2f0bee76080 service glance] Validating token access rules against request {{(pid=81767) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 16:15:44.898369 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5d51381-1286-4421-b0db-145d5ab15ac1 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a09d128dee9438283633e51f4834643', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4ee0413f9ef4d4fb1f95a660babe090', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81767) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 16:15:44.899121 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5d51381-1286-4421-b0db-145d5ab15ac1 service swift] REQUEST_METHOD: `GET` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:15:44.899439 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5d51381-1286-4421-b0db-145d5ab15ac1 service swift] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:15:44.899719 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5d51381-1286-4421-b0db-145d5ab15ac1 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:15:44.909741 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5d51381-1286-4421-b0db-145d5ab15ac1 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 16:15:44.911393 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5d51381-1286-4421-b0db-145d5ab15ac1 service swift] RBAC: Authorization granted {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 16:15:44.921635 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 1387/2821] 10.222.0.15 () {60 vars in 1384 bytes} [Mon Aug 22 16:15:44 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 125 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 22 16:15:45.000067 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cc389c3-0f82-4788-93a0-c2135fc1d316 None None] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 16:15:45.003696 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-2cc389c3-0f82-4788-93a0-c2135fc1d316 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 16:15:45.010166 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:INSERT=1,keystone:SELECT=99 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 16:15:45.107959 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cc389c3-0f82-4788-93a0-c2135fc1d316 None None] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 16:15:45.114002 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb63b101-5663-4a66-9eef-c7ca089a1daf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a98b1e4431940ab8d1fc95df2858a6c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 16:15:45.114828 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb63b101-5663-4a66-9eef-c7ca089a1daf admin admin] REQUEST_METHOD: `DELETE` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:15:45.115209 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb63b101-5663-4a66-9eef-c7ca089a1daf admin admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:15:45.115611 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb63b101-5663-4a66-9eef-c7ca089a1daf admin admin] PATH_INFO: `/v3/users/7990d5771e4d4915aa3329d733451d36` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:15:45.116973 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb63b101-5663-4a66-9eef-c7ca089a1daf admin admin] RBAC: Authorizing `identity:delete_user(user_id=7990d5771e4d4915aa3329d733451d36)` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 16:15:45.121549 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb63b101-5663-4a66-9eef-c7ca089a1daf admin admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 16:15:45.418924 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-bb63b101-5663-4a66-9eef-c7ca089a1daf admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7990d5771e4d4915aa3329d733451d36'} {{(pid=81768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 16:15:45.421008 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-bb63b101-5663-4a66-9eef-c7ca089a1daf admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7990d5771e4d4915aa3329d733451d36'} {{(pid=81768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 16:15:45.426463 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-bb63b101-5663-4a66-9eef-c7ca089a1daf admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7990d5771e4d4915aa3329d733451d36'} {{(pid=81768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 16:15:45.451231 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-bb63b101-5663-4a66-9eef-c7ca089a1daf admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7990d5771e4d4915aa3329d733451d36'} {{(pid=81768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 16:15:45.592280 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-bb63b101-5663-4a66-9eef-c7ca089a1daf admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7990d5771e4d4915aa3329d733451d36'} {{(pid=81768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 16:15:45.623468 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 1435/2822] 10.222.0.15 () {58 vars in 1224 bytes} [Mon Aug 22 16:15:44 2022] DELETE /identity/v3/users/7990d5771e4d4915aa3329d733451d36 => generated 0 bytes in 623 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 16:15:46.008712 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5d51381-1286-4421-b0db-145d5ab15ac1 service swift] Authenticating user token {{(pid=81767) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 16:15:46.041877 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5d51381-1286-4421-b0db-145d5ab15ac1 service swift] Validating token access rules against request {{(pid=81767) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 16:15:46.097603 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG dbcounter [-] [81767] Writing DB stats keystone:SELECT=100 {{(pid=81767) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 16:15:46.248436 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba21981a-5ef7-4995-afb6-fd8bec0f9238 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a98b1e4431940ab8d1fc95df2858a6c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81767) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 16:15:46.251132 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba21981a-5ef7-4995-afb6-fd8bec0f9238 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:15:46.252781 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba21981a-5ef7-4995-afb6-fd8bec0f9238 admin admin] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:15:46.253417 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba21981a-5ef7-4995-afb6-fd8bec0f9238 admin admin] PATH_INFO: `/v3/projects/21e5a94ced984deea6e3c0a0f08a994b` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:15:46.255140 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba21981a-5ef7-4995-afb6-fd8bec0f9238 admin admin] RBAC: Authorizing `identity:delete_project(project_id=21e5a94ced984deea6e3c0a0f08a994b)` {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 16:15:46.257752 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba21981a-5ef7-4995-afb6-fd8bec0f9238 admin admin] RBAC: Authorization granted {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 16:15:46.452363 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.notifications [None req-ba21981a-5ef7-4995-afb6-fd8bec0f9238 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '21e5a94ced984deea6e3c0a0f08a994b', 'request_id': 'req-ba21981a-5ef7-4995-afb6-fd8bec0f9238'} {{(pid=81767) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 16:15:46.479221 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.notifications [None req-ba21981a-5ef7-4995-afb6-fd8bec0f9238 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '21e5a94ced984deea6e3c0a0f08a994b', 'request_id': 'req-ba21981a-5ef7-4995-afb6-fd8bec0f9238'} {{(pid=81767) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 16:15:46.549213 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.notifications [None req-ba21981a-5ef7-4995-afb6-fd8bec0f9238 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '21e5a94ced984deea6e3c0a0f08a994b', 'request_id': 'req-ba21981a-5ef7-4995-afb6-fd8bec0f9238'} {{(pid=81767) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 16:15:46.607988 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.notifications [None req-ba21981a-5ef7-4995-afb6-fd8bec0f9238 admin admin] The token cache is being invalidate because project 21e5a94ced984deea6e3c0a0f08a994b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81767) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 22 16:15:46.608277 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.notifications [None req-ba21981a-5ef7-4995-afb6-fd8bec0f9238 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81767) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 16:15:46.611724 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.notifications [None req-ba21981a-5ef7-4995-afb6-fd8bec0f9238 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81767) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 16:15:46.617247 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 1388/2823] 10.222.0.15 () {58 vars in 1233 bytes} [Mon Aug 22 16:15:46 2022] DELETE /identity/v3/projects/21e5a94ced984deea6e3c0a0f08a994b => generated 0 bytes in 608 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 22 16:15:46.620888 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb63b101-5663-4a66-9eef-c7ca089a1daf admin admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 16:15:46.623501 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-bb63b101-5663-4a66-9eef-c7ca089a1daf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 16:15:46.650614 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb63b101-5663-4a66-9eef-c7ca089a1daf admin admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 16:15:46.759814 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-569807d2-be41-4936-b086-8475f4f24968 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a98b1e4431940ab8d1fc95df2858a6c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 16:15:46.760525 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-569807d2-be41-4936-b086-8475f4f24968 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:15:46.760813 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-569807d2-be41-4936-b086-8475f4f24968 admin admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:15:46.761126 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-569807d2-be41-4936-b086-8475f4f24968 admin admin] PATH_INFO: `/v3/users/543789fa6fe44ebf92b4768de626cc38` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:15:46.763224 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-569807d2-be41-4936-b086-8475f4f24968 admin admin] RBAC: Authorizing `identity:delete_user(user_id=543789fa6fe44ebf92b4768de626cc38)` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 16:15:46.765777 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-569807d2-be41-4936-b086-8475f4f24968 admin admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 16:15:46.801511 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 16:15:47.092873 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-569807d2-be41-4936-b086-8475f4f24968 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '543789fa6fe44ebf92b4768de626cc38'} {{(pid=81768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 16:15:47.093875 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-569807d2-be41-4936-b086-8475f4f24968 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '543789fa6fe44ebf92b4768de626cc38'} {{(pid=81768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 16:15:47.094705 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-569807d2-be41-4936-b086-8475f4f24968 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '543789fa6fe44ebf92b4768de626cc38'} {{(pid=81768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 16:15:47.106129 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-569807d2-be41-4936-b086-8475f4f24968 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '543789fa6fe44ebf92b4768de626cc38'} {{(pid=81768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 16:15:47.230109 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-569807d2-be41-4936-b086-8475f4f24968 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '543789fa6fe44ebf92b4768de626cc38'} {{(pid=81768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 16:15:47.268201 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 1436/2824] 10.222.0.15 () {58 vars in 1224 bytes} [Mon Aug 22 16:15:46 2022] DELETE /identity/v3/users/543789fa6fe44ebf92b4768de626cc38 => generated 0 bytes in 648 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 16:15:47.644843 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba21981a-5ef7-4995-afb6-fd8bec0f9238 admin admin] Authenticating user token {{(pid=81767) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 16:15:47.646321 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.fernet_utils [None req-ba21981a-5ef7-4995-afb6-fd8bec0f9238 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81767) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 16:15:47.698041 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba21981a-5ef7-4995-afb6-fd8bec0f9238 admin admin] Validating token access rules against request {{(pid=81767) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 16:15:47.824706 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b23f679f-2122-437d-8dda-e32b205095a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a98b1e4431940ab8d1fc95df2858a6c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81767) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 16:15:47.825465 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b23f679f-2122-437d-8dda-e32b205095a6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:15:47.825829 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b23f679f-2122-437d-8dda-e32b205095a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:15:47.826176 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b23f679f-2122-437d-8dda-e32b205095a6 admin admin] PATH_INFO: `/v3/projects/3f19a65d43ad4b1aac97ce02095dbe21` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:15:47.836112 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b23f679f-2122-437d-8dda-e32b205095a6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3f19a65d43ad4b1aac97ce02095dbe21)` {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 16:15:47.836112 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b23f679f-2122-437d-8dda-e32b205095a6 admin admin] RBAC: Authorization granted {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 16:15:48.024955 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG dbcounter [-] [81767] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81767) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 16:15:48.031924 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.notifications [None req-b23f679f-2122-437d-8dda-e32b205095a6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3f19a65d43ad4b1aac97ce02095dbe21', 'request_id': 'req-b23f679f-2122-437d-8dda-e32b205095a6'} {{(pid=81767) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 16:15:48.043677 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.notifications [None req-b23f679f-2122-437d-8dda-e32b205095a6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3f19a65d43ad4b1aac97ce02095dbe21', 'request_id': 'req-b23f679f-2122-437d-8dda-e32b205095a6'} {{(pid=81767) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 16:15:48.136469 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.notifications [None req-b23f679f-2122-437d-8dda-e32b205095a6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3f19a65d43ad4b1aac97ce02095dbe21', 'request_id': 'req-b23f679f-2122-437d-8dda-e32b205095a6'} {{(pid=81767) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 16:15:48.159638 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.notifications [None req-b23f679f-2122-437d-8dda-e32b205095a6 admin admin] The token cache is being invalidate because project 3f19a65d43ad4b1aac97ce02095dbe21 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81767) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 22 16:15:48.159956 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.notifications [None req-b23f679f-2122-437d-8dda-e32b205095a6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81767) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 16:15:48.162943 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.notifications [None req-b23f679f-2122-437d-8dda-e32b205095a6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81767) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 16:15:48.165188 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 1389/2825] 10.222.0.15 () {58 vars in 1233 bytes} [Mon Aug 22 16:15:47 2022] DELETE /identity/v3/projects/3f19a65d43ad4b1aac97ce02095dbe21 => generated 0 bytes in 521 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 16:15:54.652205 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-569807d2-be41-4936-b086-8475f4f24968 admin admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 16:15:54.655523 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-569807d2-be41-4936-b086-8475f4f24968 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 16:15:54.672399 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-569807d2-be41-4936-b086-8475f4f24968 admin admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 16:15:54.743941 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8f9597a-42d1-48d4-bff9-20f71824fb7f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a98b1e4431940ab8d1fc95df2858a6c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 16:15:54.744761 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8f9597a-42d1-48d4-bff9-20f71824fb7f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:15:54.745096 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8f9597a-42d1-48d4-bff9-20f71824fb7f admin admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:15:54.745436 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8f9597a-42d1-48d4-bff9-20f71824fb7f admin admin] PATH_INFO: `/v3/users/8630362971bf46efae56179cb5c607d4` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:15:54.746628 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8f9597a-42d1-48d4-bff9-20f71824fb7f admin admin] RBAC: Authorizing `identity:delete_user(user_id=8630362971bf46efae56179cb5c607d4)` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 16:15:54.751907 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8f9597a-42d1-48d4-bff9-20f71824fb7f admin admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 16:15:54.989669 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-b8f9597a-42d1-48d4-bff9-20f71824fb7f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8630362971bf46efae56179cb5c607d4'} {{(pid=81768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 16:15:54.990807 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-b8f9597a-42d1-48d4-bff9-20f71824fb7f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8630362971bf46efae56179cb5c607d4'} {{(pid=81768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 16:15:54.991651 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-b8f9597a-42d1-48d4-bff9-20f71824fb7f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8630362971bf46efae56179cb5c607d4'} {{(pid=81768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 16:15:55.002677 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-b8f9597a-42d1-48d4-bff9-20f71824fb7f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8630362971bf46efae56179cb5c607d4'} {{(pid=81768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 16:15:55.111546 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-b8f9597a-42d1-48d4-bff9-20f71824fb7f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8630362971bf46efae56179cb5c607d4'} {{(pid=81768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 16:15:55.138076 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 1437/2826] 10.222.0.15 () {58 vars in 1224 bytes} [Mon Aug 22 16:15:54 2022] DELETE /identity/v3/users/8630362971bf46efae56179cb5c607d4 => generated 0 bytes in 486 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 16:15:55.143303 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab10904-55bc-484b-8822-b2faf200be17 None None] REQUEST_METHOD: `POST` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:15:55.143898 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab10904-55bc-484b-8822-b2faf200be17 None None] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:15:55.144358 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab10904-55bc-484b-8822-b2faf200be17 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:15:55.181391 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.auth.core [None req-dab10904-55bc-484b-8822-b2faf200be17 None None] MFA Rules not processed for user `6b2199ae0fb0480aa917ada391f5ec54`. Rule list: `[]` (Enabled: `True`). {{(pid=81767) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 16:15:55.194695 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.fernet_utils [None req-dab10904-55bc-484b-8822-b2faf200be17 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81767) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 16:15:55.266203 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 1390/2827] 10.222.0.15 () {58 vars in 958 bytes} [Mon Aug 22 16:15:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 16:15:55.276850 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8f9597a-42d1-48d4-bff9-20f71824fb7f admin admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 16:15:55.278694 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-b8f9597a-42d1-48d4-bff9-20f71824fb7f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 16:15:55.310030 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=82,keystone:DELETE=16,keystone:INSERT=2 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 16:15:55.357895 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8f9597a-42d1-48d4-bff9-20f71824fb7f admin admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 16:15:55.364814 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13de56d3-bf8c-4406-aba1-bfafbf8641ea service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '234687adad594f1aa45775a565ffbdc7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4ee0413f9ef4d4fb1f95a660babe090', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 16:15:55.364814 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13de56d3-bf8c-4406-aba1-bfafbf8641ea service neutron] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:15:55.364814 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13de56d3-bf8c-4406-aba1-bfafbf8641ea service neutron] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:15:55.364814 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13de56d3-bf8c-4406-aba1-bfafbf8641ea service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:15:55.381118 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13de56d3-bf8c-4406-aba1-bfafbf8641ea service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 16:15:55.384772 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13de56d3-bf8c-4406-aba1-bfafbf8641ea service neutron] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 16:15:55.400098 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 1438/2828] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Aug 22 16:15:55 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 124 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 16:15:55.943194 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dab10904-55bc-484b-8822-b2faf200be17 None None] Authenticating user token {{(pid=81767) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 16:15:55.947886 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.fernet_utils [None req-dab10904-55bc-484b-8822-b2faf200be17 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81767) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 16:15:55.959264 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dab10904-55bc-484b-8822-b2faf200be17 None None] Validating token access rules against request {{(pid=81767) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 16:15:55.982462 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4066fcd7-f8ed-4135-b02d-4405c3507130 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a98b1e4431940ab8d1fc95df2858a6c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81767) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 16:15:55.982462 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4066fcd7-f8ed-4135-b02d-4405c3507130 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:15:55.982462 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4066fcd7-f8ed-4135-b02d-4405c3507130 admin admin] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:15:55.982462 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4066fcd7-f8ed-4135-b02d-4405c3507130 admin admin] PATH_INFO: `/v3/projects/58437ee91d73418d9d717997c88ac2e1` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:15:55.982857 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4066fcd7-f8ed-4135-b02d-4405c3507130 admin admin] RBAC: Authorizing `identity:delete_project(project_id=58437ee91d73418d9d717997c88ac2e1)` {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 16:15:55.982857 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4066fcd7-f8ed-4135-b02d-4405c3507130 admin admin] RBAC: Authorization granted {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 16:15:56.131552 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.notifications [None req-4066fcd7-f8ed-4135-b02d-4405c3507130 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '58437ee91d73418d9d717997c88ac2e1', 'request_id': 'req-4066fcd7-f8ed-4135-b02d-4405c3507130'} {{(pid=81767) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 16:15:56.131552 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.notifications [None req-4066fcd7-f8ed-4135-b02d-4405c3507130 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '58437ee91d73418d9d717997c88ac2e1', 'request_id': 'req-4066fcd7-f8ed-4135-b02d-4405c3507130'} {{(pid=81767) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 16:15:56.409394 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.notifications [None req-4066fcd7-f8ed-4135-b02d-4405c3507130 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '58437ee91d73418d9d717997c88ac2e1', 'request_id': 'req-4066fcd7-f8ed-4135-b02d-4405c3507130'} {{(pid=81767) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 16:15:56.421254 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.notifications [None req-4066fcd7-f8ed-4135-b02d-4405c3507130 admin admin] The token cache is being invalidate because project 58437ee91d73418d9d717997c88ac2e1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81767) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 22 16:15:56.423310 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.notifications [None req-4066fcd7-f8ed-4135-b02d-4405c3507130 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81767) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 16:15:56.423310 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.notifications [None req-4066fcd7-f8ed-4135-b02d-4405c3507130 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81767) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 16:15:56.423872 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 1391/2829] 10.222.0.15 () {58 vars in 1233 bytes} [Mon Aug 22 16:15:55 2022] DELETE /identity/v3/projects/58437ee91d73418d9d717997c88ac2e1 => generated 0 bytes in 481 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 16:15:57.910237 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9731a5b7-f130-48d7-b8d3-cf85a9469572 None None] REQUEST_METHOD: `POST` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:15:57.910756 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9731a5b7-f130-48d7-b8d3-cf85a9469572 None None] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:15:57.910756 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9731a5b7-f130-48d7-b8d3-cf85a9469572 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:15:57.929903 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.auth.core [None req-9731a5b7-f130-48d7-b8d3-cf85a9469572 None None] MFA Rules not processed for user `6413010933f6416a8d2e3ee82c1946e2`. Rule list: `[]` (Enabled: `True`). {{(pid=81768) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 16:15:57.931084 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-9731a5b7-f130-48d7-b8d3-cf85a9469572 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 16:15:57.989754 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 1439/2830] 10.222.0.15 () {58 vars in 958 bytes} [Mon Aug 22 16:15:57 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 16:15:58.025083 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4066fcd7-f8ed-4135-b02d-4405c3507130 admin admin] Authenticating user token {{(pid=81767) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 16:15:58.026476 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.fernet_utils [None req-4066fcd7-f8ed-4135-b02d-4405c3507130 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81767) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 16:15:58.084334 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4066fcd7-f8ed-4135-b02d-4405c3507130 admin admin] Validating token access rules against request {{(pid=81767) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 16:15:58.088268 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caaa596d-3812-422c-9ef2-7800d86023d7 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91f565473c824a15930e2b12863c1ed3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4ee0413f9ef4d4fb1f95a660babe090', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81767) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 16:15:58.088877 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caaa596d-3812-422c-9ef2-7800d86023d7 service glance] REQUEST_METHOD: `GET` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:15:58.089185 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caaa596d-3812-422c-9ef2-7800d86023d7 service glance] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:15:58.089421 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caaa596d-3812-422c-9ef2-7800d86023d7 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:15:58.091111 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.fernet_utils [None req-caaa596d-3812-422c-9ef2-7800d86023d7 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81767) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 16:15:58.102768 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG dbcounter [-] [81767] Writing DB stats keystone:DELETE=11,keystone:SELECT=87,keystone:INSERT=2 {{(pid=81767) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 16:15:58.164193 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caaa596d-3812-422c-9ef2-7800d86023d7 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 16:15:58.165713 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caaa596d-3812-422c-9ef2-7800d86023d7 service glance] RBAC: Authorization granted {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 16:15:58.177648 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 1392/2831] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Aug 22 16:15:58 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 153 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 16:15:58.402910 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f32a403-ebf7-47c3-8948-123b31177ee5 None None] REQUEST_METHOD: `POST` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:15:58.403240 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f32a403-ebf7-47c3-8948-123b31177ee5 None None] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:15:58.403560 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f32a403-ebf7-47c3-8948-123b31177ee5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:15:58.410397 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application [None req-6f32a403-ebf7-47c3-8948-123b31177ee5 None None] Expecting to find domain in project. The server could not comply with the request since it is either malformed or otherwise incorrect. The client is assumed to be in error.: keystone.exception.ValidationError: Expecting to find domain in project. The server could not comply with the request since it is either malformed or otherwise incorrect. The client is assumed to be in error. Aug 22 16:15:58.410397 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 16:15:58.410397 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 16:15:58.410397 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 16:15:58.410397 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 16:15:58.410397 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 16:15:58.410397 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 16:15:58.410397 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 16:15:58.410397 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 16:15:58.410397 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 16:15:58.410397 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 16:15:58.410397 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 16:15:58.410397 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/server/flask/common.py", line 1064, in wrapper Aug 22 16:15:58.411508 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application return f(*args, **kwargs) Aug 22 16:15:58.411508 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/auth.py", line 315, in post Aug 22 16:15:58.411508 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application token = authentication.authenticate_for_token(auth_data) Aug 22 16:15:58.411508 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/_shared/authentication.py", line 182, in authenticate_for_token Aug 22 16:15:58.411508 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application auth_info = core.AuthInfo.create(auth=auth) Aug 22 16:15:58.411508 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/auth/core.py", line 126, in create Aug 22 16:15:58.411508 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application auth_info._validate_and_normalize_auth_data(scope_only) Aug 22 16:15:58.411508 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/auth/core.py", line 340, in _validate_and_normalize_auth_data Aug 22 16:15:58.411508 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application self._validate_and_normalize_scope_data() Aug 22 16:15:58.411508 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/auth/core.py", line 290, in _validate_and_normalize_scope_data Aug 22 16:15:58.411508 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application project_ref = self._lookup_project(self.auth['scope']['project']) Aug 22 16:15:58.411508 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/auth/core.py", line 194, in _lookup_project Aug 22 16:15:58.411508 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application raise exception.ValidationError(attribute='domain', Aug 22 16:15:58.411508 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application keystone.exception.ValidationError: Expecting to find domain in project. The server could not comply with the request since it is either malformed or otherwise incorrect. The client is assumed to be in error. Aug 22 16:15:58.411508 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: ERROR keystone.server.flask.application Aug 22 16:15:58.412585 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 1440/2832] 10.222.0.15 () {58 vars in 1009 bytes} [Mon Aug 22 16:15:58 2022] POST /identity/v3/auth/tokens => generated 230 bytes in 9 msecs (HTTP/1.1 400) 5 headers in 186 bytes (1 switches on core 0) Aug 22 16:15:58.537265 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53be07cf-2040-4a3d-b0b2-e6748872cffc None None] REQUEST_METHOD: `POST` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:15:58.537553 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53be07cf-2040-4a3d-b0b2-e6748872cffc None None] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:15:58.537829 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53be07cf-2040-4a3d-b0b2-e6748872cffc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:15:58.557451 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.auth.core [None req-53be07cf-2040-4a3d-b0b2-e6748872cffc None None] MFA Rules not processed for user `058e76354dce412cb78b2875c88f2791`. Rule list: `[]` (Enabled: `True`). {{(pid=81767) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 16:15:58.558664 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.fernet_utils [None req-53be07cf-2040-4a3d-b0b2-e6748872cffc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81767) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 16:15:58.615893 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 1393/2833] 10.222.0.15 () {58 vars in 967 bytes} [Mon Aug 22 16:15:58 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 16:15:58.628474 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f32a403-ebf7-47c3-8948-123b31177ee5 None None] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 16:15:58.630172 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-6f32a403-ebf7-47c3-8948-123b31177ee5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 16:15:58.703660 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f32a403-ebf7-47c3-8948-123b31177ee5 None None] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 16:15:58.708873 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a9a885b-9a92-4812-8092-c292b7eea77d service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a09d128dee9438283633e51f4834643', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4ee0413f9ef4d4fb1f95a660babe090', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 16:15:58.709622 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a9a885b-9a92-4812-8092-c292b7eea77d service swift] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:15:58.709906 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a9a885b-9a92-4812-8092-c292b7eea77d service swift] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:15:58.710180 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a9a885b-9a92-4812-8092-c292b7eea77d service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:15:58.718867 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a9a885b-9a92-4812-8092-c292b7eea77d service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 16:15:58.720507 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a9a885b-9a92-4812-8092-c292b7eea77d service swift] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 16:15:58.729015 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 1441/2834] 10.222.0.15 () {60 vars in 1384 bytes} [Mon Aug 22 16:15:58 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 22 16:16:03.491336 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53be07cf-2040-4a3d-b0b2-e6748872cffc None None] Authenticating user token {{(pid=81767) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 16:16:03.491336 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.fernet_utils [None req-53be07cf-2040-4a3d-b0b2-e6748872cffc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81767) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 16:16:03.557545 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53be07cf-2040-4a3d-b0b2-e6748872cffc None None] Validating token access rules against request {{(pid=81767) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 16:16:03.563010 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa8f5600-6ad9-4545-8a88-69c86620aebb service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '08159bf8edda4246bb463f7b11b0894b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4ee0413f9ef4d4fb1f95a660babe090', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81767) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 16:16:03.563655 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa8f5600-6ad9-4545-8a88-69c86620aebb service placement] REQUEST_METHOD: `GET` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:16:03.563966 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa8f5600-6ad9-4545-8a88-69c86620aebb service placement] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:16:03.564244 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa8f5600-6ad9-4545-8a88-69c86620aebb service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:16:03.566017 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.fernet_utils [None req-aa8f5600-6ad9-4545-8a88-69c86620aebb service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81767) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 16:16:03.580560 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa8f5600-6ad9-4545-8a88-69c86620aebb service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 16:16:03.582167 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa8f5600-6ad9-4545-8a88-69c86620aebb service placement] RBAC: Authorization granted {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 16:16:03.593208 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 1394/2835] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Aug 22 16:16:03 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 16:16:04.784633 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a9a885b-9a92-4812-8092-c292b7eea77d service swift] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 16:16:04.786537 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-2a9a885b-9a92-4812-8092-c292b7eea77d service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 16:16:04.855842 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=100 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 16:16:04.860185 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a9a885b-9a92-4812-8092-c292b7eea77d service swift] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 16:16:04.866534 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f3674bc-1e2d-40ee-9715-df93cbfaef61 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a98b1e4431940ab8d1fc95df2858a6c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 16:16:04.867437 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f3674bc-1e2d-40ee-9715-df93cbfaef61 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:16:04.870825 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f3674bc-1e2d-40ee-9715-df93cbfaef61 admin admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:16:04.870825 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f3674bc-1e2d-40ee-9715-df93cbfaef61 admin admin] PATH_INFO: `/v3/users/290b9ef0f970472291c232682e4690f7` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:16:04.870825 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f3674bc-1e2d-40ee-9715-df93cbfaef61 admin admin] RBAC: Authorizing `identity:delete_user(user_id=290b9ef0f970472291c232682e4690f7)` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 16:16:04.871095 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f3674bc-1e2d-40ee-9715-df93cbfaef61 admin admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 16:16:05.074182 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-3f3674bc-1e2d-40ee-9715-df93cbfaef61 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '290b9ef0f970472291c232682e4690f7'} {{(pid=81768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 16:16:05.074969 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-3f3674bc-1e2d-40ee-9715-df93cbfaef61 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '290b9ef0f970472291c232682e4690f7'} {{(pid=81768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 16:16:05.075665 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-3f3674bc-1e2d-40ee-9715-df93cbfaef61 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '290b9ef0f970472291c232682e4690f7'} {{(pid=81768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 16:16:05.083474 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-3f3674bc-1e2d-40ee-9715-df93cbfaef61 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '290b9ef0f970472291c232682e4690f7'} {{(pid=81768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 16:16:05.155137 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-3f3674bc-1e2d-40ee-9715-df93cbfaef61 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '290b9ef0f970472291c232682e4690f7'} {{(pid=81768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 16:16:05.187302 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 1442/2836] 10.222.0.15 () {58 vars in 1224 bytes} [Mon Aug 22 16:16:04 2022] DELETE /identity/v3/users/290b9ef0f970472291c232682e4690f7 => generated 0 bytes in 403 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 16:16:05.192660 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe8992d5-b6e7-4b9e-b091-69b50bbacca4 None None] REQUEST_METHOD: `POST` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:16:05.193254 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe8992d5-b6e7-4b9e-b091-69b50bbacca4 None None] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:16:05.193254 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe8992d5-b6e7-4b9e-b091-69b50bbacca4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:16:05.222297 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.auth.core [None req-fe8992d5-b6e7-4b9e-b091-69b50bbacca4 None None] MFA Rules not processed for user `6b2199ae0fb0480aa917ada391f5ec54`. Rule list: `[]` (Enabled: `True`). {{(pid=81767) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 16:16:05.223739 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.fernet_utils [None req-fe8992d5-b6e7-4b9e-b091-69b50bbacca4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81767) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 16:16:05.260543 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG dbcounter [-] [81767] Writing DB stats keystone:SELECT=100 {{(pid=81767) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 16:16:05.302998 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 1395/2837] 10.222.0.15 () {58 vars in 958 bytes} [Mon Aug 22 16:16:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 22 16:16:05.317556 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f3674bc-1e2d-40ee-9715-df93cbfaef61 admin admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 16:16:05.321509 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-3f3674bc-1e2d-40ee-9715-df93cbfaef61 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 16:16:05.412289 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f3674bc-1e2d-40ee-9715-df93cbfaef61 admin admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 16:16:05.416461 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d16e25e-f326-4f15-b171-1e004182ce0b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '234687adad594f1aa45775a565ffbdc7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4ee0413f9ef4d4fb1f95a660babe090', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 16:16:05.417139 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d16e25e-f326-4f15-b171-1e004182ce0b service neutron] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:16:05.417470 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d16e25e-f326-4f15-b171-1e004182ce0b service neutron] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:16:05.417906 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d16e25e-f326-4f15-b171-1e004182ce0b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:16:05.428310 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d16e25e-f326-4f15-b171-1e004182ce0b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 16:16:05.429852 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d16e25e-f326-4f15-b171-1e004182ce0b service neutron] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 16:16:05.440465 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 1443/2838] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Aug 22 16:16:05 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 124 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 16:16:05.943196 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe8992d5-b6e7-4b9e-b091-69b50bbacca4 None None] Authenticating user token {{(pid=81767) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 16:16:05.944822 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.fernet_utils [None req-fe8992d5-b6e7-4b9e-b091-69b50bbacca4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81767) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 16:16:05.967241 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe8992d5-b6e7-4b9e-b091-69b50bbacca4 None None] Validating token access rules against request {{(pid=81767) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 16:16:05.979844 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-808e1b78-d9bc-44ef-a7ea-256b57efbb0c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a98b1e4431940ab8d1fc95df2858a6c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81767) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 16:16:05.980491 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-808e1b78-d9bc-44ef-a7ea-256b57efbb0c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:16:05.980851 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-808e1b78-d9bc-44ef-a7ea-256b57efbb0c admin admin] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:16:05.981031 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-808e1b78-d9bc-44ef-a7ea-256b57efbb0c admin admin] PATH_INFO: `/v3/projects/28bf590bea514dbcaedcd6c65a23bcf2` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:16:05.982525 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-808e1b78-d9bc-44ef-a7ea-256b57efbb0c admin admin] RBAC: Authorizing `identity:delete_project(project_id=28bf590bea514dbcaedcd6c65a23bcf2)` {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 16:16:05.984472 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-808e1b78-d9bc-44ef-a7ea-256b57efbb0c admin admin] RBAC: Authorization granted {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 16:16:06.122216 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.notifications [None req-808e1b78-d9bc-44ef-a7ea-256b57efbb0c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '28bf590bea514dbcaedcd6c65a23bcf2', 'request_id': 'req-808e1b78-d9bc-44ef-a7ea-256b57efbb0c'} {{(pid=81767) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 16:16:06.131905 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.notifications [None req-808e1b78-d9bc-44ef-a7ea-256b57efbb0c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '28bf590bea514dbcaedcd6c65a23bcf2', 'request_id': 'req-808e1b78-d9bc-44ef-a7ea-256b57efbb0c'} {{(pid=81767) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 16:16:06.223162 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.notifications [None req-808e1b78-d9bc-44ef-a7ea-256b57efbb0c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '28bf590bea514dbcaedcd6c65a23bcf2', 'request_id': 'req-808e1b78-d9bc-44ef-a7ea-256b57efbb0c'} {{(pid=81767) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 16:16:06.248632 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.notifications [None req-808e1b78-d9bc-44ef-a7ea-256b57efbb0c admin admin] The token cache is being invalidate because project 28bf590bea514dbcaedcd6c65a23bcf2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81767) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 22 16:16:06.249268 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.notifications [None req-808e1b78-d9bc-44ef-a7ea-256b57efbb0c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81767) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 16:16:06.250517 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.notifications [None req-808e1b78-d9bc-44ef-a7ea-256b57efbb0c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81767) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 16:16:06.256069 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 1396/2839] 10.222.0.15 () {58 vars in 1233 bytes} [Mon Aug 22 16:16:05 2022] DELETE /identity/v3/projects/28bf590bea514dbcaedcd6c65a23bcf2 => generated 0 bytes in 314 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 16:16:07.401912 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d16e25e-f326-4f15-b171-1e004182ce0b service neutron] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 16:16:07.401912 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-6d16e25e-f326-4f15-b171-1e004182ce0b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 16:16:07.477090 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d16e25e-f326-4f15-b171-1e004182ce0b service neutron] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 16:16:07.499149 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 16:16:07.540553 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8129328c-4ee4-40e1-b7e4-6a5b7a19ffeb service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '08159bf8edda4246bb463f7b11b0894b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4ee0413f9ef4d4fb1f95a660babe090', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 16:16:07.541285 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8129328c-4ee4-40e1-b7e4-6a5b7a19ffeb service placement] REQUEST_METHOD: `GET` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:16:07.541618 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8129328c-4ee4-40e1-b7e4-6a5b7a19ffeb service placement] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:16:07.541750 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8129328c-4ee4-40e1-b7e4-6a5b7a19ffeb service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:16:07.543485 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-8129328c-4ee4-40e1-b7e4-6a5b7a19ffeb service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 16:16:07.558374 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-808e1b78-d9bc-44ef-a7ea-256b57efbb0c admin admin] Authenticating user token {{(pid=81767) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 16:16:07.558870 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8129328c-4ee4-40e1-b7e4-6a5b7a19ffeb service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 16:16:07.579229 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.fernet_utils [None req-808e1b78-d9bc-44ef-a7ea-256b57efbb0c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81767) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 16:16:07.579433 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8129328c-4ee4-40e1-b7e4-6a5b7a19ffeb service placement] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 16:16:07.579433 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 1444/2840] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Aug 22 16:16:07 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 176 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 16:16:07.632169 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-808e1b78-d9bc-44ef-a7ea-256b57efbb0c admin admin] Validating token access rules against request {{(pid=81767) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 16:16:07.636784 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4c13f42-86a3-4d8f-92a3-7349093053f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a98b1e4431940ab8d1fc95df2858a6c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81767) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 16:16:07.637412 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4c13f42-86a3-4d8f-92a3-7349093053f7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:16:07.637685 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4c13f42-86a3-4d8f-92a3-7349093053f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:16:07.637953 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4c13f42-86a3-4d8f-92a3-7349093053f7 admin admin] PATH_INFO: `/v3/users/6413010933f6416a8d2e3ee82c1946e2` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:16:07.639067 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4c13f42-86a3-4d8f-92a3-7349093053f7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6413010933f6416a8d2e3ee82c1946e2)` {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 16:16:07.641060 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4c13f42-86a3-4d8f-92a3-7349093053f7 admin admin] RBAC: Authorization granted {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 16:16:07.903050 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.notifications [None req-e4c13f42-86a3-4d8f-92a3-7349093053f7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6413010933f6416a8d2e3ee82c1946e2'} {{(pid=81767) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 16:16:07.904062 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.notifications [None req-e4c13f42-86a3-4d8f-92a3-7349093053f7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6413010933f6416a8d2e3ee82c1946e2'} {{(pid=81767) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 16:16:08.008697 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.notifications [None req-e4c13f42-86a3-4d8f-92a3-7349093053f7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6413010933f6416a8d2e3ee82c1946e2'} {{(pid=81767) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 16:16:08.014112 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG dbcounter [-] [81767] Writing DB stats keystone:SELECT=83,keystone:DELETE=15,keystone:INSERT=2 {{(pid=81767) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 16:16:08.017163 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.notifications [None req-e4c13f42-86a3-4d8f-92a3-7349093053f7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6413010933f6416a8d2e3ee82c1946e2'} {{(pid=81767) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 16:16:08.029593 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.notifications [None req-e4c13f42-86a3-4d8f-92a3-7349093053f7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6413010933f6416a8d2e3ee82c1946e2'} {{(pid=81767) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 16:16:08.039268 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 1397/2841] 10.222.0.15 () {58 vars in 1224 bytes} [Mon Aug 22 16:16:07 2022] DELETE /identity/v3/users/6413010933f6416a8d2e3ee82c1946e2 => generated 0 bytes in 481 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 22 16:16:08.288332 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8129328c-4ee4-40e1-b7e4-6a5b7a19ffeb service placement] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 16:16:08.301486 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8129328c-4ee4-40e1-b7e4-6a5b7a19ffeb service placement] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 16:16:08.370826 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a038e21c-7595-4509-8007-30b8f9566acb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a98b1e4431940ab8d1fc95df2858a6c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 16:16:08.371503 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a038e21c-7595-4509-8007-30b8f9566acb admin admin] REQUEST_METHOD: `DELETE` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:16:08.371784 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a038e21c-7595-4509-8007-30b8f9566acb admin admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:16:08.372055 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a038e21c-7595-4509-8007-30b8f9566acb admin admin] PATH_INFO: `/v3/projects/dd773e1588cb44459ec79f593c35a860` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:16:08.373319 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a038e21c-7595-4509-8007-30b8f9566acb admin admin] RBAC: Authorizing `identity:delete_project(project_id=dd773e1588cb44459ec79f593c35a860)` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 16:16:08.375297 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a038e21c-7595-4509-8007-30b8f9566acb admin admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 16:16:08.529352 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-a038e21c-7595-4509-8007-30b8f9566acb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'dd773e1588cb44459ec79f593c35a860', 'request_id': 'req-a038e21c-7595-4509-8007-30b8f9566acb'} {{(pid=81768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 16:16:08.535101 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-a038e21c-7595-4509-8007-30b8f9566acb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'dd773e1588cb44459ec79f593c35a860', 'request_id': 'req-a038e21c-7595-4509-8007-30b8f9566acb'} {{(pid=81768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 16:16:08.605993 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-a038e21c-7595-4509-8007-30b8f9566acb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'dd773e1588cb44459ec79f593c35a860', 'request_id': 'req-a038e21c-7595-4509-8007-30b8f9566acb'} {{(pid=81768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 16:16:08.618686 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-a038e21c-7595-4509-8007-30b8f9566acb admin admin] The token cache is being invalidate because project dd773e1588cb44459ec79f593c35a860 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81768) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 22 16:16:08.618974 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-a038e21c-7595-4509-8007-30b8f9566acb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 16:16:08.619670 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-a038e21c-7595-4509-8007-30b8f9566acb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 16:16:08.621205 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 1445/2842] 10.222.0.15 () {58 vars in 1233 bytes} [Mon Aug 22 16:16:08 2022] DELETE /identity/v3/projects/dd773e1588cb44459ec79f593c35a860 => generated 0 bytes in 333 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 16:16:08.625389 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4c13f42-86a3-4d8f-92a3-7349093053f7 admin admin] Authenticating user token {{(pid=81767) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 16:16:08.627333 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.fernet_utils [None req-e4c13f42-86a3-4d8f-92a3-7349093053f7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81767) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 16:16:08.640354 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4c13f42-86a3-4d8f-92a3-7349093053f7 admin admin] Validating token access rules against request {{(pid=81767) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 16:16:08.697182 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf2b602c-4513-4d8f-9f24-074df2c5a8d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a98b1e4431940ab8d1fc95df2858a6c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81767) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 16:16:08.697816 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf2b602c-4513-4d8f-9f24-074df2c5a8d7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:16:08.698108 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf2b602c-4513-4d8f-9f24-074df2c5a8d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:16:08.698476 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf2b602c-4513-4d8f-9f24-074df2c5a8d7 admin admin] PATH_INFO: `/v3/users/a73acaa341eb4f538e81c4aa6b3034cf` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:16:08.699601 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf2b602c-4513-4d8f-9f24-074df2c5a8d7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a73acaa341eb4f538e81c4aa6b3034cf)` {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 16:16:08.701511 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf2b602c-4513-4d8f-9f24-074df2c5a8d7 admin admin] RBAC: Authorization granted {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 16:16:08.901044 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.notifications [None req-cf2b602c-4513-4d8f-9f24-074df2c5a8d7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a73acaa341eb4f538e81c4aa6b3034cf'} {{(pid=81767) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 16:16:08.901644 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.notifications [None req-cf2b602c-4513-4d8f-9f24-074df2c5a8d7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a73acaa341eb4f538e81c4aa6b3034cf'} {{(pid=81767) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 16:16:08.985285 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.notifications [None req-cf2b602c-4513-4d8f-9f24-074df2c5a8d7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a73acaa341eb4f538e81c4aa6b3034cf'} {{(pid=81767) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 16:16:08.993373 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.notifications [None req-cf2b602c-4513-4d8f-9f24-074df2c5a8d7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a73acaa341eb4f538e81c4aa6b3034cf'} {{(pid=81767) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 16:16:09.005798 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.notifications [None req-cf2b602c-4513-4d8f-9f24-074df2c5a8d7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a73acaa341eb4f538e81c4aa6b3034cf'} {{(pid=81767) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 16:16:09.012380 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 1398/2843] 10.222.0.15 () {58 vars in 1224 bytes} [Mon Aug 22 16:16:08 2022] DELETE /identity/v3/users/a73acaa341eb4f538e81c4aa6b3034cf => generated 0 bytes in 387 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 16:16:09.275939 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a038e21c-7595-4509-8007-30b8f9566acb admin admin] Authenticating user token {{(pid=81768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 16:16:09.277422 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.fernet_utils [None req-a038e21c-7595-4509-8007-30b8f9566acb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 16:16:09.293162 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a038e21c-7595-4509-8007-30b8f9566acb admin admin] Validating token access rules against request {{(pid=81768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 16:16:09.338757 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 16:16:09.356078 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-397b1d3f-a2a5-4225-99b8-34d62cc4d3b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b2199ae0fb0480aa917ada391f5ec54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a98b1e4431940ab8d1fc95df2858a6c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 16:16:09.356942 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-397b1d3f-a2a5-4225-99b8-34d62cc4d3b7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:16:09.357400 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-397b1d3f-a2a5-4225-99b8-34d62cc4d3b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:16:09.357823 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-397b1d3f-a2a5-4225-99b8-34d62cc4d3b7 admin admin] PATH_INFO: `/v3/projects/f8def62fc2424a13a013f65f44bd4f5f` {{(pid=81768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:16:09.359519 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-397b1d3f-a2a5-4225-99b8-34d62cc4d3b7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f8def62fc2424a13a013f65f44bd4f5f)` {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 16:16:09.362796 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-397b1d3f-a2a5-4225-99b8-34d62cc4d3b7 admin admin] RBAC: Authorization granted {{(pid=81768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 16:16:09.479840 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-397b1d3f-a2a5-4225-99b8-34d62cc4d3b7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f8def62fc2424a13a013f65f44bd4f5f', 'request_id': 'req-397b1d3f-a2a5-4225-99b8-34d62cc4d3b7'} {{(pid=81768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 16:16:09.485783 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-397b1d3f-a2a5-4225-99b8-34d62cc4d3b7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f8def62fc2424a13a013f65f44bd4f5f', 'request_id': 'req-397b1d3f-a2a5-4225-99b8-34d62cc4d3b7'} {{(pid=81768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 16:16:09.565880 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-397b1d3f-a2a5-4225-99b8-34d62cc4d3b7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f8def62fc2424a13a013f65f44bd4f5f', 'request_id': 'req-397b1d3f-a2a5-4225-99b8-34d62cc4d3b7'} {{(pid=81768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 16:16:09.578523 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-397b1d3f-a2a5-4225-99b8-34d62cc4d3b7 admin admin] The token cache is being invalidate because project f8def62fc2424a13a013f65f44bd4f5f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81768) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 22 16:16:09.578935 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-397b1d3f-a2a5-4225-99b8-34d62cc4d3b7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 16:16:09.579732 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG keystone.notifications [None req-397b1d3f-a2a5-4225-99b8-34d62cc4d3b7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 16:16:09.581410 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: [pid: 81768|app: 0|req: 1446/2844] 10.222.0.15 () {58 vars in 1233 bytes} [Mon Aug 22 16:16:09 2022] DELETE /identity/v3/projects/f8def62fc2424a13a013f65f44bd4f5f => generated 0 bytes in 306 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 22 16:16:19.008266 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG dbcounter [-] [81767] Writing DB stats keystone:SELECT=62,keystone:DELETE=12,keystone:INSERT=1 {{(pid=81767) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 16:16:19.575123 dv-cinder-842541-5-iscsi devstack@keystone.service[81768]: DEBUG dbcounter [-] [81768] Writing DB stats keystone:SELECT=32,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 16:18:51.541086 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf2b602c-4513-4d8f-9f24-074df2c5a8d7 admin admin] Authenticating user token {{(pid=81767) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 16:18:51.543089 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.fernet_utils [None req-cf2b602c-4513-4d8f-9f24-074df2c5a8d7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81767) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 16:18:51.696209 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf2b602c-4513-4d8f-9f24-074df2c5a8d7 admin admin] Validating token access rules against request {{(pid=81767) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 16:18:51.759942 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d65ad5a-719d-4e1a-8ab5-efe94197b97d service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '08159bf8edda4246bb463f7b11b0894b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4ee0413f9ef4d4fb1f95a660babe090', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81767) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 16:18:51.760556 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d65ad5a-719d-4e1a-8ab5-efe94197b97d service placement] REQUEST_METHOD: `GET` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 16:18:51.760839 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d65ad5a-719d-4e1a-8ab5-efe94197b97d service placement] SCRIPT_NAME: `/identity` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 16:18:51.761112 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d65ad5a-719d-4e1a-8ab5-efe94197b97d service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81767) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 16:18:51.763499 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.fernet_utils [None req-8d65ad5a-719d-4e1a-8ab5-efe94197b97d service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81767) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 16:18:51.775323 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d65ad5a-719d-4e1a-8ab5-efe94197b97d service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 16:18:51.776960 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d65ad5a-719d-4e1a-8ab5-efe94197b97d service placement] RBAC: Authorization granted {{(pid=81767) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 16:18:51.787600 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: [pid: 81767|app: 0|req: 1399/2845] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Aug 22 16:18:51 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 247 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 16:19:01.783119 dv-cinder-842541-5-iscsi devstack@keystone.service[81767]: DEBUG dbcounter [-] [81767] Writing DB stats keystone:SELECT=89 {{(pid=81767) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}}